ARTICLE
The API Gateway Pattern
From Microservices Patterns by Chris Richardson
___________________________________________________________________
Save 37% on Microservice Patterns.
Just enter code fccrichardson into the discount code box at checkout at manning.com.
___________________________________________________________________
In this article, I describe the API gateway pattern. I discuss its benefits and drawbacks. I also describe various design issues that you must address when developing an API gateway.
As you may know, there are numerous drawbacks with services accessing services directly. It’s often impractical for a client to perform API composition over the Internet. The lack of encapsulation makes it difficult for developers to change service decomposition and APIs. Services sometimes use communication protocols that aren’t suitable outside of the firewall. Consequently, a much better approach is to use what’s known as an API gateway.
An API gateway is a service which is the entry point into the application from the outside world. It’s responsible for request routing, API composition, and other functions, such as authentication. Let’s take a look at the API gateway pattern.
Overview of the API gateway pattern
The drawbacks of clients making multiple requests in order to display information to the user are well known (I describe them in my book!). A much better approach is for a client to make a single request to what’s known as an API gateway. An API gateway is a service which is the single entry-point for API requests into an application from outside the firewall. It’s similar to the Facade pattern from object-oriented design. Like a facade, an API gateway encapsulates the application’s internal architecture and provides an API to its clients. It might also have other responsibilities, such authentication, monitoring, and rate limiting. Figure 1 shows the relationship between the clients, the API gateway, and the services.
The API gateway is responsible for request routing, API composition, and protocol translation. All API requests from external clients first go to the API gateway. The API gateway routes some requests to the appropriate service. The gateway handles other requests using the API Composition pattern and invoking multiple services and aggregating the results. It might also translate between client-friendly protocols such as HTTP and WebSockets and client-unfriendly protocols which are used by the services.
Request routing
One of the key functions of an API gateway is request routing. An API gateway implements some API operations by routing requests to the corresponding service. When it receives a request, the API gateway consults a routing map that specifies which service to route the request to. A routing map might, for example, map an HTTP method and path to the HTTP URL of a service. This function is identical to the reverse proxying features provided by web servers such as NGINX.
API Composition
An API gateway typically does more than reverse proxying. It might also implement some API operations using API composition. In my book, I use an example application called FTGO to teach how to move from monolithic architecture to microservices. The FTGO API gateway, for example, implements the Get Order Details API operation using API composition. As figure 2 shows, the mobile application makes one request to the API gateway, which fetches the order details from multiple services.
The FTGO API gateway provides a coarse-grained API that enables mobile clients to retrieve the data they need with a single request. For example, the mobile client makes a single getOrderDetails() request to the API gateway.
Protocol translation
An API gateway might also perform protocol translation. The API gateway might provide a RESTful API to external clients even though the application services use a mixture of protocols internally include REST and gRPC. When needed, the implementation of some API operations translate between the RESTful external API and the internal gRPC-based APIs.
The API gateway provides each client with client-specific API
An API gateway could provide a single one-size-fits-all (OSFA) API. The problem with a single API is that different clients often have different requirements. For instance, a third-party application might require the Get Order Details API operation to return the complete Order details, while a mobile client only needs a subset of the data. One way to solve this problem is to give clients the option of specifying in a request which fields and related objects the server should return. This approach is adequate for a public API that must serve a broad range of third-party application, but it often doesn’t give clients the control they need.
A better approach is for the API gateway to provide each client with its own API. For example, the FTGO API gateway can implement provide the FTGO mobile client with an API which is specifically designed to meet its requirements. It might even have different APIs for the Android and iPhone mobile applications. The API gateway also implements a public API for third-party developers to use.
Implementing edge functions
Although an API gateway’s primary responsibilities are API routing and composition, it may also implement what are known as edge functions. An edge function is, as the name suggests, a request processing function which is implemented at the edge of an application.
Examples of edge functions that an application might implement include:
- authentication — verifying the identity of the client making the request
- authorization — verifying that the client is authorized to perform that particular operation
- rate limiting — limiting how many requests per second are allowed from either a specific client and/or from all clients
- caching — cache responses to reduce the number of requests made to the services
- metrics collection — collect metrics on API usage for billing analytics purposes
- request logging — log requests
Three different places in your application allow you to implement these edge functions.
The first option is to implement these edge functions in the backend services. This might make sense for some functions such as caching, metrics collection, and possibly authorization, but it’s generally more secure if the application authenticates requests on the edge before they reach the services.
The second option is to implement these edge functions in an edge service which is upstream from the API gateway. The edge service is the first point of contact for an external client. It authenticates the request and performs other edge processing before passing it to the API gateway.
An important benefit of using a dedicated edge service is that it separates concerns. The API gateway focusses on API routing and composition. Another benefit is that it centralizes responsibility for critical edge functions such as authentication. This particularly valuable when, as I describe below, an application has multiple API gateways that are possibly written using a variety of languages and frameworks. The drawback of this approach is that increases network latency because of the extra hop. It also adds to the complexity of the application.
As a result, it’s often convenient to use the third option and implement these edge functions, like authorization, in the API gateway itself. One less network hop exists, improving latency. Fewer moving parts reduces complexity.
API gateway architecture
An API gateway has a layered, modular architecture. Its architecture, which is shown in figure 3, consists of two layers, the API layer and a common layer. The API layer consists of one or more independent API modules. Each API module implements an API for a particular client. The common layer implements shared functionality including edge functions such as authentication.
In this example, the API gateway has three API modules:
- Mobile API, which implements the API for the FTGO mobile client
- Browser API, which implements the API to the JavaScript application running in the browser
- Public API, which implements the API for third-party developers.
An API module implements each API operation in one of two ways. Some API operations map directly to single service API operation. An API module implements these operation by routing requests to the corresponding service API operation. It might implement these API operations using a generic routing module that reads a configuration file describing the routing rules.
An API module implements other, more complex API operations using API composition. The implementation of this API operation consist of custom code. Each API operation implementation handles requests by invoking multiple services and combining the results.
API gateway ownership model
An important question that you must answer is who is responsible for the development of the API gateway and its operation? One option is for a separate team to be responsible for the API gateway. The drawback of this option is that it’s similar to SOA, where an Enterprise Service Bus (ESB) team was responsible for all ESB development. If a developer working on the mobile application needs access to a particular service, they must submit a request to the API gateway team and wait for them to expose the API. This kind of centralized bottleneck in the organization is counter to the philosophy of the microservice architecture, which promotes loosely-coupled autonomous teams.
A better approach, which has been promoted by Netflix, is for the client teams — the mobile, web, and public API teams — to own the API module that exposes their API. An API gateway team is responsible for developing the Common module and for the operational aspects of the gateway. This ownership model, which is shown in figure 4, gives the teams control over their APIs.
When a team needs to change their API, they check in the changes to the source repository for the API gateway. In order to work well, the API gateway’s deployment pipeline must be fully automated. Otherwise, the client teams are often blocked waiting for the API gateway team to deploy the new version.
Using the Backends for front ends pattern
One concern with an API gateway is that responsibility for it is blurred. Multiple teams contribute to the same code base. An API gateway team is responsible for its operation. Although not as bad as an SOA ESB, this blurring of responsibilities is counter to the microservice architecture philosophy of “if you build, it you own it.”
The solution is to have an API gateway for each client, the Backends for front ends (BFF) pattern, which was pioneered by Phil Calçado (http://philcalcado.com/) and his colleagues at SoundCloud. As figure 5 shows, each API module becomes its own standalone API gateway which is developed and operated by a single client team.
The Public API team owns and operates their API gateway, the mobile team owns and operates theirs, and so on. In theory, different API gateways could be developed using different technology stacks, but this risks duplicating code for common functionality such as the code that implements edge functions. Ideally, all API gateways use the same technology stack. The common functionality is a shared library implemented by the API gateway team.
As well as clearly defining responsibilities, the BFF pattern has other benefits. The API modules are isolated from one another, which improves reliability. One misbehaving API can’t readily impact other APIs. The BFF pattern also improves observability because different API modules are different processes. Another benefit of the BFF pattern is that each API is independently scalable. And the BFF pattern reduces startup time, because each API gateway is a smaller, simpler application.
Benefits and drawbacks of an API gateway
As you might expect, the API gateway pattern has both benefits and drawbacks.
Benefits of an API gateway
A major benefit of using an API gateway is that it encapsulates the internal structure of the application. Rather than having to invoke specific services, clients talk to the gateway. The API gateway provides each client with a client-specific API. This reduces the number of round trips between the client and application. It also simplifies the client code.
Drawbacks of an API gateway
The API gateway pattern also has some drawbacks. It’s yet another highly available component that must be developed, deployed, and managed. This creates the risk that the API gateway becomes a development bottleneck. Developers must update the API gateway in order to expose their services’ API. It’s important that the process for updating the API gateway is as lightweight as possible. Otherwise, developers are forced to wait in line in order to update the gateway.
Despite these drawbacks, for most real-world applications it makes sense to use an API gateway. If necessary, you can use the Backends for front ends patterns to enable the teams to develop and deploy their APIs independently.
Netflix as an example of an API gateway
A great example of an API gateway is the Netflix API. The Netflix streaming service is available on hundreds of different kinds of devices, including televisions, Blu-ray players, smart phones, and so on. Initially, Netflix attempted to have a one-size-fits-all style API for their streaming service (http://www.programmableweb.com/news/why-rest-keeps-me-night/2012/05/15). Unfortunately, they soon discovered that it didn’t work well because of the diverse range of devices and their different needs. Today, they use an API gateway that implements a separate API for each device. The client device team develops and owns the API implementation.
In the first version of the Netflix API gateway, each client team implemented their API using Groovy scripts that perform routing and API composition. Each script invoked one or more service APIs using Java client libraries provided by the service teams. On the one hand, this works well and client developers have written over thousands of scripts. The Netflix API gateway handles billions of requests per day; on average, each API calls fans out to six to seven backend services. Netflix has found this monolithic architecture to be somewhat cumbersome.
As a result, Netflix is now moving to an API gateway architecture which is similar to the Backends for front-end patterns. In this new architecture, client teams write API modules using NodeJS. Each API module runs its own Docker container. The scripts don’t invoke the services directly. Instead, they invoke a second “API gateway,” which exposes the service APIs using Netflix Falcor. Netflix Falcor is an API technology which does declarative, dynamic API composition, and enables a client to invoke multiple services using a single request. This new architecture has a number of benefits. The API modules are isolated from one another, which improves reliability and observability. Also, the client API module is independently scalable.
Now you know all about the API gateway pattern.
___________________________________________________________________
If you want to learn more about the book, read the free first chapter of Microservice Patterns on liveBook here.
___________________________________________________________________
About the author:
Chris Richardson is a developer and architect. He is a Java Champion, a JavaOne rock star and the author of POJOs in Action, which describes how to build enterprise Java applications with frameworks such as Spring and Hibernate. Chris was also the founder of the original CloudFoundry.com, an early Java PaaS for Amazon EC2. Today, he is a recognized thought leader in microservices. Chris is the creator of http://microservices.io , a website describing how to develop and deploy microservices. He provides microservices consulting and training and is working on his third startup http://eventuate.io , an application platform for developing microservices.
Originally published at freecontent.manning.com.
