Some companies, including our customer, develop the product through a partner network. For example, large online stores are integrated with the delivery service - you order a product and soon receive a tracking number for the package. Another example is when you buy insurance or an Aeroexpress ticket along with your air ticket.
For this, one API is used, which must be issued to partners through the API Gateway. We have solved this problem. In this article we will tell you the details.
Given: an ecosystem and an API portal with an interface where users are registered, receive information, etc. We need to make a convenient and reliable API Gateway. In the process, we needed to provide
- registration,
- API connection control,
- monitoring how users use the end system,
- accounting for business indicators.
In the article we will talk about our experience in creating API Gateway, during which we solved the following tasks:
- user authentication,
- user authorization,
- modification of the original request,
- request proxying,
- response postprocessing.
There are two types of API management:
1. Standard, which works as follows. Before connecting, the user tests the features, then pays and embeds on his site. Most often they are used in small and medium-sized businesses.
2. Large B2B API Management, when the company first makes a business decision to connect, becomes a partner of the company with a contractual obligation, and then connects to the API. And after all the formalities are settled, the company receives test access, passes testing and goes into production. But this is not possible without a management decision to connect.
Our solution
In this part, we will talk about creating an API Gateway.
The end users of the created API gateway are partners of our customer. We already have the necessary contracts for each of them. We will only need to extend the functionality by marking the granted access to the gateway. Accordingly, a controlled connection and management process is needed.
Of course, it was possible to take some ready-made solution for solving the problem of API Management and creating an API Gateway in particular. For example, this could be. It did not suit us, because in our case we already had an API portal and a huge ecosystem built around it. All users have already been registered, they already understood where and how they can get the necessary information. The necessary interfaces already existed in the API portal, we only needed the API Gateway. Actually, we are engaged in its development.
What we call the API Gateway is a kind of proxy. Here we again had a choice - you can write your own proxy, or you can choose something ready-made. In this case, we went the second way and chose the nginx + Lua bundle. Why? We needed reliable, tested software that supports scaling. After the implementation, we did not want to check both the correctness of the business logic and the correctness of the proxy.
Every web server has a request processing pipeline. In the case of nginx, it looks like this:
(diagram from )
Our goal was to fit into this pipeline at a point where we can modify the original request.
We want to create a transparent proxy so that the request remains functionally the same as it came. We only control access to the final API, help the request get to it. In case the request was incorrect, the final API should show the error, but not us. The only reason we can deny a request is because the client doesn't have access.
Already exists for nginx + . Lua is a scripting language, it is very lightweight and easy to learn. Thus, we implemented the necessary logic using Lua.
The nginx configuration (an analogy to the route of the application), where all the work is done, is quite understandable. Noteworthy here is the last directive - post_action.
location /middleware {
more_clear_input_headers Accept-Encoding;
lua_need_request_body on;
rewrite_by_lua_file 'middleware/rewrite.lua';
access_by_lua_file 'middleware/access.lua';
proxy_pass https://someurl.com;
body_filter_by_lua_file 'middleware/body_filter.lua';
post_action /process_session;
}
Consider what happens in this configuration:
more_clear_input_headers — clears the value of the headers specified after the directive.
lua_need_request_body - controls whether the original request body should be read before executing the rewrite/access/access_by_lua directives or not. By default, nginx does not read the body of a client request, and if you need to access it, then this directive should be set to on.
rewrite_by_lua_file - the path to the script, which describes the logic for modifying the request
access_by_lua_file — the path to the script, which describes the logic that checks for access to the resource.
proxy_pass — url to which the request will be proxied.
body_filter_by_lua_file — the path to the script, which describes the logic for filtering the request before returning it to the client.
Finally, post_action - an officially undocumented directive with which you can perform some other actions after the response is given to the client.
Next, we will describe in order how we solved our problems.
Authorization/authentication and request modification
Authorization
We built authorization and authentication using certificate access. There is a root certificate. Each new client of the customer is generated his personal certificate, with which he can access the API. This certificate is configured in the server section of the nginx settings.
ssl on;
ssl_certificate /usr/local/openresty/nginx/ssl/cert.pem;
ssl_certificate_key /usr/local/openresty/nginx/ssl/cert.pem;
ssl_client_certificate /usr/local/openresty/nginx/ssl/ca.crt;
ssl_verify_client on;Modification
A fair question may arise: what to do with a certified client if we suddenly want to disconnect it from the system? Do not reissue certificates for all other clients.
So we smoothly approached the next task - modifying the original request. The initial request of the client, generally speaking, is not valid for the final system. One of the tasks is to add the missing parts to the request to make it valid. The point is that the missing data is different for each client. We know that a client comes to us with a certificate from which we can take a fingerprint and extract the necessary client data from the database.
If at some point you need to disconnect the client from our service, his data will disappear from the database and he will not be able to do anything.
Working with customer data
We needed to make the solution highly available, especially how we get customer data. The difficulty is that the primary source of this data is a third-party service that does not guarantee uninterrupted and sufficiently high speed.
Therefore, we needed to ensure high availability of customer data. As a tool, we have chosen which provides us with:
- quick access to data
- the ability to organize a cluster of several nodes with data replicated on different nodes.
We went with the simplest strategy for delivering data to the cache:
Work with the end system takes place within sessions and there is a limit on the maximum number. If the client has not closed the session, we will have to do this.
The open session data comes from the end system and is initially processed on the Lua side. We decided to use Hazelcast to store this data with a .NET job. Then, at some intervals, we check the right to life of open sessions and close the rotten ones.
Accessing Hazelcast from both Lua and .NET
There are no Lua clients to work with Hazelcast, but Hazelcast has a REST API, which we decided to use. For .NET there is , through which we planned to access Hazelcast data on the .NET side. But it was not there.
When storing data via REST and retrieving data via a .NET client, different serializers and deserializers are used. Therefore, it is impossible to put data through REST, but get it through the .NET client and vice versa.
If there are interested, we will tell you more about this problem in a separate article. Spoiler - on the schematic.
Logging and monitoring
Our corporate standard for logging through .NET is Serilog, all logs end up in Elasticsearch, and we analyze them through Kibana. I would like to do something similar in this case. The only one to work with Elastic on Lua, which was found, broke on the very first require. And we used Fluentd.
- open source solution for providing a single layer of application logging. Allows you to collect logs from different layers of the application, and then broadcast them into a single source.
API Gateway works in K8S, so we decided to add a container with fluentd in the same pody in order to write logs to the existing open tcp port fluentd.
We also explored how fluentd would behave if it had no connection to Elasticsearch. For two days, requests were continuously sent to the gateway, logs were sent to fluentd, but fluentd was banned from IP Elastic. After the connection was restored, fluentd perfectly overtook absolutely all the logs in Elastic.
Conclusion
The chosen approach to implementation allowed us to deliver a really working product to the combat environment in just 2.5 months.
If someday you happen to do such things, we advise you to first of all clearly understand what problem you are solving and what resources you already have. Be aware of the complexities of integrating with existing API management systems.
Understand for yourself what exactly you are going to develop - only the business logic for processing requests, or, as it could be in our case, the entire proxy. Do not forget that everything you do yourself must be thoroughly tested afterwards.
Source: habr.com
