Hi!
My name is Sergey, I work as an infrastructure engineer in the tinkoff.ru platform API team.
In this article, I will talk about the problems that our team faced when preparing balancers based on for various projects. I will also tell you about the tool that allowed me to overcome most of them.
Nginx is a multifunctional and actively developing proxy server. It features a large number of modules, . Each project imposes certain requirements on the functionality of the balancer and Nginx version (for example, the presence of http/2 and grpc proxying), and the composition of its modules.
We want to see a fresh version with the right set of modules running on a specific Linux distribution. In our case, these are deb- and rpm-based systems. The option with containers is not considered in this article.
We want to quickly change the functionality of our balancers. And here the question immediately arises - how to achieve this, spending as few resources as possible? And it would be even better to set up the process so that we can set a finite number of input parameters, and at the output we get an artifact in the form of a deb / rpm package for the desired OS.
As a result, a number of problems can be formulated:
- There are not always packages with a fresh version of Nginx.
- There are no packages with the required modules.
- Compiling and building a package by hand is time-consuming and tedious.
- There is no description of how a particular Nginx instance is built.
To solve these problems, a certain tool suggests itself that would take a specification in a human-readable format as input and build an Nginx package with the necessary functionality based on it.
Having not found a suitable option for us in the vastness of the github, we decided to create our own tool - .
Specifications
In our tool, we wanted to create a description of the specification in the form of code, which can then be put into a Git repository. To do this, we chose the usual format for such things - yaml. Specification example:
nginx_version: 1.14.1
output_package: deb
modules:
- module:
name: nginx-auth-ldap
git_url: https://github.com/kvspb/nginx-auth-ldap.git
git_branch: master
dependencies:
- libldap2-dev
- module:
name: ngx_http_substitutions_filter_module
git_url: https://github.com/yaoweibin/ngx_http_substitutions_filter_module.git
- module:
name: headers-more-nginx-module
web_url: https://github.com/openresty/headers-more-nginx-module/archive/v0.261.zip
- module:
name: nginx-module-vts
git_url: https://github.com/vozlt/nginx-module-vts.git
git_tag: v0.1.18
- module:
name: ngx_devel_kit
git_url: https://github.com/simplresty/ngx_devel_kit.git
git_tag: v0.3.0
- module:
name: ngx_cache_purge
git_url: https://github.com/FRiCKLE/ngx_cache_purge.git
- module:
name: ngx_http_dyups_module
git_url: https://github.com/yzprofile/ngx_http_dyups_module.git
- module:
name: nginx-brotli
git_url: https://github.com/eustas/ngx_brotli.git
git_tag: v0.1.2
- module:
name: nginx_upstream_check_module
git_url: https://github.com/yaoweibin/nginx_upstream_check_module.git
- module:
name: njs
git_url: https://github.com/nginx/njs.git
git_tag: 0.2.5
config_folder_path: nginx
Here we indicate that we want to see a deb package with Nginx version 1.14.2 with the required set of modules. The section with modules is optional. For each of them, you can set:
- Title.
- Address where you can get it:
- Git repository. You can also specify a branch or tag.
- Web link to the archive.
- Local link to the archive.
Some modules require additional dependencies to be installed, for example nginx-auth-ldap requires libldap2-dev to be installed. Necessary dependencies can also be specified when describing the module.
Environment
In our tool, you can quickly get an environment with installed utilities for compiling, building a package, and other auxiliary software. A docker container with everything you need is the best fit here (there are already a couple of examples of docker files for ubuntu and centos in the repository).
After the specification is compiled and the environment is prepared, we run our builder, having previously installed its dependencies:
pip3 install -r requirements.txt
./main.py build -f [ΠΊΠΎΠ½ΡΠΈΠ³_ΡΠ°ΠΉΠ»].yaml -r [Π½ΠΎΠΌΠ΅Ρ_ΡΠ΅Π²ΠΈΠ·ΠΈΠΈ]
The revision number is optional here and is used to version builds. It is written to the meta information of the package, which makes it easy to update it on servers.
You can see what's going on in the logs. Here is an example of the highlights:
builder - INFO - Parse yaml file: example.config.yaml
builder - INFO - Download scripts for build deb package
builder - INFO - Downloading nginx src...
builder - INFO - --> http://nginx.org/download/nginx-1.14.1.tar.gz
builder - INFO - Downloading 3d-party modules...
builder - INFO - Module nginx-auth-ldap will download by branch
builder - INFO - -- Done: nginx-auth-ldap
builder - INFO - -- Done: ngx_http_substitutions_filter_module
builder - INFO - Module headers-more-nginx-module will downloading
builder - INFO - Module nginx-module-vts will download by tag
builder - INFO - -- Done: nginx-module-vts
builder - INFO - Module ngx_devel_kit will download by tag
builder - INFO - -- Done: ngx_devel_kit
builder - INFO - -- Done: ngx_cache_purge
builder - INFO - -- Done: ngx_http_dyups_module
builder - INFO - Downloading dependencies
builder - INFO - Building .deb package
builder - INFO - Running 'dh_make'...
builder - INFO - Running 'dpkg-buildpackage'...
dpkg-deb: building package 'nginx' in '../nginx_1.14.1-1_amd64.deb'.
So, with just a couple of commands, we create an environment and the necessary Nginx assembly, and the package appears in the directory where the script is launched from.
Embedding
We can also embed our tool in CI/CD processes. Any of the many CI systems that exist today can help with this, for example or .
As a result, every time the specification changes in the Git repository, the build of the artifact is automatically started. The revision number is tied to the build run counter.
With a little more time, you can set up pushing the artifact to your local package repository, Nexus, Artifactory, and so on.
An additional advantage is that the yaml configuration file can be connected to Ansible or another automatic configuration system, and take from there the version number and package type that we want to deploy.
What's next
The project has not been completed yet. Here's what we're working on right now:
- We expand the possibility of configuring, but at the same time we keep it as simple as possible. You don't want to define a thousand parameters if you only need two, and the rest is fine by default. This includes compilation flags (now you can change them in the internal configuration file src / config.py), installation paths, user to run.
- We add options for automatically sending the package to various artifact stores.
- Executing a custom command when loading a module (for example, to use you must first apply a patch of a specific version)
- Adding tests:
- The package installs correctly.
- Nginx is up to date and built with the required flags and modules.
- The necessary paths, accounts, and so on are created.
But you can use this tool now, as well as suggest improvements - wellcome!
Source: habr.com