CD setup via gitlab

I once thought about automating the deployment of my project. gitlab.com kindly provides all the tools for this, and of course I decided to use it by figuring it out and writing a small deployment script. In this article, I share my experience with the community.

TL;DR

1. Set up VPS: disable root, password login, install dockerd, configure ufw
2. Generate certificates for server and client docs.docker.com/engine/security/https/#create-a-ca-server-and-client-keys-with-openssl Enable dockerd control via tcp socket: remove the -H fd:// option from the docker config.
3. Set paths to certificates in docker.json
4. Register in the gitlab variables in the CI / CD settings with the contents of the certificates. Write a .gitlab-ci.yml script for deployment.

I will show all examples on the distribution Debian.

Initial VPS setup

Here you bought an instance for example on DO, the first thing to do is to protect your server from the aggressive outside world. I will not prove or assert anything, I will just show the /var/log/messages log of my virtual server:

Screenshot

First, install the ufw firewall:

apt-get update && apt-get install ufw

Enable the default policy: block all incoming connections, allow all outgoing connections:

ufw default deny incoming
ufw default allow outgoing

Important: do not forget to allow connection via ssh:

ufw allow OpenSSH

The general syntax is: Allow connection on port: ufw allow 12345, where 12345 is the port number or service name. Deny: ufw deny 12345

Turn on firewall:

ufw enable

We exit the session and log in again via ssh.

Add a user, assign him a password, and add him to the sudo group.

apt-get install sudo
adduser scoty
usermod -aG sudo scoty

Next, according to the plan, you should disable password login. to do this, copy your ssh key to the server:

ssh-copy-id root@10.101.10.28

The ip of the server must be yours. Now try to log in under the user created earlier, you do not need to enter a password anymore. Next, in the configuration settings, change the following:

sudo nano /etc/ssh/sshd_config

disable password login:

PasswordAuthentication no

Restart the sshd daemon:

sudo systemctl reload sshd

Now if you or someone else tries to log in as root, it will fail.

Next, we install dockerd, I won’t describe the process here, since everything can already be changed, follow the link to the official website and go through the steps of installing docker on your virtual machine: https://docs.docker.com/install/linux/docker-ce/debian/

Certificate generation

To control the docker daemon remotely, an encrypted TLS connection is required. To do this, you need to have a certificate and a key that you need to generate and transfer to your remote machine. Follow the steps given in the instructions on the official docker website: https://docs.docker.com/engine/security/https/#create-a-ca-server-and-client-keys-with-openssl All generated *.pem files for the server, namely ca.pem, server.pem, key.pem, should be placed in the /etc/docker directory on the server.

docker setup

In the docker daemon startup script, remove the -H df:// option, this option tells which host the docker daemon can be controlled on.

# At /lib/systemd/system/docker.service
[Service]
Type=notify
ExecStart=/usr/bin/dockerd

Next, create a settings file if it does not already exist and set the options:

/etc/docker/docker.json

{
  "hosts": [
    "unix:///var/run/docker.sock",
    "tcp://0.0.0.0:2376"
  ],
  "labels": [
    "is-our-remote-engine=true"
  ],
  "tls": true,
  "tlscacert": "/etc/docker/ca.pem",
  "tlscert": "/etc/docker/server.pem",
  "tlskey": "/etc/docker/key.pem",
  "tlsverify": true
}

Allow connections on port 2376:

sudo ufw allow 2376

Restart dockerd with new settings:

sudo systemctl daemon-reload && sudo systemctl restart docker

Check:

sudo systemctl status docker

If everything is green, then we consider that we have successfully configured docker on the server.

Setting up continuous delivery on gitlab

In order for the gitalab worker to be able to execute commands on a remote docker host, you need to decide how and where to store certificates and a key for an encrypted connection to dockerd. I solved this problem by simply writing to the variables in the gitlbab settings:

Spoiler header

Just output the contents of the certificates and key via cat: cat ca.pem. Copy and paste into variable values.

Let's write a script for deployment through gitlab. The docker-in-docker (dind) image will be used.

.gitlab-ci.yml

image:
  name: docker/compose:1.23.2
  # перепишем entrypoint , чтобы работало в dind
  entrypoint: ["/bin/sh", "-c"]

variables:
  DOCKER_HOST: tcp://docker:2375/
  DOCKER_DRIVER: overlay2

services:
  - docker:dind

stages:
  - deploy

deploy:
  stage: deploy
  script:
    - bin/deploy.sh # скрипт деплоя тут

The contents of the deployment script with comments:

bin/deploy.sh

#!/usr/bin/env sh
# Падаем сразу, если возникли какие-то ошибки
set -e
# Выводим, то , что делаем
set -v

# 
DOCKER_COMPOSE_FILE=docker-compose.yml
# Куда деплоим
DEPLOY_HOST=185.241.52.28
# Путь для сертификатов клиента, то есть в нашем случае - gitlab-воркера
DOCKER_CERT_PATH=/root/.docker

# проверим, что в контейнере все имеется
docker info
docker-compose version

# создаем путь (сейчас работаем в клиенте - воркере gitlab'а)
mkdir $DOCKER_CERT_PATH
# изымаем содержимое переменных, при этом удаляем лишние символы добавленные при сохранении переменных.
echo "$CA_PEM" | tr -d 'r' > $DOCKER_CERT_PATH/ca.pem
echo "$CERT_PEM" | tr -d 'r' > $DOCKER_CERT_PATH/cert.pem
echo "$KEY_PEM" | tr -d 'r' > $DOCKER_CERT_PATH/key.pem
# на всякий случай даем только читать
chmod 400 $DOCKER_CERT_PATH/ca.pem
chmod 400 $DOCKER_CERT_PATH/cert.pem
chmod 400 $DOCKER_CERT_PATH/key.pem

# далее начинаем уже работать с удаленным docker-демоном. Собственно, сам деплой
export DOCKER_TLS_VERIFY=1
export DOCKER_HOST=tcp://$DEPLOY_HOST:2376

# проверим, что коннектится все успешно
docker-compose 
  -f $DOCKER_COMPOSE_FILE 
  ps

# логинимся в docker-регистри, тут можете указать свой "местный" регистри
docker login -u $DOCKER_USER -p $DOCKER_PASSWORD

docker-compose 
  -f $DOCKER_COMPOSE_FILE 
  pull app
# поднимаем приложение
docker-compose 
  -f $DOCKER_COMPOSE_FILE 
  up -d app

The main problem was to "pull out" the contents of the certificates in the normal form from the gitlab CI / CD variables. I couldn't figure out why the connection to the remote host didn't work. I looked at the sudo journalctl -u docker log on the host, there is an error with the handshake. I decided to look at what is generally stored in variables, for this you can see cat -A $DOCKER_CERT_PATH/key.pem. Overcame the error by adding the removal of the caret character tr -d 'r'.

Further, you can add post-release tasks to the script at your discretion. You can check out the working version in my repository https://gitlab.com/isqad/gitlab-ci-cd

Source: habr.com