CRI-O as a replacement for Docker as a runtime for Kubernetes: Setting up on CentOS 8

Hello! My name is Sergey, I'm DevOps at Surf. The DevOps department at Surf aims not only to establish interaction between specialists and integrate workflows, but also to actively research and implement relevant technologies both in their own infrastructure and in the customer's infrastructure.

Below I will talk a little about the changes in the technological stack for containers that we met while studying the distribution. 8 CentOS and what is create it and how to quickly set up an executable environment with it for Kubernetes.

Why is Docker missing from the standard distribution of CentOS 8

After installing the latest major releases RHEL8 or 8 CentOS not to be overlooked: these distributions and official repositories are missing the application Docker, which ideologically and functionally replace packages podman, Buildah (present in the distribution by default) and create it. This is due to the practical implementation of standards developed, among other things, by Red Hat as part of the Open Container Initiative (OCI) project.

The goal of OCI, which is part of The Linux Foundation, is to create open industry standards for container formats and executable environments that would solve several problems at once. Firstly, they did not contradict the philosophy of Linux (for example, in that part of it that each program must perform some one action, and Docker is a sort of combine all-in-one). Secondly, we could eliminate all the existing shortcomings in the software Docker. Thirdly, they would be fully compatible with the business requirements put forward by the leading commercial platforms for deploying, managing and maintaining containerized applications (for example, Red Hat OpenShift).

Disadvantages Docker and the advantages of the new software have already been described in some detail in this article, and a detailed description of how the entire software stack offered as part of the OCI project and its architectural features can be found in the official documentation and articles both from Red Hat itself (good article in the Red Hat blog), and in third-party reviews.

It is important to note what functionality the components of the proposed stack have:

• podman - direct interaction with containers and image storage through the runC process;
• Buildah - assembly and loading into the registry of images;
• create it - runtime environment for container orchestration systems (for example, Kubernetes).

I think that in order to understand the general scheme of interaction between the components of the stack, it is advisable to give here a connection diagram Kubernetes c runC and low-level libraries using create it:

create it и Kubernetes follow the same release and support cycle (the compatibility matrix is ​​very simple: major releases Kubernetes и create it coincide), and this, taking into account the focus on full and comprehensive testing of the work of this stack by developers, gives us the right to expect the maximum achievable stability in work in any usage scenarios (relative lightness is also beneficial here). create it compared with Docker due to purposeful limitation of functionality).

When you install Kubernetes "right way" way (according to the OCI, of course) using create it on 8 CentOS We ran into some minor difficulties, which, however, we successfully overcame. I will be glad to share with you the installation and configuration instructions, which in total will take 10 minutes at the most.

How to Deploy Kubernetes on CentOS 8 Using CRI-O Environment

Prerequisites: Have at least one host (2 cores, 4 GB RAM, at least 15 GB storage) with 8 CentOS (recommended installation profile "Server"), as well as entries for it in the local DNS (in extreme cases, you can get by with an entry in /etc/hosts). And don't forget disable swap.

We perform all operations on the host on behalf of the root user, be careful.

1. In the first step, we will set up the OS, install and configure the preliminary dependencies for CRI-O.
   • Update OS:

     dnf -y update
     

   • Next, you need to configure the firewall and SELinux. Here, everything depends on the environment in which our host or hosts will work. You can either set up a firewall as recommended in documentation, or if you are on a trusted network or use a third-party firewall, change the default zone to trusted or turn off the firewall:

     firewall-cmd --set-default-zone trusted
     
     firewall-cmd --reload

     To turn off the firewall, you can use the following command:

     systemctl disable --now firewalld
     

     SELinux needs to be disabled or set to "permissive" mode:

     setenforce 0
     
     sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

   • load the necessary kernel modules and packages, configure automatic loading of the “br_netfilter” module at system startup:

     modprobe overlay
     
     modprobe br_netfilter
     
     echo "br_netfilter" >> /etc/modules-load.d/br_netfilter.conf
     
     dnf -y install iproute-tc
     

   • To activate packet forwarding and correctly process traffic, we will make the appropriate settings:

     cat > /etc/sysctl.d/99-kubernetes-cri.conf <<EOF
     net.bridge.bridge-nf-call-iptables = 1
     net.ipv4.ip_forward = 1
     net.bridge.bridge-nf-call-ip6tables = 1
     EOF
     

     apply the settings made:

     sysctl --system

   • set the required version create it (major version create it, as already mentioned, are the same as the required version Kubernetes) since the latest stable version Kubernetes currently 1.18:

     export REQUIRED_VERSION=1.18
     

     add the necessary repositories:

     dnf -y install 'dnf-command(copr)'
     
     dnf -y copr enable rhcontainerbot/container-selinux
     
     curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/CentOS_8/devel:kubic:libcontainers:stable.repo
     
     curl -L -o /etc/yum.repos.d/devel:kubic:libcontainers:stable:cri-o:$REQUIRED_VERSION.repo https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable:cri-o:$REQUIRED_VERSION/CentOS_8/devel:kubic:libcontainers:stable:cri-o:$REQUIRED_VERSION.repo

   • now we can install create it:

     dnf -y install cri-o
     

     Pay attention to the first nuance that we meet during the installation process: you need to edit the configuration create it before starting the service, since the required conmon component has a different location than the one specified:

     sed -i 's//usr/libexec/crio/conmon//usr/bin/conmon/' /etc/crio/crio.conf

     Now you can activate and start the daemon create it:

     systemctl enable --now crio
     

     You can check the status of the daemon:

     systemctl status crio
     

2. Installation and activation Kubernetes.
   • Add the required repository:

     cat <<EOF > /etc/yum.repos.d/kubernetes.repo
     [kubernetes]
     name=Kubernetes
     baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-$basearch
     enabled=1
     gpgcheck=1
     repo_gpgcheck=1
     gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
     exclude=kubelet kubeadm kubectl
     EOF
     

     Now we can install Kubernetes (version 1.18, as mentioned above):

     dnf install -y kubelet-1.18* kubeadm-1.18* kubectl-1.18* --disableexcludes=kubernetes

   • The second important nuance: since we do not use the daemon Docker, but use the daemon create it, before launch and initialization Kubernetes you need to make the appropriate settings in the /var/lib/kubelet/config.yaml configuration file, after creating the necessary directory:

     mkdir /var/lib/kubelet
     
     cat <<EOF > /var/lib/kubelet/config.yaml
     apiVersion: kubelet.config.k8s.io/v1beta1
     kind: KubeletConfiguration
     cgroupDriver: systemd
     EOF

   • The third important point that we encounter during installation: despite the fact that we have indicated the driver used group, and its configuration through the arguments passed kublet outdated (which is directly indicated in the documentation), we need to add arguments to the file, otherwise our cluster will not be initialized:

     cat /dev/null > /etc/sysconfig/kubelet
     
     cat <<EOF > /etc/sysconfig/kubelet
     KUBELET_EXTRA_ARGS=--container-runtime=remote --cgroup-driver=systemd --container-runtime-endpoint='unix:///var/run/crio/crio.sock'
     EOF

   • Now we can activate the daemon kublet:

     sudo systemctl enable --now kubelet
     

     To customize control-plane or worker nodes in minutes, you can use this script.

3. It's time to initialize our cluster.
   • To initialize the cluster, run the command:

     kubeadm init --pod-network-cidr=10.244.0.0/16
     

     Be sure to write down the command to join the cluster "kubeadm join ...", which is offered at the end of the output, or at least the specified tokens.

   • Install the plugin (CNI) for the Pod network. I recommend using Calico. Possibly more popular Flannel has compatibility issues with nftables, and Calico - the only CNI implementation recommended and fully tested by the project Kubernetes:

     kubectl --kubeconfig /etc/kubernetes/admin.conf apply -f https://docs.projectcalico.org/v3.15/manifests/calico.yaml 

   • To connect a worker node to our cluster, you need to configure it according to instructions 1 and 2, or use script, then run the command from the "kubeadm init ..." output we recorded in the previous step:

     kubeadm join $CONTROL_PLANE_ADDRESS:6443 --token $TOKEN 
         --discovery-token-ca-cert-hash $TOKEN_HASH

   • Let's check that our cluster is initialized and started working:

     kubectl --kubeconfig=/etc/kubernetes/admin.conf get pods -A
     

   Ready! You can already host payloads on your K8s cluster.

What lies ahead for us

I hope that the instructions above helped save you some time and nerves.
The outcome of the processes taking place in the industry often depends on how they are accepted by the main mass of end users and developers of other software in the relevant niche. It is not entirely clear where the OCI initiatives will lead in a few years, but we will be happy to follow it. You can share your opinion right now in the comments.

Stay tuned!

This article was created thanks to the following sources:

• Section on Container runtimes Kubernetes documentation
• Page CRI-O project on the Internet
• Red Hat Blog Articles: this one, this and many others

Source: habr.com