Platform enables streamlined creation , including in cloud service provider infrastructure, on virtualization platforms, or in bare-metal systems. To truly create a cloud platform, we had to tightly control all elements used and thus enhance the reliability of the complex automation process.
The obvious solution was to use Red Hat Enterprise Linux CoreOS (a variant of Red Hat Enterprise Linux) and CRI-O as the standard, and here’s why…
Since the theme of sailing is quite suitable for drawing analogies when explaining the workings of Kubernetes and containers, let’s discuss the business problems that CoreOS and CRI-O solve, using the example . In 1803, Mark Brunel faced the task of manufacturing 100,000 tackle blocks for the needs of the growing British navy. A tackle block is a type of rigging used to secure ropes to sails. Until the early 19th century, these blocks were made by hand, but Brunel was able to automate production and begin making standardized blocks using machines. Automating this process meant that all blocks were virtually identical, could be easily replaced in case of breakage, and could be manufactured in large quantities.
Now imagine that Brunel had to do this job for 20 different ship models (Kubernetes versions) and for five different planets with completely different ocean currents and winds (cloud providers). Moreover, it was essential that all ships (OpenShift clusters), regardless of the planets they navigated, behaved the same way from the perspective of the captains (operators managing the clusters). Continuing the nautical analogy, the ship captains do not care which tackle blocks (CRI-O) are being used on their ships – what matters to them is that these blocks are sturdy and reliable.
Before OpenShift 4, a very similar business challenge stood in front of the cloud platform. New nodes must be created at the time of cluster creation, in case of a failure in one of the nodes, or when scaling the cluster. When creating and initializing a new node, the critical host components, including CRI-O, must be properly configured. As in any other production, it is essential to provide the 'raw materials' at the outset. In the case of ships, the raw materials are metal and wood. However, when creating a host for deploying containers in the OpenShift 4 cluster, configuration files and provided API servers must be available as inputs. After that, OpenShift will ensure the required level of automation throughout the lifecycle, offering necessary product support for end users and thereby recouping investments in the platform.
OpenShift 4 was designed to facilitate easy system updates throughout the platform's lifecycle (for versions 4.X) for all major cloud computing providers, virtualization platforms, and even bare metal systems. To achieve this, nodes must be built on interchangeable components. When a cluster requires a new version of Kubernetes, it also receives the corresponding version of CRI-O on CoreOS. Since the version of CRI-O is closely tied to Kubernetes, this greatly simplifies any rearrangements for testing, troubleshooting, or support. Furthermore, this approach helps reduce costs for end users and Red Hat.
This represents a fundamentally new perspective on Kubernetes clusters, laying the groundwork for planning new, highly useful and appealing features. CRI-O (the open container runtime project Container Runtime Interface — Open Container Initiative, abbreviated CRI-OCI) has proven to be the most successful choice for mass node creation necessary for working with OpenShift. CRI-O will replace the previously used Docker engine, offering OpenShift users – yes, you heard it right – a boring container engine specifically created to work with Kubernetes.
The world of open containers
The world has long been moving towards open containers. Whether in Kubernetes or at lower levels, is leading to an ecosystem of innovation at every level.
It all started with the creation of the Open Containers Initiative At this early stage, specifications for container and were established. This ensured that tools could use a single standard and a unified format to work with them. Later, specifications for were added, enabling users to easily share .
Then, the Kubernetes community developed a unified standard for a pluggable interface, called the This allowed Kubernetes users to plug in various engines to work with containers in addition to Docker.
Engineers from Red Hat and Google recognized an existing market need for a container engine capable of receiving requests from Kubelet via the CRI protocol and presented containers compatible with the aforementioned OCI specifications. Thus, But wait, didn’t we say this material would be about CRI-O? That is indeed the case; with the release of the project was renamed CRI-O.
Fig. 1.
Innovations with CRI-O and CoreOS
With the launch of the OpenShift 4 platform, the used in the platform was changed, as Docker was replaced by CRI-O, offering an economical, stable, simple, and straightforward environment for running containers that evolves alongside Kubernetes. This significantly simplifies cluster support and configuration. Configuring the container engine and host, as well as managing them, becomes automated within OpenShift 4.
Wait, how is that?
That's right, with the advent of OpenShift 4, there is no longer any need to connect to separate hosts and install the container engine, configure storage, set up servers for discovery, or configure the network. The OpenShift 4 platform has been completely redesigned for use with not only from the perspective of end-user applications but also from the perspective of base operations at the platform level, such as image deployment, system configuration, or installing updates.
Kubernetes has always allowed users to manage applications by defining the desired state and using , to ensure that the actual state closely matches the defined state. This opens up significant opportunities both in terms of development and operations. Developers can define the required state, to the operator in the form of a YAML or JSON file, and then the operator can create the necessary application instance in the operational environment, with the operational state of that instance fully aligning with the specified state.
By using operators on the platform, OpenShift 4 introduces this new paradigm (using the concept of desired and actual state) for the management of RHEL CoreOS and CRI-O. The tasks of configuring and managing the version of the operating system and the container engine are automated through what is known as the . The MCO significantly simplifies the work of the cluster administrator, essentially automating the final stages of installation as well as subsequent operations after installation (day two operations). All of this makes OpenShift 4 a true cloud platform. We will discuss this in more detail later.
Running Containers
Users have been able to use the CRI-O engine on the OpenShift platform since version 3.7 in Tech Preview status and since version 3.9 in Generally Available status (currently supported). Additionally, Red Hat extensively uses in OpenShift Online starting with version 3.10. All of this has allowed the team working on CRI-O to gain extensive experience in mass container deployment on large Kubernetes clusters. To get a basic understanding of how Kubernetes uses CRI-O, let's look at the following illustration, which shows the principles of the architecture.
Fig. 2. How containers operate in a Kubernetes cluster
CRI-O simplifies the creation of new container hosts by synchronizing the entire upper layer when initializing new nodes, as well as during the release of new versions of the OpenShift platform. A comprehensive revision of the entire platform allows for transactional updates/rollbacks and prevents mutual locking in dependencies between the container host core, the container engine, nodes (Kubelets), and the Kubernetes Master node. With centralized management of all platform components, along with version control and management, a clear path from state A to state B can always be tracked. This streamlines the update process, enhances security, improves performance reporting, and helps reduce costs associated with updates and installation of new versions.
Demonstrating the power of interchangeable components
As mentioned earlier, using the Machine Config Operator to manage the container host and container engine in OpenShift 4 provides a new level of automation that was not possible on the Kubernetes platform previously. To showcase these new capabilities, we will demonstrate how you might make changes to the crio.conf file. To avoid confusion with terminology, try to focus on the outcomes.
First, let's create what is called a Container Runtime Config. Think of it as a Kubernetes resource that represents the configuration for CRI-O. In reality, it's a specialized version of what is known as MachineConfig, which represents any configuration deployed on a RHEL CoreOS machine within an OpenShift cluster.
This custom resource, called ContainerRuntimeConfig, was designed to simplify CRI-O configuration for cluster administrators. It is a powerful tool, as it can be applied only to specific nodes depending on MachineConfigPool settings. Think of it as a group of machines serving the same purpose.
Take note of the last two lines that we are going to change in the /etc/crio/crio.conf file. These two lines are very similar to the lines in the crio.conf file, namely:
vi ContainerRuntimeConfig.yaml
Output:
apiVersion: machineconfiguration.openshift.io/v1
kind: ContainerRuntimeConfig
metadata:
name: set-log-and-pid
spec:
machineConfigPoolSelector:
matchLabels:
debug-crio: config-log-and-pid
containerRuntimeConfig:
pidsLimit: 2048
logLevel: debug
Now let's send this file to the Kubernetes cluster and check if it has been actually created. Note that the process is exactly the same as with any other Kubernetes resource:
oc create -f ContainerRuntimeConfig.yaml
oc get ContainerRuntimeConfig
Output:
NAME AGE
set-log-and-pid 22h
After we created the ContainerRuntimeConfig, we need to modify one of the MachineConfigPools to let Kubernetes know that we want to apply this configuration to a specific group of machines in the cluster. In this case, we will modify the MachineConfigPool for the master nodes:
oc edit MachineConfigPool/master
Output (the main essence is left for clarity):
...
metadata:
creationTimestamp: 2019-04-10T23:42:28Z
generation: 1
labels:
debug-crio: config-log-and-pid
operator.machineconfiguration.openshift.io/required-for-upgrade: ""
...
At this point, the MCO starts creating a new crio.conf file for the cluster. The complete configuration file can be reviewed using the Kubernetes API. Remember, ContainerRuntimeConfig is just a specialized version of MachineConfig, so we can see the result by looking at the relevant lines in MachineConfigs:
oc get MachineConfigs | grep rendered
Output:
rendered-master-c923f24f01a0e38c77a05acfd631910b 4.0.22-201904011459-dirty 2.2.0 16h
rendered-master-f722b027a98ac5b8e0b41d71e992f626 4.0.22-201904011459-dirty 2.2.0 4m
rendered-worker-9777325797fe7e74c3f2dd11d359bc62 4.0.22-201904011459-dirty 2.2.0 16h
Note that the generated configuration file for the master nodes is a newer version than the original configurations. To view it, run the following command. By the way, this might be one of the best one-liners in the history of Kubernetes:
python3 -c "import sys, urllib.parse; print(urllib.parse.unquote(sys.argv[1]))" $(oc get MachineConfig/rendered-master-f722b027a98ac5b8e0b41d71e992f626 -o YAML | grep -B4 crio.conf | grep source | tail -n 1 | cut -d, -f2) | grep pid
Output:
pids_limit = 2048
Now let's ensure that the configuration was applied to all master nodes. First, we'll get a list of nodes in the cluster:
oc get node | grep master
Output:
ip-10-0-135-153.us-east-2.compute.internal Ready master 23h v1.12.4+509916ce1
ip-10-0-154-0.us-east-2.compute.internal Ready master 23h v1.12.4+509916ce1
ip-10-0-166-79.us-east-2.compute.internal Ready master 23h v1.12.4+509916ce1
Now let's take a look at the installed file. You will see that the file has been updated with the new values for the pid and debug directives that we specified in the ContainerRuntimeConfig resource. The elegance itself:
oc debug node/ip-10-0-135-153.us-east-2.compute.internal — cat /host/etc/crio/crio.conf | egrep 'debug||pid'
Output:
...pids_limit = 2048
...
log_level = "debug"
...
All these changes in the cluster were made without starting SSH. The entire operation was performed by accessing the Kubernetes master node. This means that these new parameters were configured only on the master nodes. The worker nodes, meanwhile, remained unchanged, demonstrating the advantages of the Kubernetes methodology using desired and current states applicable to container hosts and container engines with interchangeable elements.
The above example illustrates the ability to make changes in a small OpenShift Container Platform 4 cluster with three worker nodes or in a large production cluster with 3000 nodes. In any case, the workload will be the same – quite small – just configure the ContainerRuntimeConfig file and change one label in the MachineConfigPool. And you can achieve this with any version of the OpenShift Container Platform 4.X used in Kubernetes throughout its lifecycle.
Often, technology companies evolve so rapidly that we cannot explain why we choose certain technologies for core components. Container engines have historically been the component that users interact with directly. Since the popularity of containers naturally began with the emergence of container engines, users frequently show interest in them. This is another reason why Red Hat opted for CRI-O. Containers are evolving, and today the focus is on orchestration, and we concluded that CRI-O provides the best experience when working with OpenShift 4.
Source: habr.com
