🥇 Introduction to Helm 3

Note: translation.: May 16 of this year marks a significant milestone in the development of the package manager for Kubernetes — Helm. On this day, the first alpha release of the upcoming major version of the project — 3.0 — was announced. Its release will bring substantial and long-awaited changes to Helm, which many in the Kubernetes community have high hopes for. We are among them, as we actively use Helm for application deployment: we have integrated it into our CI/CD implementation tool. werf and from time to time, we contribute to the development of upstream. This translation combines 7 notes from the official Helm blog, timed to the first alpha release of Helm 3, discussing the project's history and the key features of Helm 3. The author is Matt "bacongobbler" Fisher, a Microsoft employee and one of the key maintainers of Helm.

On October 15, 2015, the project known today as Helm was born. Just a year after its founding, the Helm community joined Kubernetes, actively working on Helm 2 along the way. In June 2018, Helm became a part of the CNCF as an incubating project. Fast forward to the present — and the first alpha release of the new Helm 3 is already on the way (this release has already taken place in mid-May — ed.).

In this material, I will talk about how it all began, how we reached the current stage, present some unique features available in the first alpha release of Helm 3, and explain how we plan to evolve further.

Summary:

the history of Helm's creation;
a fond farewell to Tiller;
chart repositories;
release management;
changes in chart dependencies;
library charts;
what's next?

The History of Helm's Creation

Birth

Helm 1 started as an Open Source project created by Deis. We were a small startup, acquired by Microsoft in the spring of 2017. Our other Open Source project, also named Deis, had a tool deisctl, which was used (among other things) for installing and operating the Deis platform in a Fleet cluster.At that time, Fleet was one of the first platforms for orchestrating containers.

In mid-2015, we decided to change course and migrated Deis (then renamed to Deis Workflow) from Fleet to Kubernetes. One of the first tools to be redesigned was the installation tool. deisctlWe used it for installing and managing Deis Workflow in the Fleet cluster.

Helm 1 was designed in the image of well-known package managers, such as Homebrew, apt, and yum. Its primary task was to simplify tasks such as packaging and installing applications in Kubernetes. Helm was officially introduced in 2015 at the KubeCon conference in San Francisco.

Our first attempt with Helm worked, but it came with serious limitations. It took a set of Kubernetes manifests, enhanced with generators as input YAML blocks, (front-matter)*, and loaded the results into Kubernetes.

* Note: translation.: From the first version of Helm, YAML syntax was chosen to describe Kubernetes resources, and Jinja templates and Python scripts were supported for writing configurations. We wrote more about this and the structure of the first version of Helm in the chapter 'A Brief History of Helm' of this material. this material.

For example, to replace a field in a YAML file, you needed to add the following construct to the manifest:

#helm:generate sed -i -e s|ubuntu-debootstrap|fluffy-bunny| my/pod.yaml

It's great that today we have templating engines, isn't it?

For many reasons, this early Kubernetes installer required a hardcoded list of manifest files and only executed a small fixed sequence of events. It was so difficult to use that the Deis Workflow R&D team had a tough time trying to port their product to this platform—however, the seeds of the idea had already been sown. Our first attempt became a great learning opportunity: we realized that we were truly passionate about creating pragmatic tools that solve everyday problems for our users.

Building on the experience of past mistakes, we set out to develop Helm 2.

Creating Helm 2

At the end of 2015, the Google team reached out to us. They were working on a similar tool for Kubernetes. The Deployment Manager for Kubernetes was a port of an existing tool that was used for Google Cloud Platform. 'Would we like to spend a few days discussing the similarities and differences?' they asked.

In January 2016, the Helm and Deployment Manager teams met in Seattle to exchange ideas. The discussions ended with an ambitious plan: to combine both projects to create Helm 2. Along with Deis and Google, the guys from SkippBox (now part of Bitnami — translator's note), joined the development team, and we started working on Helm 2.

We wanted to maintain the simplicity of using Helm, but add the following:

chart templates for customization;
in-cluster management for teams;
a first-class chart repository;
a stable package format with signing capability;
a strong commitment to semantic versioning and maintaining backward compatibility between versions.

To achieve these goals, a second component was added to the Helm ecosystem. This in-cluster component was called Tiller and was responsible for installing and managing Helm charts.

Since the release of Helm 2 in 2016, Kubernetes has undergone several significant innovations. Role-based access control (RBAC) has ultimately replaced attribute-based access control (ABAC). New resource types were introduced (Deployments were still in beta at that time). Custom Resource Definitions were invented (originally named Third Party Resources or TPRs). Most importantly, a set of best practices emerged.

Against the backdrop of all these changes, Helm continued to serve the Kubernetes community faithfully. After three years and numerous additions, it became clear that substantial changes were needed in the codebase for Helm to continue meeting the growing needs of the evolving ecosystem.

A Fond Farewell to Tiller

During the development of Helm 2, we introduced Tiller as part of our integration with Google’s Deployment Manager. Tiller played a crucial role for teams operating within a shared cluster: it enabled various specialists managing the infrastructure to interact with the same set of releases.

Since Role-Based Access Control (RBAC) was enabled by default in Kubernetes 1.6, working with Tiller in production became more complicated. Due to the multitude of possible security policies, our stance was to offer a permissive configuration by default. This allowed newcomers to experiment with Helm and Kubernetes without needing to first dive into security settings. Unfortunately, this permissive configuration could endow users with overly broad permissions they did not need. DevOps and SRE engineers had to learn additional operational steps to deploy Tiller in a multi-tenant cluster.

By learning how community members utilize Helm in specific situations, we understood that Tiller's release management system did not need to rely on an in-cluster component to maintain states or function as a central hub for release information. Instead, we could simply obtain information from the Kubernetes API server, generate the chart on the client side, and store the installation record in Kubernetes.

The main task of Tiller could be accomplished without it, so one of our first decisions regarding Helm 3 was to completely eliminate Tiller.

With the removal of Tiller, Helm's security model has been radically simplified. Helm 3 now supports all modern security, identification, and authorization methods of the current Kubernetes. Helm's permissions are defined using the kubeconfig file.Cluster administrators can limit user rights with any level of granularity. Releases are still stored within the cluster, and the rest of Helm's functionality remains intact.

Chart Repositories

At a high level, a chart repository is a location where charts can be stored and shared. The Helm client packages and uploads charts to the repository. Simply put, a chart repository is a primitive HTTP server with an index.yaml file and some packaged charts.

While there are some advantages to having the chart repository API meet basic storage requirements, it also has several disadvantages:

Chart repositories are poorly compatible with most security implementations required in a production environment. Having a standard API for authentication and authorization is crucial in production scenarios.
Helm’s tools for tracking chart provenance, used for signing, integrity verification, and provenance of charts, are an optional part of the chart publishing process.
In multi-user scenarios, the same chart can be uploaded by another user, effectively doubling the storage space required for the same content. Smarter repositories have been developed to address this issue, but they are not part of the formal specification.
Using a single index file for searching, storing metadata, and retrieving charts has complicated the development of secure multi-user implementations.

Project Docker Distribution (also known as Docker Registry v2) is the successor to Docker Registry and effectively serves as a toolkit for packaging, shipping, storing, and delivering Docker images. Many leading cloud services offer products based on Distribution. Due to this increased attention, the Distribution project has benefited from years of improvements, best practices in security, and real-world testing, making it one of the most successful unsung heroes in the Open Source world.

But did you know that the Distribution project was designed to distribute any form of content, not just container images?

Thanks to the efforts of Open Container Initiative (or OCI), Helm charts can be hosted on any instance of Distribution. While this process is still experimental, work on supporting logins and other features necessary for full Helm 3 functionality is not yet complete, but we are very excited about the opportunity to learn from the discoveries made by the OCI and Distribution teams over the years. And with their mentorship and guidance, we are learning what it means to operate a highly available service at scale.

A more detailed description of some upcoming changes in Helm chart repositories is available at the link.

Release Management

In Helm 3, the application state is tracked within the cluster by a pair of objects:

release object — represents an instance of an application;
release version secret — represents the desired state of the application at a specific point in time (e.g., the release of a new version).

Call helm install creates release object and release version secret. The call helm upgrade requires the presence of release object (which it can change) and creates a new release version secret containing new values and a prepared manifest.

Release object contains information about the release, where the release is a specific installation of a named chart and values. This object describes high-level metadata about the release. The release object is maintained throughout the application lifecycle and is the owner of all release version secrets, as well as all objects that are directly created by the Helm chart.

Release version secret links the release to a series of revisions (installation, updates, rollbacks, deletions).

In Helm 2, revisions were exclusively sequential. The call helm install created v1, the subsequent upgrade generated v2, and so on. Release and release version secret were collapsed into a single object known as revision. Revisions were stored in the same namespace as Tiller, which meant that each release was 'global' in terms of namespace; as a result, only one instance of a name could be used.

In Helm 3, each release is associated with one or more release version secrets. The release object always describes the current release deployed in Kubernetes. Each release version secret describes only one version of that release. An upgrade, for example, will create a new release version secret and then modify the release object to point to this new version. In the case of a rollback, previous release version secrets can be used to revert the release to its prior state.

After the removal of Tiller, Helm 3 stores release data in the same namespace as the release. This change allows a chart with the same release name to be installed in another namespace, and the data is retained between updates/restarts of the cluster in etcd. For example, WordPress can be installed in the namespace 'foo' and then in the namespace 'bar', and both releases can be named 'wordpress'.

Changes in chart dependencies

Charts packaged (with helm package) for use with Helm 2, can be installed with Helm 3; however, the chart development workflow has been completely reworked, so some adjustments are necessary to continue developing charts with Helm 3. Specifically, the chart dependency management system has changed.

The chart dependency management has shifted from requirements.yaml and requirements.lock to Chart.yaml and Chart.lock. This means that charts that used the command helm dependency, require some configuration to work in Helm 3.

Let's consider an example. We'll add a dependency to a chart in Helm 2 and see what changes when transitioning to Helm 3.

In Helm 2 requirements.yaml it looked like this:

dependencies:
- name: mariadb
  version: 5.x.x
  repository: https://kubernetes-charts.storage.googleapis.com/
  condition: mariadb.enabled
  tags:
    - database

In Helm 3, the same dependency will be reflected in your Chart.yaml:

dependencies:
- name: mariadb
  version: 5.x.x
  repository: https://kubernetes-charts.storage.googleapis.com/
  condition: mariadb.enabled
  tags:
    - database

Charts are still downloaded and placed in the directory charts/, so subcharts (subcharts), located in the directory charts/, will continue to function unchanged.

Introducing Library Charts

Helm 3 supports a class of charts known as library charts (library chart). This chart is used by other charts but does not create any release artifacts by itself. Library chart templates can only declare elements define. Other content is simply ignored. This allows users to reuse and share code snippets that can be leveraged across multiple charts, thereby avoiding duplication and adhering to the principle of DRY.

Library charts are declared in the section dependencies in the file Chart.yaml. Installation and management are no different from other charts.

dependencies:
  - name: mylib
    version: 1.x.x
    repository: quay.io

We are eagerly anticipating the use cases that this component will open up for chart developers, as well as the best practices that may arise from library charts.

What's next?

Helm 3.0.0-alpha.1 is the foundation upon which we begin building the new version of Helm. In this article, I've outlined some interesting features of Helm 3. Many of these are still in early stages of development, which is perfectly normal; the purpose of an alpha release is to test the idea, gather feedback from early users, and validate our assumptions.

Once the alpha version is released (let's remember that this has already happened - translator's note), we will start accepting patches for Helm 3 from the community. It is essential to build a solid foundation that allows for the development and acceptance of new features, while users can feel engaged in the process by opening tickets and submitting fixes.

In this article, I aimed to highlight some significant improvements coming to Helm 3; however, this list is by no means exhaustive. The comprehensive plan for Helm 3 includes innovations such as enhanced upgrade strategies, deeper integration with OCI registries, and the use of JSON schemas for validating chart values. We also plan to clean up the codebase and update the parts that have been neglected over the past three years.

If you feel we’ve missed something, we would love to hear your thoughts!

Join the discussion in our Slack channels:

#helm-users for questions and casual chat with the community;
#helm-dev for discussing pull requests, code, and bugs.

You can also engage in our weekly Public Developer Calls on Thursdays at 7:30 PM MSK. The meetings focus on discussing tasks that key developers and the community are working on, as well as topics for the week. Anyone is welcome to join and participate in the meeting. The link is available in the Slack channel #helm-dev.