You can read about the helmfile itself and examples of its use in
We will get acquainted with non-obvious ways to describe releases in helmfile
Let's say we have a bunch of helm charts (for example, let's say postgres and some backend application) and several environments (several kubernetes clusters, several namespaces, or several of both). We take helmfile, read the documentation and start describing our environments and releases:
.
├── envs
│ ├── devel
│ │ └── values
│ │ ├── backend.yaml
│ │ └── postgres.yaml
│ └── production
│ └── values
│ ├── backend.yaml
│ └── postgres.yaml
└── helmfile.yaml
helmfile.yaml
environments:
devel:
production:
releases:
- name: postgres
labels:
app: postgres
wait: true
chart: stable/postgresql
version: 8.4.0
values:
- envs/{{ .Environment.Name }}/values/postgres.yaml
- name: backend
labels:
app: backend
wait: true
chart: private-helm-repo/backend
version: 1.0.5
needs:
- postgres
values:
- envs/{{ .Environment.Name }}/values/backend.yaml
We have 2 environments: devel, production - each contains its own values for helm release charts. We will deploy to them like this:
helmfile -n <namespace> -e <env> apply
Different versions of helm charts in different environments
What if we need to roll out different versions of the backend to different environments? How to parameterize release version? Environment values come to the rescue, available through {{ .Values }}
helmfile.yaml
environments:
devel:
+ values:
+ - charts:
+ versions:
+ backend: 1.1.0
production:
+ values:
+ - charts:
+ versions:
+ backend: 1.0.5
...
- name: backend
labels:
app: backend
wait: true
chart: private-helm-repo/backend
- version: 1.0.5
+ version: {{ .Values.charts.versions.backend }}
...
Different set of applications in different environments
Great, but what if we don't need to production
roll out postgres, because we know that we don’t need to shove the database into k8s and we have a wonderful separate postgres cluster for sale? To solve this problem, we have labels (labels)
helmfile -n <namespace> -e devel apply
helmfile -n <namespace> -e production -l app=backend apply
This is great, but I personally prefer to describe which applications to deploy to the environment not with launch arguments, but in the description of the environments themselves. What to do? You can put the description of releases in a separate folder, in the description of the environment, make a list of the necessary releases and "hook" only the necessary releases, ignoring the rest
.
├── envs
│ ├── devel
│ │ └── values
│ │ ├── backend.yaml
│ │ └── postgres.yaml
│ └── production
│ └── values
│ ├── backend.yaml
│ └── postgres.yaml
+ ├── releases
+ │ ├── backend.yaml
+ │ └── postgres.yaml
└── helmfile.yaml
helmfile.yaml
environments:
devel:
values:
- charts:
versions:
backend: 1.1.0
- apps:
- postgres
- backend
production:
values:
- charts:
versions:
backend: 1.0.5
- apps:
- backend
- releases:
- - name: postgres
- labels:
- app: postgres
- wait: true
- chart: stable/postgresql
- version: 8.4.0
- values:
- - envs/{{ .Environment.Name }}/values/postgres.yaml
- - name: backend
- labels:
- app: backend
- wait: true
- chart: private-helm-repo/backend
- version: {{ .Values.charts.versions.backend }}
- needs:
- - postgres
- values:
- - envs/{{ .Environment.Name }}/values/backend.yaml
+ ---
+ bases:
+ {{- range .Values.apps }}
+ - releases/{{ . }}.yaml
+ {{- end }}
releases/postgres.yaml
releases:
- name: postgres
labels:
app: postgres
wait: true
chart: stable/postgresql
version: 8.4.0
values:
- envs/{{ .Environment.Name }}/values/postgres.yaml
releases/backend.yaml
releases:
- name: backend
labels:
app: backend
wait: true
chart: private-helm-repo/backend
version: {{ .Values.charts.versions.backend }}
needs:
- postgres
values:
- envs/{{ .Environment.Name }}/values/backend.yaml
Note
Using bases:
it is necessary to use yaml delimiter ---
so that releases (and other parts like helmDefaults) can be templated with values from environments
In this case, the postgres release will not even be included in the description for production. Very comfortably!
Overridden global values for releases
Of course, it's great that you can set values for helm charts for each environment, but what if we have several environments described, and we want, say, to set the same for all affinity
, but we do not want to configure it by default in the charts themselves, which are stored in turnips.
In this case, we could set 2 files with values for each release: the first with default values that will determine the values of the chart itself, and the second with values for the environment, which in turn will override the default ones.
.
├── envs
+ │ ├── default
+ │ │ └── values
+ │ │ ├── backend.yaml
+ │ │ └── postgres.yaml
│ ├── devel
│ │ └── values
│ │ ├── backend.yaml
│ │ └── postgres.yaml
│ └── production
│ └── values
│ ├── backend.yaml
│ └── postgres.yaml
├── releases
│ ├── backend.yaml
│ └── postgres.yaml
└── helmfile.yaml
releases/backend.yaml
releases:
- name: backend
labels:
app: backend
wait: true
chart: private-helm-repo/backend
version: {{ .Values.charts.versions.backend }}
needs:
- postgres
values:
+ - envs/default/values/backend.yaml
- envs/{{ .Environment.Name }}/values/backend.yaml
envs/default/values/backend.yaml
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app.kubernetes.io/name
operator: In
values:
- backend
topologyKey: "kubernetes.io/hostname"
Defining global values for helm charts of all releases at the environment level
Suppose we have several ingresses in several releases - we could manually define for each chart hosts:
, but in our case the domain is the same, so why not put it in some global variable and just substitute its value in the charts? To do this, those files with values that we want to parameterize will have to have the extension .gotmpl
so that helmfile knows to run it through the template engine.
.
├── envs
│ ├── default
│ │ └── values
- │ │ ├── backend.yaml
- │ │ ├── postgres.yaml
+ │ │ ├── backend.yaml.gotmpl
+ │ │ └── postgres.yaml.gotmpl
│ ├── devel
│ │ └── values
│ │ ├── backend.yaml
│ │ └── postgres.yaml
│ └── production
│ └── values
│ ├── backend.yaml
│ └── postgres.yaml
├── releases
│ ├── backend.yaml
│ └── postgres.yaml
└── helmfile.yaml
helmfile.yaml
environments:
devel:
values:
- charts:
versions:
backend: 1.1.0
- apps:
- postgres
- backend
+ - global:
+ ingressDomain: k8s.devel.domain
production:
values:
- charts:
versions:
backend: 1.0.5
- apps:
- backend
+ - global:
+ ingressDomain: production.domain
---
bases:
{{- range .Values.apps }}
- releases/{{ . }}.yaml
{{- end }}
envs/default/values/backend.yaml.gotmpl
ingress:
enabled: true
paths:
- /api
hosts:
- {{ .Values.global.ingressDomain }}
envs/default/values/postgres.yaml.gotmpl
ingress:
enabled: true
paths:
- /
hosts:
- postgres.{{ .Values.global.ingressDomain }}
Note
Obviously, ingress in the postgres chart is something extremely dubious, so this article is just given as a spherical example in a vacuum and in order not to introduce some new release into the article just for the sake of describing ingress
Substituting secrets from environment values
By analogy with the above example, you can also substitute encrypted with
.
├── envs
│ ├── default
│ │ └── values
│ │ ├── backend.yaml
│ │ └── postgres.yaml
│ ├── devel
│ │ ├── values
│ │ │ ├── backend.yaml
│ │ │ └── postgres.yaml
+ │ │ └── secrets.yaml
│ └── production
│ ├── values
│ │ ├── backend.yaml
│ │ └── postgres.yaml
+ │ └── secrets.yaml
├── releases
│ ├── backend.yaml
│ └── postgres.yaml
└── helmfile.yaml
helmfile.yaml
environments:
devel:
values:
- charts:
versions:
backend: 1.1.0
- apps:
- postgres
- backend
- global:
ingressDomain: k8s.devel.domain
+ secrets:
+ - envs/devel/secrets.yaml
production:
values:
- charts:
versions:
backend: 1.0.5
- apps:
- backend
- global:
ingressDomain: production.domain
+ secrets:
+ - envs/production/secrets.yaml
---
bases:
{{- range .Values.apps }}
- releases/{{ . }}.yaml
{{- end }}
envs/devel/secrets.yaml
secrets:
elastic:
password: ENC[AES256_GCM,data:hjCB,iv:Z1P6/6xBJgJoKLJ0UUVfqZ80o4L84jvZfM+uH9gBelc=,tag:dGqQlCZnLdRAGoJSj63rBQ==,type:int]
...
envs/production/secrets.yaml
secrets:
elastic:
password: ENC[AES256_GCM,data:ZB/VpTFk8f0=,iv:EA//oT1Cb5wNFigTDOz3nA80qD9UwTjK5cpUwLnEXjs=,tag:hMdIUaqLRA8zuFBd82bz6A==,type:str]
...
envs/default/values/backend.yaml.gotmpl
elasticsearch:
host: elasticsearch
port: 9200
password: {{ .Values | getOrNil "secrets.elastic.password" | default "password" }}
envs/devel/values/backend.yaml
elasticsearch:
host: elastic-0.devel.domain
envs/production/values/backend.yaml
elasticsearch:
host: elastic-0.production.domain
Note
Incidentally, getOrNil
is a special function for go templates in helmfile which, even if .Values.secrets
will not exist, will not throw an error, but will allow as a result using the function default
substitute default value
Conclusion
The described things seem pretty obvious, but information on a convenient description of deploying to several environments using helmfile is very scarce, and I love IaC (Infrastructure-as-Code) and want to have a clear description of the deployment state.
In conclusion, I want to add that the variables for the default environment can, in turn, be parameterized by the environment variables of the OS of a certain runner from which the deployment will be launched, and thus get dynamic environments
helmfile.yaml
environments:
default:
values:
- global:
clusterDomain: {{ env "CLUSTER_DOMAIN" | default "cluster.local" }}
ingressDomain: {{ env "INGRESS_DOMAIN" }}
Source: habr.com