🏆 Validation of Kubernetes YAML for compliance with best practices and policies

Note: translation.: With the increasing number of YAML configurations for K8s environments, the need for automated validation is becoming more relevant. The author of this review not only selected existing solutions for this task but also examined how they work using a Deployment example. It turned out to be quite informative for those interested in this topic.

TL;DR: The article compares six static tools for validating and assessing Kubernetes YAML files for compliance with best practices and requirements.

Kubernetes workloads are typically defined as YAML documents. One issue with YAML is the complexity of setting constraints or relationships between manifest files.

What if we need to ensure that all images deployed in the cluster are sourced from a trusted registry?

How do we prevent Deployments from being sent to the cluster if they do not have PodDisruptionBudgets specified?

Integrating static testing allows for the detection of errors and policy violations at the development stage. This increases the guarantees of correctness and safety of resource definitions and enhances the likelihood that production workloads will adhere to best practices.

The ecosystem of static checks for Kubernetes YAML files can be divided into the following categories:

API validators. Tools in this category check the YAML manifest for compliance with the requirements of the Kubernetes API server.
Ready-made testers. Tools in this category come with pre-defined tests for security, compliance with best practices, etc.
Custom validators. Representatives of this category allow for the creation of custom tests in various languages, such as Rego and JavaScript.

In this article, we will describe and compare six different tools:

kubeval;
kube-score;
config-lint;
copper;
conftest;
Polaris.

Well, let’s get started!

Validating Deployments

Before we begin comparing the tools, let's create a foundation on which we will test them.

The manifest below contains a number of errors and discrepancies with best practices: how many of them can you find?

apiVersion: apps/v1
kind: Deployment
metadata:
  name: http-echo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: http-echo
  template:
    metadata:
      labels:
        app: http-echo
    spec:
      containers:
      - name: http-echo
        image: hashicorp/http-echo
        args: ["-text", "hello-world"]
        ports:
        - containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
  name: http-echo
spec:
  ports:
  - port: 5678
    protocol: TCP
    targetPort: 5678
  selector:
    app: http-echo

(base-valid.yaml)

We will use this YAML to compare different tools.

The manifest above base-valid.yaml and other manifests from this article can be found at Git repository.

The manifest describes a web application whose main task is to respond with the message "Hello World" on port 5678. It can be deployed with the following command:

kubectl apply -f hello-world.yaml

And to check its operation:

kubectl port-forward svc/http-echo 8080:5678

Now go to http://localhost:8080 and confirm that the application is running. But does it follow the best practices? Let's check.

1. Kubeval

At its core, kubeval is based on the idea that any interaction with Kubernetes occurs through its REST API. In other words, you can use the API schema to verify whether a given YAML conforms to it. Let’s look at an example.

Installation instructions for kubeval are available on the project's website.

At the time of writing the original article, version 0.15.0 was available.

After installation, let's "feed" it the manifest provided above:

$ kubeval base-valid.yaml
PASS - base-valid.yaml contains a valid Deployment (http-echo)
PASS - base-valid.yaml contains a valid Service (http-echo)

If successful, kubeval will exit with code 0. You can check it like this:

$ echo $?
0

Now let's try kubeval with another manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: http-echo
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: http-echo
    spec:
      containers:
      - name: http-echo
        image: hashicorp/http-echo
        args: ["-text", "hello-world"]
        ports:
        - containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
  name: http-echo
spec:
  ports:
  - port: 5678
    protocol: TCP
    targetPort: 5678
  selector:
    app: http-echo

(kubeval-invalid.yaml)

Can you spot the problem at a glance? Let's run:

$ kubeval kubeval-invalid.yaml
WARN - kubeval-invalid.yaml contains an invalid Deployment (http-echo) - selector: selector is required
PASS - kubeval-invalid.yaml contains a valid Service (http-echo)

# check the exit code
$ echo $?
1

The resource fails validation.

Deployments using the API version apps/v1, must include a selector that matches the pod's label. The manifest above does not include a selector, so kubeval reported an error and exited with a non-zero code.

I wonder what will happen if we run it kubectl apply -f with this manifest?

Well, let’s give it a try:

$ kubectl apply -f kubeval-invalid.yaml
error: error validating "kubeval-invalid.yaml": error validating data: ValidationError(Deployment.spec):
missing required field "selector" in io.k8s.api.apps.v1.DeploymentSpec; if you choose to ignore these errors,
turn validation off with --validate=false

This is exactly the error that kubeval warned about. You can fix it by adding a selector:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: http-echo
spec:
  replicas: 2
  selector:          # !!!
    matchLabels:     # !!!
      app: http-echo # !!!
  template:
    metadata:
      labels:
        app: http-echo
    spec:
      containers:
      - name: http-echo
        image: hashicorp/http-echo
        args: ["-text", "hello-world"]
        ports:
        - containerPort: 5678
---
apiVersion: v1
kind: Service
metadata:
  name: http-echo
spec:
  ports:
  - port: 5678
    protocol: TCP
    targetPort: 5678
  selector:
    app: http-echo

(base-valid.yaml)

The advantage of tools like kubeval is that such errors can be caught early in the deployment cycle.

Additionally, these checks do not require access to the cluster: they can be performed offline.

By default, kubeval checks resources against the latest version of the Kubernetes API schema. However, in most cases, you may need to validate against a specific Kubernetes release. This can be done using the flag --kubernetes-version:

$ kubeval --kubernetes-version 1.16.1 base-valid.yaml

Note that the version should be specified in the format Major.Minor.Patch.

To view the list of supported versions for validation, refer to the JSON schema on GitHub, which kubeval uses for validation. If you need to run kubeval offline, download the schemas and specify their local location using the flag --schema-location.

In addition to individual YAML files, kubeval can also work with directories and stdin.

Moreover, Kubeval easily integrates into the CI pipeline. Those wishing to conduct tests before submitting manifests to the cluster will be pleased to know that kubeval supports three output formats:

Plain text;
JSON;
Test Anything Protocol (TAP).

And any of the formats can be used for further parsing of the output to create a summary of results in the desired format.

One drawback of kubeval is that it currently does not validate against Custom Resource Definitions (CRDs). However, you can configure kubeval to ignore them.

Kubeval is an excellent tool for checking and evaluating resources; however, it should be emphasized that passing the test does not guarantee that the resource adheres to best practices.

For example, using a tag latest The container does not comply with best practices. However, kubeval does not consider this an error and does not report it. This means that the validation of such YAML will finish without warnings.

But what if you need to evaluate YAML and identify violations like the tag latest? Как проверить YAML-файл на соответствие лучшим практикам?

2. Kube-score

Kube-score analyzes YAML manifests and evaluates them against built-in tests. These tests are selected based on security recommendations and best practices, such as:

Running containers not as root.
Having health checks for pods.
Setting resource requests and limits.

At the end of the test, three results are issued: OK, WARNING and CRITICAL.

You can try Kube-score online or install it locally.

As of the writing of the original article, the latest version of kube-score was 1.7.0.

Let's test it on our manifest base-valid.yaml:

$ kube-score score base-valid.yaml

apps/v1/Deployment http-echo
[CRITICAL] Container Image Tag
  · http-echo -> Image with latest tag
      Using a fixed tag is recommended to avoid accidental upgrades
[CRITICAL] Pod NetworkPolicy
  · The pod does not have a matching network policy
      Create a NetworkPolicy that targets this pod
[CRITICAL] Pod Probes
  · Container is missing a readinessProbe
      A readinessProbe should be used to indicate when the service is ready to receive traffic.
      Without it, the Pod is risking to receive traffic before it has booted. It is also used during
      rollouts, and can prevent downtime if a new version of the application is failing.
      More information: https://github.com/zegl/kube-score/blob/master/README_PROBES.md
[CRITICAL] Container Security Context
  · http-echo -> Container has no configured security context
      Set securityContext to run the container in a more secure context.
[CRITICAL] Container Resources
  · http-echo -> CPU limit is not set
      Resource limits are recommended to avoid resource DDOS. Set resources.limits.cpu
  · http-echo -> Memory limit is not set
      Resource limits are recommended to avoid resource DDOS. Set resources.limits.memory
  · http-echo -> CPU request is not set
      Resource requests are recommended to make sure that the application can start and run without
      crashing. Set resources.requests.cpu
  · http-echo -> Memory request is not set
      Resource requests are recommended to make sure that the application can start and run without crashing.
      Set resources.requests.memory
[CRITICAL] Deployment has PodDisruptionBudget
  · No matching PodDisruptionBudget was found
      It is recommended to define a PodDisruptionBudget to avoid unexpected downtime during Kubernetes
      maintenance operations, such as when draining a node.
[WARNING] Deployment has host PodAntiAffinity
  · Deployment does not have a host podAntiAffinity set
      It is recommended to set a podAntiAffinity that stops multiple pods from a deployment from
      being scheduled on the same node. This increases availability in case the node becomes unavailable.

YAML passes kubeval checks, while kube-score points out the following shortcomings:

Readiness checks are not configured.
Resource requests and limits for CPU and memory are missing.
Pod disruption budgets are not set.
Anti-affinity rules are absent. (anti-affinity) to maximize availability.
The container runs as root.

All of these are reasonable observations about the shortcomings that need to be addressed for Deployment to be more efficient and reliable.

The command kube-score outputs information in a human-readable format including all types of violations WARNING and CRITICAL, which is very helpful during development.

Those wishing to use this tool in a CI pipeline can enable more concise output using the flag --output-format ci (in this case, tests with results are also displayed OK):

$ kube-score score base-valid.yaml --output-format ci

[OK] http-echo apps/v1/Deployment
[OK] http-echo apps/v1/Deployment
[CRITICAL] http-echo apps/v1/Deployment: (http-echo) CPU limit is not set
[CRITICAL] http-echo apps/v1/Deployment: (http-echo) Memory limit is not set
[CRITICAL] http-echo apps/v1/Deployment: (http-echo) CPU request is not set
[CRITICAL] http-echo apps/v1/Deployment: (http-echo) Memory request is not set
[CRITICAL] http-echo apps/v1/Deployment: (http-echo) Image with latest tag
[OK] http-echo apps/v1/Deployment
[CRITICAL] http-echo apps/v1/Deployment: The pod does not have a matching network policy
[CRITICAL] http-echo apps/v1/Deployment: Container is missing a readinessProbe
[CRITICAL] http-echo apps/v1/Deployment: (http-echo) Container has no configured security context
[CRITICAL] http-echo apps/v1/Deployment: No matching PodDisruptionBudget was found
[WARNING] http-echo apps/v1/Deployment: Deployment does not have a host podAntiAffinity set
[OK] http-echo v1/Service
[OK] http-echo v1/Service
[OK] http-echo v1/Service
[OK] http-echo v1/Service

Similar to kubeval, kube-score returns a non-zero exit code when there is a test that ends in an error. CRITICAL. This similar processing can also be enabled for WARNING.

Moreover, there is the option to check resources for compliance with different API versions (as in kubeval). However, this information is hardcoded in kube-score: you cannot choose a different version of Kubernetes. This limitation can become a significant issue if you plan to upgrade your cluster or have multiple clusters with different K8s versions.

Note that there is already an issue suggesting to implement this possibility.

You can learn more about kube-score at the official website.

Kube-score tests are an excellent tool for implementing best practices, but what if you need to modify a test or add your own rules? Unfortunately, this cannot be done.

Kube-score is not extensible: you cannot add or customize policies.

If you need to write custom tests to check compliance with the policies adopted in your company, you can use one of the following four tools: config-lint, copper, conftest, or polaris.

3. Config-lint

Config-lint is a tool for validating configuration files in YAML, JSON, Terraform, CSV formats, and Kubernetes manifests.

It can be installed using the instructions on the project's website.

The current release as of the writing of this article is 1.5.0.

Config-lint does not have built-in tests for validating Kubernetes manifests.

To run any tests, corresponding rules need to be created. They are recorded in YAML files called "rulesets" (rulesets), and have the following structure:

version: 1
description: Rules for Kubernetes spec files
type: Kubernetes
files:
  - "*.yaml"
rules:
   # list of rules

(rule.yaml)

Let's examine it more closely:

Field type specifies what type of configuration will be used by config-lint. For K8s manifests, it is recommendations only Kubernetes.
In the field files in addition to the files, a directory can also be specified.
Field rules is intended for defining custom tests.

Suppose you want to ensure that images in a Deployment are always downloaded from a trusted repository like my-company.com/myapp:1.0. The rule for config-lint that performs this check would look as follows:

- id: MY_DEPLOYMENT_IMAGE_TAG
  severity: FAILURE
  message: Deployment must use a valid image tag
  resource: Deployment
  assertions:
    - every:
        key: spec.template.spec.containers
        expressions:
          - key: image
            op: starts-with
            value: "my-company.com/"

(rule-trusted-repo.yaml)

Each rule must specify the following attributes:

id — a unique identifier for the rule;
severity — can be FAILURE, WARNING and NON_COMPLIANT;
message — if the rule is violated, this line's content will be displayed;
resource — the type of resource to which this rule applies;
assertions — a list of conditions that will be evaluated against this resource.

In the rule above assertion called every checks that all containers in the Deployment (key: spec.templates.spec.containers) use trusted images (i.e., those starting with my-company.com/).

The complete ruleset looks as follows:

version: 1
description: Rules for Kubernetes spec files
type: Kubernetes
files:
  - "*.yaml"
rules:

 - id: DEPLOYMENT_IMAGE_REPOSITORY # !!!
    severity: FAILURE
    message: Deployment must use a valid image repository
    resource: Deployment
    assertions:
      - every:
          key: spec.template.spec.containers
          expressions:
            - key: image
              op: starts-with
              value: "my-company.com/"

(ruleset.yaml)

To test the rule, let's save it as check_image_repo.yaml. Let's run the check against the file. base-valid.yaml:

$ config-lint -rules check_image_repo.yaml base-valid.yaml

[
  {
  "AssertionMessage": "Every expression fails: And expression fails: image does not start with my-company.com/",
  "Category": "",
  "CreatedAt": "2020-06-04T01:29:25Z",
  "Filename": "test-data/base-valid.yaml",
  "LineNumber": 0,
  "ResourceID": "http-echo",
  "ResourceType": "Deployment",
  "RuleID": "DEPLOYMENT_IMAGE_REPOSITORY",
  "RuleMessage": "Deployment must use a valid image repository",
  "Status": "FAILURE"
  }
]

The check was unsuccessful. Now let's check the next manifest with a valid image repository:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: http-echo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: http-echo
  template:
    metadata:
      labels:
        app: http-echo
    spec:
      containers:
      - name: http-echo
         image: my-company.com/http-echo:1.0 # !!!
         args: ["-text", "hello-world"]
         ports:
         - containerPort: 5678

(image-valid-mycompany.yaml)

Running the same test with the manifest provided above. No issues found:

$ config-lint -rules check_image_repo.yaml image-valid-mycompany.yaml
[]

Config-lint is a promising framework that allows you to create your own tests to validate Kubernetes YAML manifests using YAML DSL.

But what if you need more complex logic and tests? Is YAML's capability not too limited for this? What if you could create tests using a full programming language?

4. Copper

Copper V2 is a framework for validating manifests using custom tests (similar to config-lint).

However, it differs from the latter in that it does not use YAML to describe tests. Instead, tests can be created in JavaScript. Copper provides a library with several basic tools, helping to read information about Kubernetes objects and report errors.

The sequence of steps for installing Copper can be found in the official documentation..

2.0.1 is the most recent release of this utility at the time of writing the original article.

Like config-lint, Copper does not have built-in tests. Let's write one. It should validate that deployments use container images exclusively from trusted repositories like my-company.com.

Create a file check_image_repo.js with the following content:

$$.forEach(function($){
    if ($.kind === 'Deployment') {
        $.spec.template.spec.containers.forEach(function(container) {
            var image = new DockerImage(container.image);
            if (image.registry.lastIndexOf('my-company.com/') != 0) {
                errors.add_error('no_company_repo',"Image " + $.metadata.name + " is not from my-company.com repo", 1)
            }
        });
    }
});

Now, to validate our manifest base-valid.yaml, use the command copper validate:

$ copper validate --in=base-valid.yaml --validator=check_image_tag.js

Check no_company_repo failed with severity 1 due to Image http-echo is not from my-company.com repo
Validation failed

It is clear that with copper, more complex tests can be conducted — for example, checking domain names in Ingress manifests or rejecting pods running in privileged mode.

Copper comes with various utility functions:

DockerImage reads the specified input file and creates an object with the following attributes:
- name — image name,
- tag — image tag,
- registry — image registry,
- registry_url — protocol (https://) and image registry,
- fqin — full image location.
Function findByName helps to find a resource by the specified type (kind) and name (name) from the input file.
Function findByLabels helps to find a resource by the specified type (kind) and labels (labels).

You can check all available utility functions here.

By default, it loads the entire input YAML file into a variable $$ and makes it available for scripts (a familiar method for those with jQuery experience).

The main advantage of Copper is obvious: you don't need to learn a specialized language and can use various JavaScript capabilities to create your own tests, such as string interpolation, functions, etc.

It should also be noted that the current version of Copper works with the ES5 version of the JavaScript engine, not ES6.

Details are available at official project website.

However, if you are not very fond of JavaScript and prefer a language specifically designed for making queries and describing policies, you should look into conftest.

5. Conftest

Conftest is a framework for validating configuration data. It is also suitable for testing/verifying Kubernetes manifests. Tests are described using a specialized query language Rego.

You can install conftest using the instructions, as described on the project's website.

At the time of writing the original article, the latest available version was 0.18.2.

Similarly to config-lint and copper, conftest comes without any built-in tests. Let's try it out and write our own policy. As in previous examples, we will check whether container images are pulled from a trusted source.

Create a directory conftest-checks, and within it, create a file named check_image_registry.rego with the following content:

package main

deny[msg] {

  input.kind == "Deployment"
  image := input.spec.template.spec.containers[_].image
  not startswith(image, "my-company.com/")
  msg := sprintf("image '%v' doesn't come from my-company.com repository", [image])
}

Now let's test base-valid.yaml via conftest:

$ conftest test --policy ./conftest-checks base-valid.yaml

FAIL - base-valid.yaml - image 'hashicorp/http-echo' doesn't come from my-company.com repository
1 tests, 1 passed, 0 warnings, 1 failure

The test has failed as expected because the images come from an untrusted source.

In the Rego file, we define a block deny. Its truth is considered a violation. If there are multiple blocks, conftest checks them independently, and the truth of any of the blocks is interpreted as a violation. deny In addition to default output, conftest supports JSON, TAP, and table formats—an extremely useful feature if you need to integrate reports into an existing CI pipeline. You can set the desired format using the flag

--output To aid in policy debugging, conftest has a flag.

--trace . It outputs a trace of how conftest parses the specified policy files.Policies in conftest can be published and shared in OCI registries (Open Container Initiative) as artifacts.

They allow you to publish an artifact or retrieve an existing artifact from a remote registry. Let's try to publish the policy we created to the local Docker registry using

Commands push and pull conftest push Run a local Docker registry:.

$ docker run -it --rm -p 5000:5000 registry

In another terminal, navigate to the previously created directory

$ conftest push 127.0.0.1:5000/amitsaha/opa-bundle-example:latest conftest-checks and execute the following command:

If the command was successful, you will see a message like this:

2020/06/10 14:25:43 pushed bundle with digest: sha256:e9765f201364c1a8a182ca637bc88201db3417bacc091e7ef8211f6c2fd2609c

Now create a temporary directory and execute the command

conftest pull . It will download the package created by the previous command into it:$ cd $(mktemp -d) $ conftest pull 127.0.0.1:5000/amitsaha/opa-bundle-example:latest

In the temporary directory, a subdirectory will appear

policy , containing our policy file:$ tree . └── policy └── check_image_registry.rego

Tests can be conducted directly from the repository:

$ conftest test --update 127.0.0.1:5000/amitsaha/opa-bundle-example:latest base-valid.yaml .. FAIL - base-valid.yaml - image 'hashicorp/http-echo' doesn't come from my-company.com repository 2 tests, 1 passed, 0 warnings, 1 failure

Unfortunately, DockerHub is not supported yet. So consider yourself lucky if you are using

Azure Container Registry (ACR) or your own registry. The artifact format is the same as that of

Open Policy Agent (OPA) packages, allowing conftest to run tests from existing OPA packages. Learn more about sharing policies and other features of conftest at

6. Polaris official project website.

The last tool to be discussed in this article is

(We already translated its announcement last year Polaris. note. translator. . — translator's note.)

Polaris can be installed in a cluster or used in command-line mode. As you may have guessed, it allows for static analysis of Kubernetes manifests.

When working in command-line mode, built-in tests are available that cover areas such as security and best practices (similar to kube-score). Additionally, you can create your own tests (like in config-lint, copper, and conftest).

In other words, Polaris combines the advantages of both categories of tools: with built-in and custom tests.

To install Polaris in command-line mode, use the instructions on the project's website..

At the time of writing the original article, version 1.0.3 was available.

After the installation completes, you can run polaris on the manifest base-valid.yaml using the following command:

$ polaris audit --audit-path base-valid.yaml

It will output a string in JSON format with a detailed description of the tests performed and their results. The output will have the following structure:

{
  "PolarisOutputVersion": "1.0",
  "AuditTime": "0001-01-01T00:00:00Z",
  "SourceType": "Path",
  "SourceName": "test-data/base-valid.yaml",
  "DisplayName": "test-data/base-valid.yaml",
  "ClusterInfo": {
    "Version": "unknown",
    "Nodes": 0,
    "Pods": 2,
    "Namespaces": 0,
    "Controllers": 2
  },
  "Results": [
    /* long list */
  ]
}

The full output is available here.

Like kube-score, Polaris identifies issues in areas where the manifest does not comply with best practices:

Health checks for pods are missing.
No tags are specified for container images.
The container runs as root.
Requests and limits for memory and CPU are not specified.

Each test is assigned a severity level based on its results: warning or danger. To learn more about the available built-in tests, refer to the documentation.

If details are not needed, you can specify the flag --format score. In this case, Polaris will output a number in the range from 1 to 100 — score i.e., the rating:

$ polaris audit --audit-path test-data/base-valid.yaml --format score
68

The closer the score is to 100, the higher the level of compliance. If you check the exit code of the command polaris audit, you will find that it is equal to 0.

To make polaris audit exit with a non-zero code, you can use two flags:

Flag --set-exit-code-below-score takes a threshold value as an argument in the range of 1-100. In this case, the command will exit with code 4 if the score is below the threshold. This is very convenient when you have a certain threshold (say, 75), and you need to receive an alert if the score drops below it.
Flag --set-exit-code-on-danger will lead to the team finishing with code 3 if any of the danger tests fail.

Now let's try to create a custom test that checks whether the image is taken from a trusted repository. Custom tests are defined in YAML format, and the test itself is described using JSON Schema.

The following YAML code snippet describes a new test called checkImageRepo:

checkImageRepo:
  successMessage: Image registry is valid
  failureMessage: Image registry is not valid
  category: Images
  target: Container
  schema:
    '$schema': http://json-schema.org/draft-07/schema
    type: object
    properties:
      image:
        type: string
        pattern: ^my-company.com/.+$

Let's take a closer look at it:

successMessage — this line will be output if the test is successful;
failureMessage — this message will be shown in case of failure;
category — indicates one of the categories: Images, Health Checks, Security, Networking and Resources;
target— defines the type of object to which the test applies (spec) Possible values: Container, Pod or Controller;
The test itself is defined in the object schema using JSON schema. In this test, the keyword pattern is used to compare the image source with the required format.

To run the above test, you need to create the following Polaris configuration:

checks:
  checkImageRepo: danger
customChecks:
  checkImageRepo:
    successMessage: Image registry is valid
    failureMessage: Image registry is not valid
    category: Images
    target: Container
    schema:
      '$schema': http://json-schema.org/draft-07/schema
      type: object
      properties:
        image:
          type: string
          pattern: ^my-company.com/.+$

(polaris-conf.yaml)

Let's break down the file:

In the field checks define the tests and their severity level. Since it is desirable to receive a warning when the image is taken from an untrusted source, we set the severity level here danger.
The test checkImageRepo is then defined in the object customChecks.

Save the file as custom_check.yaml. Now you can run polaris audit with the YAML manifest that requires checking.

Let's test our manifest base-valid.yaml:

$ polaris audit --config custom_check.yaml --audit-path base-valid.yaml

The command polaris audit ran only the custom test specified above, and it did not succeed.

If you fix the image to my-company.com/http-echo:1.0, Polaris will finish successfully. The manifest with changes is already in the repository, so you can check the previous command on the manifest. image-valid-mycompany.yaml.

Now the question arises: how to run built-in tests alongside custom ones? It's easy! Just add the IDs of the built-in tests to the configuration file. As a result, it will look like this:

checks:
  cpuRequestsMissing: warning
  cpuLimitsMissing: warning
  # Other inbuilt checks..
  # ..
  # custom checks
  checkImageRepo: danger # !!!
customChecks:
  checkImageRepo:        # !!!
    successMessage: Image registry is valid
    failureMessage: Image registry is not valid
    category: Images
    target: Container
    schema:
      '$schema': http://json-schema.org/draft-07/schema
      type: object
      properties:
        image:
          type: string
          pattern: ^my-company.com/.+$

(config_with_custom_check.yaml)

An example of a complete configuration file is available here.

Check manifest base-valid.yaml, using built-in and custom tests can be done by running the command:

$ polaris audit --config config_with_custom_check.yaml --audit-path base-valid.yaml

Polaris augments built-in tests with custom ones, thus combining the best of both worlds.

On the other hand, the inability to use more powerful languages like Rego or JavaScript may be a limiting factor in creating more sophisticated tests.

Additional information about Polaris is available at project site.

Summary

While there are many tools for validating and auditing Kubernetes YAML files, it is important to have a clear understanding of how tests will be designed and executed.

For example, if you take Kubernetes manifests going through a pipeline, kubeval could be the first step in such a pipeline. It would check whether object definitions conform to the Kubernetes API schema.

After completing such a check, one could move on to more sophisticated tests, such as compliance with best practices and specific policies. This is where kube-score and Polaris would come in handy.

For those with complex requirements and the need to fine-tune tests, copper, config-lint, and conftest would be suitable..

Conftest and config-lint use YAML to specify custom tests, while copper provides access to a full programming language, making it quite an attractive choice.

On the other hand, should one take advantage of one of these tools and thus create all tests manually, or prefer Polaris and just add what is needed? There is no clear answer to this question..

The table below provides a brief description of each tool:

Tool
The purpose
Disadvantages
Custom tests

kubeval
Checks YAML manifests against a specific version of the API schema
Cannot work with CRD
No

kube-score
Analyzes YAML manifests for compliance with best practices
Cannot select your version of the Kubernetes API for resource checks
No

copper
A general framework for creating custom JavaScript tests for YAML manifests
No built-in tests. Sparse documentation
Yes

config-lint
A general framework for creating tests in a domain-specific language embedded in YAML. Supports various configuration formats (e.g., Terraform)
No ready-made tests. Built-in assertions and functions may be insufficient
Yes

conftest
A framework for creating custom tests in Rego (a specialized query language). Enables sharing policies through OCI bundles
No built-in tests. You need to learn Rego. Docker Hub is not supported for publishing policies
Yes

Polaris
Analyzes YAML manifests for compliance with standard best practices. Allows creating custom tests using JSON Schema
Capabilities for tests based on JSON Schema may be inadequate
Yes

Since these tools do not depend on access to a Kubernetes cluster, they are easy to install. They allow filtering of source files and provide quick feedback to authors of pull requests in projects.