{"id":35594,"date":"2019-10-31T22:05:15","date_gmt":"2019-10-31T19:05:15","guid":{"rendered":"https:\/\/prohoster.info\/blog\/gitops-sravnenie-metodov-pull-i-push\/"},"modified":"2019-10-31T22:05:15","modified_gmt":"2019-10-31T19:05:15","slug":"gitops-sravnenie-metodov-pull-i-push","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/gitops-sravnenie-metodov-pull-i-push","title":{"rendered":"GitOps: Comparison of Pull and Push Methods","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Note: translation.<\/b>: In the Kubernetes community, a trend known as GitOps is gaining prominent popularity, which we personally witnessed, during<\/a><\/noindex> KubeCon Europe 2019. This term was relatively recently coined<\/a><\/noindex> by the head of Weaveworks \u2014 Alexis Richardson \u2014 and refers to the application of familiar tools for developers (primarily Git, from which the name derives) to solve operational tasks. Specifically, it concerns managing Kubernetes through storing its configurations in Git and automatically deploying changes to the cluster. Matthias Jg discusses the two approaches to deployment in this article.<\/i><\/p>\n

\"GitOps:
\n
\nLast year (in fact, this formally occurred in August 2017 \u2014 note from the translator)<\/i> A new approach to deploying applications in Kubernetes has emerged. It's called GitOps, based on the fundamental idea that version tracking of deployments is managed within a secure Git repository.<\/a><\/noindex><\/p>\n

The main advantages of this approach are as follows<\/b>:<\/p>\n

    \n
  1. Versioning of deployments and change history<\/b>. The entire cluster state is stored in the Git repository, and deployments are updated only through commits. Additionally, all changes can be tracked via the commit history.<\/li>\n
  2. Rollbacks using familiar Git commands<\/b>. Simple git reset<\/code> allows for rolling back changes in deployments; previous states are always accessible.<\/li>\n
  3. Ready access control<\/b>. Typically, Git systems contain a lot of sensitive data, so most companies pay special attention to securing it. Consequently, this protection also applies to deployment operations.<\/li>\n
  4. Policies for deployments<\/b>. Most Git systems initially support policies for different branches \u2014 for example, only pull requests can update the master, and changes must be reviewed and accepted by another team member. Just like access control, the same policies apply to deployment updates.<\/li>\n<\/ol>\n

    \nAs you can see, the GitOps method offers numerous advantages. Over the past year, two approaches have gained particular popularity. One is based on push, while the other is based on pull. Before we dive into them, let\u2019s first look at what typical Kubernetes deployments look like.<\/p>\n

    Deployment Methods<\/h2>\n

    \nIn recent years, various methods and tools for deployments have been established in Kubernetes:<\/p>\n

      \n
    1. Based on native Kubernetes templates\/Kustomize<\/b>This is the simplest way to deploy applications in Kubernetes. The developer creates basic YAML files and applies them. To avoid repeatedly rewriting the same templates, Kustomize was developed (it turns Kubernetes templates into modules). Note: translation.<\/b>: Kustomize has been integrated into kubectl since the release of Kubernetes 1.14.<\/a><\/noindex>.<\/i><\/li>\n
    2. Helm Charts<\/b>. Helm charts allow creating sets of templates, init containers, sidecars, etc., which provide more flexible configuration options for deploying applications than the template-based approach. This method relies on templated YAML files. Helm fills these with various parameters and then sends them to Tiller \u2014 the cluster component that deploys them in the cluster and allows for updates and rollbacks. Importantly, Helm essentially just inserts the necessary values into the templates and applies them the same way as in the traditional approach. (for more information on how it all works and how to use it, read our Helm article<\/a><\/noindex> - translator's note)<\/i>. There is a wide variety of ready-made Helm charts covering a broad range of tasks.<\/li>\n
    3. Alternative Tools<\/b>. There are many alternative tools. What unites them is that they convert certain template files into understandable Kubernetes YAML files and then apply them.<\/li>\n<\/ol>\n

      \nIn our work, we constantly use Helm charts for important tools (as much of it is pre-configured, which significantly simplifies life) and 'clean' Kubernetes YAML files for deploying our own applications.<\/p>\n

      Pull & Push<\/h2>\n

      \nIn one of my recent blog posts, I introduced a tool Weave Flux<\/a><\/noindex>, allowing templates to be committed to the Git repository and updating deployments after each commit or container push. My experience shows that this tool is one of the key elements in promoting the pull approach, so I will refer to it often. If you want to learn more about how to use it, here is a link to the article<\/a><\/noindex>.<\/p>\n

      NB!<\/b> All the benefits of using GitOps remain for both approaches.<\/i><\/p>\n

      Pull-based approach<\/h2>\n

      \n\"GitOps:
      \n
      \nThe pull approach is based on the fact that all changes are applied from inside the cluster. Inside the cluster, there is an operator that regularly checks the associated Git repositories and Docker Registry. If there are any changes, the state of the cluster is updated from within. This process is generally considered quite safe, as no external client has administrator access to the cluster.<\/p>\n

      Pros:<\/b><\/p>\n

        \n
      1. No external client has the rights to make changes to the cluster; all updates are applied from within.<\/li>\n
      2. Some tools also allow for synchronizing updates of Helm charts and linking them to the cluster.<\/li>\n
      3. Docker Registry can be scanned for new versions. If a new image appears, the Git repository and deployment are updated to the new version.<\/li>\n
      4. Pull tools can be distributed across different namespaces with different Git repositories and access rights. This allows for a multitenant model. For example, Team A can use Namespace A, Team B can use Namespace B, and the infrastructure team can use a global space.<\/li>\n
      5. Typically, tools are quite lightweight.<\/li>\n
      6. In conjunction with tools like the operator Bitnami Sealed Secrets<\/a><\/noindex>, secrets can be stored in an encrypted form in the Git repository and retrieved from within the cluster.<\/li>\n
      7. There is no connection to CD pipelines, as deployments occur within the cluster.<\/li>\n<\/ol>\n

        \nCons<\/b>:<\/p>\n

          \n
        1. Managing deployment secrets from Helm charts is more complex than regular ones, as they first need to be generated in a format like sealed secrets, then decrypted by an internal operator, and only then do they become available for the pull tool. After that, a release can be launched in Helm with values already in the deployed secrets. The simplest way is to create a secret with all the Helm values used for the deployment, decrypt it, and commit it to Git.<\/li>\n
        2. By applying the pull approach, you become tied to tools that operate on pulls. This limits the ability to configure the deployment process in the cluster. For example, working with Kustomize is complicated because it has to be executed before the final templates go to Git. I'm not saying you can't use separate tools, but they are harder to integrate into the deployment process.<\/li>\n<\/ol>\n

          <\/p>\n

          Push-Based Approach<\/h2>\n

          \n\"GitOps:
          \n
          \nIn the push approach, an external system (mainly CD pipelines) triggers deployments in the cluster after a commit to the Git repository or upon successful completion of the preceding CI pipeline. In this method, the system has access to the cluster.<\/p>\n

          Advantages<\/b>:<\/p>\n

            \n
          1. Security is determined by the Git repository and the build pipeline.<\/li>\n
          2. Deploying Helm charts is easier, with support for Helm plugins.<\/li>\n
          3. Managing secrets is simpler, as secrets can be applied in pipelines and stored in Git in encrypted form (depending on user preferences).<\/li>\n
          4. No binding to a specific tool, as any types can be used.<\/li>\n
          5. Container version updates can be initiated by the build pipeline.<\/li>\n<\/ol>\n

            \nCons<\/b>:<\/p>\n

              \n
            1. Data for accessing the cluster resides within the build system.<\/li>\n
            2. Updating container deployments is still easier to carry out with the pull process.<\/li>\n
            3. Strong dependency on the CD system, as the pipelines we need may have originally been written for GitLab Runners, and then the team decides to switch to Azure DevOps or Jenkins... leading to the necessity of migrating a large number of build pipelines.<\/li>\n<\/ol>\n

              <\/p>\n

              Conclusion: Push or Pull?<\/h2>\n

              \nAs it often happens, each approach has its pros and cons. Some tasks are easier to accomplish with one method and harder with another. Initially, I deployed manually, but after coming across several articles about Weave Flux, I decided to implement GitOps processes for all projects. For basic templates, it turned out to be simple, but then I began to face difficulties with Helm charts. At that time, Weave Flux only offered a nascent version of the Helm Chart Operator, but even now some tasks are more complex due to the need to manually create secrets and apply them. You could argue that the pull approach is much more secure since the cluster credentials are not accessible outside of it, which significantly enhances security and justifies the extra effort.<\/p>\n

              After some reflection, I came to the surprising conclusion that this is not the case. When it comes to components that require maximum protection, this list includes secret storage and CI\/CD systems, as well as Git repositories. The information within them is quite vulnerable and needs maximum protection. Moreover, if someone gains access to your Git repository and can push code there, they can deploy whatever they want (regardless of the approach taken, whether it\u2019s pull or push), and infiltrate the cluster systems. Thus, the most critical components requiring protection are the Git repository and CI\/CD systems, not the cluster credentials. If you have well-configured policies and security measures for such systems and cluster credentials are extracted in pipelines only as secrets, the additional security of the pull approach may not be as valuable as initially presumed.<\/p>\n

              So, if the pull approach is more labor-intensive and does not provide a security advantage, wouldn't it be more logical to use only the push approach? However, someone might argue that with the push approach, you are too tied to the CD system, and perhaps it\u2019s better not to do this to facilitate migrations in the future.<\/p>\n

              In my opinion (as always), it is best to use what best fits the specific case or to combine approaches. Personally, I use both approaches: Weave Flux for pull-based deployments, which mainly include our own services, and the push approach with Helm and plugins that simplify applying Helm charts to the cluster and allow for easy secret creation. I believe there will never be a single solution that fits all cases because there are always many nuances that depend on the specific application scenario. That said, I strongly recommend GitOps \u2014 it significantly simplifies life and enhances security.<\/p>\n

              I hope my experience on this topic will help you determine which method suits your type of deployments better, and I would be glad to hear your opinion.<\/p>\n

              P.S. Note from the translator<\/h2>\n

              \nOne downside of the pull model is the difficulty of committing rendered manifests to Git; however, there's no downside that the CD pipeline in the pull model lives separately from the deployment and essentially becomes a pipeline of the category Continuous Apply<\/i>. Therefore, even more effort will be needed to gather the status from all deployments and provide access to logs\/status, ideally linked to the CD system.<\/p>\n

              In this sense, the push model allows for some guarantees of rollout because the lifespan of the pipeline can be made equal to the lifespan of the rollout.<\/p>\n

              We have tested both models and arrived at the same conclusions as the author of the article:<\/p>\n

                \n
              1. The pull model is suitable for organizing updates of system components across a large number of clusters (see the article on addon-operator<\/a><\/noindex>).<\/li>\n
              2. The push model based on GitLab CI is well-suited for deploying applications using Helm charts. In this case, the deployment of applications within the pipelines is tracked using the tool werf<\/a><\/noindex>. By the way, in the context of this project, we often heard the term \"GitOps\" when discussing the pressing issues of DevOps engineers at our booth at KubeCon Europe '19.<\/li>\n<\/ol>\n

                <\/p>\n

                Our experience with data in the etcd Kubernetes cluster directly (without K8s API)<\/h2>\n

                \nAlso read in our blog:<\/p>\n