{"id":37667,"date":"2019-10-31T22:18:58","date_gmt":"2019-10-31T19:18:58","guid":{"rendered":"https:\/\/prohoster.info\/blog\/podderzhka-monorepo-i-multirepo-v-werf-i-pri-chyom-zdes-docker-registry\/"},"modified":"2019-10-31T22:18:58","modified_gmt":"2019-10-31T19:18:58","slug":"podderzhka-monorepo-i-multirepo-v-werf-i-pri-chyom-zdes-docker-registry","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/podderzhka-monorepo-i-multirepo-v-werf-i-pri-chyom-zdes-docker-registry","title":{"rendered":"Support for monorepo and multirepo in werf and its connection to Docker Registry","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Support
\n
\nThe topic of monorepositories has been discussed numerous times and typically sparks quite active debates. When creating werf<\/a><\/noindex> as an Open Source tool aimed at improving the process of building application code from Git into Docker images (and their subsequent delivery to Kubernetes), we rarely ponder which choice is better. Our primary goal is to provide everything necessary for supporters of different opinions (as long as it doesn't contradict common sense, of course).<\/p>\n

The recently introduced support for mono-repo in werf is a good example of this. But first, let's clarify how this support is related to using werf and what Docker Registry has to do with it\u2026<\/a><\/noindex><\/p>\n

Issues<\/h2>\n

\nImagine this situation. In a company, there are numerous development teams working on independent projects. Most applications operate in Kubernetes, meaning they are containerized. For storing containers and images, a registry is required. The company uses Docker Hub with a single account COMPANY<\/code>. Similar to most source code management systems, Docker Hub does not allow the creation of a nested hierarchy of repositories<\/b>, such as COMPANY\/PROJECT\/IMAGE<\/code>. In this case\u2026 how do you store non-monolithic applications in the registry without creating a separate account for each project? <\/p>\n

\"Support
\n
\nPerhaps the situation described is familiar to some, but let's consider the question of organizing application storage in general, i.e., without tying it to the example and Docker Hub mentioned above.<\/p>\n

Solutions<\/h3>\n

\nIf the application is monolithic<\/b>, there are no issues, and we simply store the images in the project's container registry.<\/p>\n

When the application is represented by several components, microservices<\/b>, a specific approach is required. Taking the example of a typical web application consisting of two images: frontend<\/code> and backend<\/code> \u2014 possible options are:<\/p>\n

    \n
  1. Store images in separate nested repositories:\n

    \"Support<\/li>\n

  2. Store everything in one repository, considering the image name in the tag, for example, as follows:\n

    \"Support<\/li>\n<\/ol>\n

    \nNB<\/b>: Actually, there is another option for saving in different repositories, PROJECT-frontend<\/code> and PROJECT-backend<\/code>, but we will not consider it due to the complexity of support, organization, and distribution of rights among users.<\/i><\/p>\n

    Support in werf<\/h2>\n

    \nInitially, werf was limited to nested repositories \u2014 fortunately, most registries support this feature. Starting from version v1.0.4-alpha.3<\/a><\/noindex>, support for registries that do not support nesting<\/b>, including Docker Hub, has been added. From this moment on, users have a choice of how to store application images.<\/p>\n

    The implementation is available under the option --images-repo-mode=multirepo|monorepo<\/code> (by default, multirepo<\/code>, i.e., storage in nested repositories). It defines the patterns by which images are stored in the registry. Simply choose the desired mode when using the main commands, and everything else will remain unchanged.<\/p>\n

    Since most werf options can be set via environment variables<\/b>, in CI\/CD systems, the storage mode can usually be easily specified globally for the entire project. For example, in the case of GitLab<\/b> , it is sufficient to add an environment variable in the project settings: Settings -> CI \/ CD -> Variables: WERF_IMAGES_REPO_MODE: multirepo|monorepo<\/code><\/i>.<\/p>\n

    When it comes to publishing images and deploying applications (you can read in detail about these processes in the relevant sections of the documentation: Publish process<\/a><\/noindex> and Deploy process<\/a><\/noindex>), the mode merely defines the template for working with the image.<\/p>\n

    The devil is in the details.<\/h3>\n

    \nThe distinction and primary difficulty in adding a new storage method lies in the process of cleaning the registry (for cleaning capabilities supported in werf, see Cleaning process<\/a><\/noindex>)<\/i>.<\/p>\n

    ). When cleaning, werf takes into account the images used in Kubernetes clusters, as well as the policies configured by the user. The policies are based on the categorization of tags into strategies. The currently supported strategies are:<\/p>\n

      \n
    1. 3 strategies related to Git primitives such as tag, branch, and commit;<\/li>\n
    2. 1 strategy for arbitrary user-defined tags.<\/li>\n<\/ol>\n

      \nWe save information about the tag strategy when publishing the image in the labels of the final image. The value itself \u2014 the so-called meta tag<\/b> \u2014 is necessary for applying part of the policies. For example, when deleting a branch or tag from a Git repository, it makes sense to also delete any associated unused<\/i> images from the registry, which is covered by part of our policies.<\/p>\n

      When stored in a single repository (monorepo<\/code>), within the image tag, besides the meta tag, the image name can also be stored: PROJECT:frontend<\/b>-META-TAG<\/code>. To separate them, we did not introduce any specific delimiter, but simply added the necessary value to the label of the final image at publication.<\/p>\n

      NB<\/b>: If you are interested in seeing everything described in the source code of werf, you can start at PR 1684<\/a><\/noindex>.<\/i><\/p>\n

      In this article, we will not focus further on the issues and justification of our approach: regarding tagging strategies, data storage in labels, and the publishing process as a whole\u2014everything about this is detailed in Dmitry Stolyarov's recent report: \"werf is our tool for CI\/CD in Kubernetes<\/a><\/noindex>\u00bb.<\/p>\n

      In summary<\/h2>\n

      \nThe lack of support for registry without nesting was not a blocking factor for us or the known users of werf\u2014since it is always possible to set up a separate image registry (or switch to the hypothetical Container Registry in Google Cloud)\u2026 However, removing such a limitation seemed logical in order for the tool to be more convenient for a broader DevOps community. In implementing this, we faced the main challenge of reworking the container registry cleanup mechanism. Now that everything is ready, it\u2019s nice to realize that it has become easier for someone, and we (as the main developers of the project) do not foresee significant difficulties in further supporting this feature.<\/p>\n

      Stay with us, and soon we will tell you about other innovations in werf<\/a><\/noindex>!<\/p>\n

      P.S.<\/h2>\n

      \nAlso read in our blog:<\/p>\n