This is the second article in a series of articles that will cover limitations when downloading container images.
Π we took a closer look at the images stored in Docker Hub, the largest registry of container images. We're writing this to give you a better understanding of how our updated Terms of Service will impact development teams using Docker Hub to manage container images and CICD pipelines.
Download frequency limits were previously announced in our . We take a closer look at the frequency limits that will come into effect on November 1, 2020:
Free plan, anonymous users: 100 downloads in 6 hours
Free plan, authorized users: 200 downloads in 6 hours
Pro plan: unlimited
Team plan: unlimited
Docker download frequency is defined as the number of manifest requests to Docker Hub. Image download frequency limits depend on the type of account requesting the image, not the type of image owner account. For anonymous (unauthorized) users, the download frequency is tied to the ip-address.
Note You will receive more subtleties and best practice cases . Moreover, you can go through it when it is convenient for you - both in time and in mood.
We're getting questions from customers and the community regarding container image layers. We don't consider image layers when limiting download frequency, because we limit manifest downloads, and the number of layers (blob requests) is currently unlimited. This change is based on community feedback to make it more user friendly so users don't have to count layers on every look they use.
Detailed analysis of Docker Hub image download frequencies
We spent a lot of time analyzing the download of images from Docker Hub to determine the reason for the speed limit, as well as exactly how to limit it. What we saw confirmed that virtually all users are downloading images at a predictable rate for typical workflows. However, there is a noticeable influence of a small number of anonymous users, for example, about 30% of all downloads come from only 1% of anonymous users.
The new limits are based on this analysis, so most of our users will not be affected. These limits are made to reflect normal usage by developers - learning Docker, developing code, building images, and so on.
Helping developers to better understand download frequency limits
Now that we understood the impact, and also where the boundaries should be, we had to determine the technical conditions for the operation of these restrictions. Restricting the download of images from the Docker registry is quite difficult. You won't find an API for downloads in the registry description - it just doesn't exist. In fact, downloading an image is a combination of manifest requests and blobs in the API, and they are executed differently, depending on the state of the client and the requested image.
For example, if you already have an image, Docker Engine will issue a request for a manifest, understand that it already has all the necessary layers based on the accepted manifest, and then stop. On the other hand, if you are downloading an image that supports multiple architectures, a manifest request will return a list of image manifests for each supported architecture. The Docker Engine will then issue another manifest request for the particular architecture it is running on, in return it will get a list of all layers in the image. It will then query for each missing layer (blob).
Note This topic is covered more widely in , in which we will analyze all its tools: from basic abstractions to network parameters, the nuances of working with various operating systems and programming languages. You will get acquainted with the technology and understand where and how best to use Docker.
It turns out that downloading an image is actually one or two manifest requests, as well as from zero to infinity - requests for layers (blob). Historically, Docker has tracked download frequency on a layer-by-layer basis, as this is most related to bandwidth usage. But nevertheless, we listened to the community, which is more difficult, because you need to keep track of the requested number of layers, which will lead to ignoring best practices regarding working with the Dockerfile, and also more intuitive for users who want to just work with the registry without much understanding of the details .
So we limit the number of requests based on manifest requests. This is directly related to downloading images, which is easy for users to understand. There is really a small nuance - if you try to download an image that already exists, the request will still be taken into account, even if you do not download the layers. In any case, we hope that this method of limiting the frequency of downloads will be both fair and user-friendly.
Looking forward to your feedback
We will monitor the restrictions and make appropriate adjustments based on common use cases to ensure that the restrictions are appropriate for each type of user, and in particular, we will try to never prevent developers from doing their work.
Stay tuned in the coming weeks for another article on tweaking CI and combat systems in light of these changes.
Finally, as part of our support for the open source community, we will be providing new pricing plans for open source until November 1st. To apply, please fill out the form .
For more information about the latest changes to the terms of service, please visit .
For those who need to raise their image download frequency limits, Docker offers unlimited image downloads as a feature. . As always, we welcome feedback and questions. .
Source: habr.com