This is the second article in a series; it will discuss the limitations when downloading container images.
In We have detailed the images stored in Docker Hub, the largest container image registry. We write about this to help you understand how our updated Terms of Service will affect developer teams using Docker Hub for managing container images and CI/CD pipelines.
The download rate limitations were announced earlier in our . We will take a closer look at the limitations on download rates that will take effect on November 1, 2020:
Free plan, anonymous users: 100 downloads every 6 hours
Free plan, authorized users: 200 downloads every 6 hours
Pro plan: no limits
Team plan: no limits
The download rate from Docker is defined as the number of manifest requests to Docker Hub. Download rate limitations depend on the type of account requesting the image, not the type of account of the image owner. For anonymous (unauthorized) users, the download rate is tied to the IP address.
N.B. You will get more insights and best practice cases . You can take it at your convenience—both in timing and mood.
We receive questions from clients and the community regarding image layers. We do not count image layers against the download rate because we restrict downloading manifests, and the number of layers (blob requests) is currently unlimited. This change is based on community feedback to make it more user-friendly, so users don’t need to count layers on each image they use.
A detailed analysis of the download rates of Docker Hub images
We spent a lot of time analyzing the downloading of images from Docker Hub to determine the cause of the speed limitation, as well as how exactly to impose such limitations. What we observed confirmed that virtually all users download images at a predictable speed for typical workflows. However, there is a noticeable impact from a small number of anonymous users; for example, about 30% of all downloads come from just 1% of anonymous users.
The new limitations are based on this analysis, so most of our users will not be affected. These limitations are designed to reflect typical usage patterns among developers — exploring Docker, writing code, creating images, etc.
Helping developers better understand download rate limiting
Now that we have understood the impact as well as where the boundaries should lie, we needed to define the technical conditions for these limitations. Limiting the download of images from the Docker registry is quite challenging. You won’t find an API for uploads mentioned in the registry description — it simply doesn’t exist. In fact, downloading an image consists of a combination of manifest and blob requests in the API, and they are executed differently depending on the client's state and the image being requested.
For example, if you already have an image, the Docker Engine will issue a manifest request, understand that it already has all the necessary layers based on the received manifest, and then stop. On the other hand, if you are downloading an image that supports multiple architectures, the manifest request will return a list of image manifests for each supported architecture. The Docker Engine will then issue another manifest request for the specific architecture it’s running on, in response receiving a list of all the image layers. It will then request each missing layer (blob).
N.B. This topic is explored more broadly in , where we will cover all its tools: from basic abstractions to network parameters, nuances of working with various operating systems, and programming languages. You will get acquainted with the technology and understand where and how best to use Docker.
It turns out that downloading an image is actually one or two manifest requests, as well as anywhere from zero to infinity—requests for layers (blobs). Historically, Docker tracked download frequency based on layers since this is most related to bandwidth usage. However, we've listened to the community that this can be more complicated, as it requires tracking the number of requested layers, which can lead to ignoring best practices regarding working with Dockerfile, as well as being less intuitive for users who simply want to work with the registry without getting into the details.
So we are limiting the number of requests based on manifest requests. This is directly related to image downloads, which is easy for users to understand. There is a small nuance—if you try to download an image that you already have, the request will still be counted, even if you won't be downloading layers. In any case, we hope that this method of limiting download frequency will be both fair and user-friendly.
We welcome your feedback.
We will monitor the limitations and make appropriate adjustments based on typical use cases to ensure that the restrictions fit every type of user, and in particular, we will strive to never interfere with developers carrying out their work.
Stay tuned for updates in the coming weeks; there will be another article about setting up CI and production systems in light of these changes.
Finally, as part of our support for the open-source developer community, by November 1 we will provide new pricing plans for open-source projects. To apply, you need to fill out a form. .
For more information on the latest changes to the terms of service, please refer to .
For those who need to lift the download frequency limits, Docker offers unlimited image downloads as a feature of As always, we look forward to your feedback and questions. .
Source: habr.com
