Note: translation.: American DevOps engineer Sid Palas, utilizing as a point of reference, conducted a cost comparison of Managed Kubernetes services (in various configurations) from leading global cloud providers. An additional benefit of his work was the publication of a corresponding Jupyter Notebook, allowing users (with minimal Python knowledge) to adjust the calculations to their needs.
TL;DR: Azure and Digital Ocean do not charge for the compute resources used for the control plane, making them suitable choices for deploying multiple small clusters. For launching a small number of large clusters, GKE is the best fit. Furthermore, significant savings can be achieved by using spot/preemptible/low-priority nodes or by 'committing' to long-term use of the same nodes (this applies to all platforms).
Cluster size (number of workers)
General information
about the introduction of a charge of 10 cents per hour for each cluster in GKE prompted me to analyze the prices of key offerings in managed Kubernetes.
This announcement upset some...
The main subjects of the article are:
- Google Kubernetes Engine (GCP) – ;
- Elastic Kubernetes Service (AWS) – ;
- Azure Kubernetes Service (Azure) – ;
- Kubernetes on Digital Ocean – .
Cost breakdown
The total costs of using Kubernetes on each of these platforms consist of the following components:
- Management fee for the cluster;
- Load balancing (for Ingress);
- Compute resources (vCPU and memory) of workers;
- Egress traffic;
- Persistent storage;
- Data processing by the load balancer.
Additionally, cloud providers offer significant discounts if a client is willing/can use preemptible, or low-priority nodes OR commits to using the same nodes for 1-3 years.
It is important to emphasize that while cost is a good basis for comparing and assessing service providers, other factors should also be considered:
- Uptime (Service Level Agreement);
- The surrounding cloud ecosystem;
- Available K8s versions;
- Quality of documentation/tools.
However, these factors are beyond the scope of this article/study. In non-price factors for EKS, AKS, and GKE are examined in detail.
Jupyter Notebook
To simplify the search for the most cost-effective solution, I developed , incorporating plotly + ipywidgets. It allows comparing provider offers for different cluster sizes and service sets.
You can practice with the live version of the notebook at Binder:
Please let me know if the calculations or initial estimates are incorrect (you can do this via an issue or pull request on GitHub — ).
Conclusions
Unfortunately, there are too many nuances to provide more specific recommendations than those included in the TL;DR paragraph at the very beginning. However, some conclusions can still be drawn:
- Unlike GKE and EKS, AKS and Digital Ocean do not charge for the resources of the control plane. AKS and DO are more advantageous if the architecture includes many small clusters (e.g., one cluster per developer or client).
- The slightly less expensive computing resources of GKE make it more advantageous as cluster sizes increase*.
- Using spot nodes or long-term commitments to nodes can reduce costs by more than 50%. Note: Digital Ocean does not offer such discounts.
- The cost of outbound traffic in Google is higher; however, the determining factor in calculations is the cost of computing resources (unless your cluster is a source of significant outbound data).
- Selecting machine types according to CPU and memory workload needs can help avoid unnecessary charges for unutilized resources.
- Digital Ocean charges less for vCPU and more for memory compared to other platforms — this can be a decisive factor for some types of computing workloads.
* Note: the analysis uses data for general-purpose compute nodes (general-purpose). These are n1 GCP Compute Engine instances, m5 AWS ec2, D2v3 Azure virtual machines, and DO droplets with dedicated CPUs. Research can also be conducted among other types of virtual machines (burstable, entry-level). At first glance, the cost of virtual machines linearly depends on the number of vCPUs and the amount of memory, but I am not sure that this assumption will hold for heavily non-standard memory/CPU ratios.
In this article , published in 2018, a benchmark cluster with 100 vCPU cores and 400 GB of memory was used. For comparison, based on my calculations, a similar cluster on each of these platforms (for on-demand instances) would cost the following amount:
- AKS: 51465 USD/year
- EKS: 43138 USD/year
- GKE: 30870 USD/year
- DO: 36131 USD/year
I hope this article, along with the notebook, helps you evaluate the main offerings of managed Kubernetes and/or save on cloud infrastructure by taking advantage of discounts and other opportunities.
P.S. from the translator
Also read in our blog:
- «»;
- «»;
- «»;
- «».
Source: habr.com
