{"id":56259,"date":"2020-02-08T00:00:00","date_gmt":"2020-02-07T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/rabochie-uzly-kubernetes-mnogo-malenkih-ili-neskolko-bolshih"},"modified":"2020-02-18T14:04:30","modified_gmt":"2020-02-18T11:04:30","slug":"rabochie-uzly-kubernetes-mnogo-malenkih-ili-neskolko-bolshih","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/rabochie-uzly-kubernetes-mnogo-malenkih-ili-neskolko-bolshih","title":{"rendered":"Kubernetes Worker Nodes: Many Small or a Few Large?","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

\"Kubernetes
\nWhen creating a Kubernetes cluster, questions may arise: how many worker nodes to set up and of what type? What is better for an on-premise cluster: to buy several powerful servers or to utilize a dozen old machines in your data center? And in the cloud, is it better to take eight single-core instances or two quad-core instances? <\/p>\n

The answers to these questions are in the article by Daniel Weibel, software engineer and instructor for the Learnk8s training project<\/a><\/noindex> translated by the team Kubernetes aaS from Mail.ru<\/a><\/noindex>.
\n<\/a><\/noindex><\/p>\n

Cluster Capacity<\/h2>\n

\nIn general, a Kubernetes cluster can be considered as a large 'supernode'. Its total computing power is the sum of the capabilities of all constituent nodes. <\/p>\n

There are several ways to achieve the desired target capacity of the cluster. For example, we need a cluster with a total capacity of 8 processor cores and 32 GB of RAM, because the application set requires such resources. Then, we can set up two nodes with 16 GB of memory each or four nodes with 8 GB of memory, two quad-core processors or four dual-core processors.<\/p>\n

Here are just two possible ways to create a cluster:<\/p>\n

\"Kubernetes
\nBoth options yield a cluster with the same capacity, but the configuration below has four smaller nodes, while the configuration above has two larger ones. <\/p>\n

Which option is better?<\/h3>\n

\nTo answer this question, let's consider the advantages of both options. We summarized them in a table.<\/p>\n

Several large nodes<\/p>\n

Many small nodes<\/p>\n

Easier management of the cluster (if it's on-premise)<\/p>\n

Smooth auto-scaling<\/p>\n

Cheaper (if on-premise) <\/p>\n

Cost is quite similar (in the cloud) <\/p>\n

Can run resource-intensive applications <\/p>\n

Full replication<\/p>\n

Resources are utilized more efficiently (less overhead for system daemons)
\nHigher cluster fault tolerance<\/p>\n

Note that we are only discussing worker nodes. The choice of the number and size of master nodes is a completely different topic.<\/p>\n

So, let's discuss each point from the table in more detail.<\/p>\n

First option: several large nodes<\/h2>\n

\nThe most extreme option is a single worker node for the entire capacity of the cluster. In the example above, this would be one worker node with 16 CPU cores and 16 GB of RAM.<\/p>\n

Advantages <\/h3>\n

\nPlus #1. Easier management<\/b>
\nIt's easier to manage multiple machines than an entire fleet. Updates and fixes are rolled out faster and synchronization is simpler. The number of failures in absolute terms is also reduced.<\/p>\n

Note that all of the above relates to your own hardware, your own servers, and not to cloud instances.<\/p><\/blockquote>\n

\nIn the cloud, the situation is different. The management is handled by the cloud service provider. Thus, managing ten nodes in the cloud doesn't significantly differ from managing a single node.<\/p>\n

Traffic routing and load balancing between pods in the cloud is done automatically<\/a><\/noindex>: incoming internet traffic is directed to the main load balancer, which directs traffic to the port of one of the nodes (the NodePort service exposes a port in the range of 30000-32767 on each node in the cluster). The rules set by kube-proxy redirect traffic from the node to the pod. Here\u2019s what it looks like for ten pods on two nodes:<\/p>\n

\"Kubernetes
\nPlus #2. Lower costs per node<\/b>
\nA powerful machine is more expensive, but the price increase is not necessarily linear. In other words, one ten-core server with 10 GB of memory is usually cheaper than ten single-core servers with the same amount of memory.<\/p>\n

However, note that this rule typically does not apply to cloud services. In current pricing schemes from all major cloud service providers, prices increase linearly with capacity.<\/p><\/blockquote>\n

\nThus, in the cloud, it is generally not possible to save on more powerful servers.<\/p>\n

Plus #3. You can run resource-intensive applications<\/b>
\nSome applications require powerful servers in the cluster. For example, if a machine learning system needs 8 GB of memory, you cannot run it on nodes with 1 GB; you need at least one large working node.<\/p>\n

Cons <\/h3>\n

\nDisadvantage No. 1. Many pods per node<\/b>
\nIf the same task is performed on fewer nodes, naturally, there will be more pods on each of them.<\/p>\n

This can become a problem.<\/p>\n

The reason is that each module adds some overhead to the container runtime (e.g., Docker), as well as kubelet and cAdvisor.<\/p>\n

For example, kubelet regularly probes the health of all containers on the node \u2014 the more containers there are, the more work kubelet has to do.<\/p>\n

CAdvisor collects resource usage statistics for all containers on the node, while kubelet regularly requests this information and provides it via the API. The more containers there are, the more work there is for both cAdvisor and kubelet.<\/p>\n

If the number of modules increases, it can slow down the system and even undermine its reliability.<\/p>\n

\"Kubernetes
\nIn the Kubernetes repository, some have complained<\/a><\/noindex>, that nodes jump between Ready\/NotReady statuses as kubelet's regular checks of all containers on the node take too much time.
\nFor this reason, Kubernetes It is recommended to place no more than 110 pods per node<\/a><\/noindex>. Depending on the node's performance, you may run more pods on a node, but it's hard to predict whether issues will arise or everything will work well. It\u2019s advisable to test the performance in advance. <\/p>\n

Drawback #2. Replication Limitations<\/b>
\nA small number of nodes limits the effective replication degree of applications. For example, if you have a highly available application with five replicas but only two nodes, the effective replication degree of the application decreases to two.<\/p>\n

Five replicas can only be distributed across two nodes, and if one of them fails, it immediately disables multiple replicas.<\/p>\n

If you have five nodes or more, each replica will run on a separate node, and the failure of one node will remove at most one replica.<\/p>\n

Thus, high availability requirements may necessitate a certain minimum number of nodes in the cluster.<\/p>\n

Drawback #3. Worse Failure Consequences<\/b>
\nWith a small number of nodes, each failure carries more serious consequences. For example, if you have only two nodes and one of them fails, half of your modules disappear immediately.<\/p>\n

Of course, Kubernetes will move the workload from the failed node to others. But if there are few nodes, there may not be enough free capacity. As a result, some of your applications will be unavailable until you bring the failed node back online.<\/p>\n

Thus, the more nodes there are, the less impact hardware failures have.<\/p>\n

Drawback #4. Larger Autoscaling Steps<\/b>
\nIn Kubernetes, there is a cluster auto-scaling system for cloud infrastructure that allows for the automatic addition or removal of nodes based on current needs. With larger nodes, auto-scaling becomes more abrupt and unwieldy. For example, adding an additional node in a two-node setup will increase the cluster capacity by 50% immediately. You will have to pay for these resources even if you do not need them.<\/p>\n

Therefore, if you plan to use cluster auto-scaling, the smaller the nodes, the more flexible and cost-effective scaling you will achieve.<\/p>\n

Now let's consider the advantages and disadvantages of having many small nodes.<\/p>\n

Option two: multiple small nodes<\/h2>\n

\nThe advantages of this approach essentially stem from the disadvantages of the opposite variant with a few large nodes.<\/p>\n

Advantages<\/h3>\n

\nPlus #1. Lesser failure impact<\/b>
\nThe more nodes, the fewer pods on each node. For example, if you have a hundred modules across ten nodes, there will be an average of ten modules on each node.<\/p>\n

Thus, if one of the nodes fails, you only lose 10% of the workload. It is likely that only a small number of replicas are affected, and the applications as a whole remain operational.<\/p>\n

Additionally, the remaining nodes will likely have enough free resources for the workload of the failed node, allowing Kubernetes to freely reschedule pods, and your applications can relatively quickly return to a functional state.<\/p>\n

Plus #2. Good replication<\/b>
\nIf there are enough nodes, the Kubernetes scheduler can assign different nodes to all replicas. Thus, in the case of a node failure, only one replica will be affected, and the application will remain available.<\/p>\n

Cons <\/h3>\n

\nMinus #1. Harder management<\/b>
\nA large number of nodes is more difficult to manage. For example, each Kubernetes node must interact with all others, meaning that the number of connections grows quadratically, and all these links need to be tracked.<\/p>\n

The node controller in the Kubernetes controller manager regularly checks all nodes in the cluster for health \u2014 the more nodes, the greater the load on the controller.<\/p>\n

The load on the etcd database also increases \u2014 each kubelet and kube-proxy calls watcher<\/a><\/noindex> for etcd (via API), to which etcd must relay object updates.<\/p>\n

In general, each worker node adds extra load on the system components of the master nodes.<\/p>\n

\"Kubernetes
\nOfficially, Kubernetes supports clusters with up to 5000 nodes.<\/a><\/noindex>However, in practice, even 500 nodes can cause non-trivial issues.<\/a><\/noindex>.<\/p>\n

To manage a large number of worker nodes, it is advisable to choose more powerful master nodes. For example, kube-up automatically installs<\/a><\/noindex> the appropriate VM size for the master node based on the number of worker nodes. This means the more worker nodes you have, the more powerful the master nodes should be. <\/p>\n

To address these specific issues, there are specialized solutions like Virtual Kubelet.<\/a><\/noindex>This system allows bypassing limitations and building clusters with a huge number of worker nodes.<\/p>\n

Drawback #2. More overhead.<\/b>
\nOn each Kubernetes worker node, a set of system daemons run \u2014 these include the container runtime (e.g., Docker), kube-proxy, and kubelet, including cAdvisor. Collectively, they consume a fixed amount of resources.<\/p>\n

If you have many small nodes, the share of this overhead on each node is higher. For example, imagine that all system daemons of one node together consume 0.1 CPU core and 0.1 GB of memory. If you have one ten-core node with 10 GB of memory, then daemons consume 1% of the cluster's capacity. On the other hand, on ten single-core nodes with 1 GB of memory each, daemons will take up 10% of the cluster's capacity.<\/p>\n

Thus, the fewer nodes there are, the more efficiently the infrastructure is utilized.<\/p>\n

Drawback #3. Inefficient resource usage.<\/b>
\nOn small nodes, there may be a situation where the remaining fragments of resources are too small to allocate any workload, so they remain unused.<\/p>\n

For example, each pod requires 0.75 GB of memory. If you have ten nodes, each with 1 GB of memory, you can run ten pods \u2014 in the end, there will be 0.25 GB of unused memory left on each node.<\/p>\n

This means that 25% of the memory of the entire cluster is wasted.<\/p>\n

On a large node with 10 GB of memory, you can run 13 such pods \u2014 and only one fragment of 0.25 GB will remain unused.<\/p>\n

In this case, only 2.5% of the memory is wasted.<\/p>\n

Thus, resources are utilized more efficiently on larger nodes.<\/p>\n

Several large nodes or many small ones?<\/h2>\n

\nSo, what is better: several large nodes in a cluster or many small ones? As always, there is no definitive answer. Much depends on the type of application.<\/p>\n

For example, if an application requires 10 GB of memory, the choice for larger nodes is evident. However, if the application requires tenfold replication for high availability, it is unlikely to be wise to risk placing replicas on only two nodes\u2014there should be at least ten nodes in the cluster.<\/p>\n

In intermediate situations, make your choice based on the advantages and disadvantages of each option. Some arguments may be more relevant to your situation than others.<\/p>\n

It is also not necessary to make all nodes the same size. There is nothing preventing you from starting with nodes of one size and then adding nodes of another size, combining them in the cluster. The working nodes in a Kubernetes cluster can be fully heterogeneous. So, it's possible to try to combine the advantages of both approaches.<\/p>\n

There is no single recipe, and each situation has its nuances; only production will reveal the truth.<\/p>\n

Translation prepared by the cloud platform team Mail.ru Cloud Solutions<\/a><\/noindex>.<\/i><\/p>\n

More about Kubernetes: 25 useful tools for managing and deploying clusters<\/a><\/noindex>.<\/p>\n

Source: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u041f\u0440\u0438 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u0438 \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 Kubernetes \u043c\u043e\u0433\u0443\u0442 \u0432\u043e\u0437\u043d\u0438\u043a\u0430\u0442\u044c \u0432\u043e\u043f\u0440\u043e\u0441\u044b: \u0441\u043a\u043e\u043b\u044c\u043a\u043e \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0440\u0430\u0431\u043e\u0447\u0438\u0445 \u0443\u0437\u043b\u043e\u0432 \u0438 \u043a\u0430\u043a\u043e\u0433\u043e \u0442\u0438\u043f\u0430? \u0427\u0442\u043e \u043b\u0443\u0447\u0448\u0435 \u0434\u043b\u044f \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430 on-premise: \u043a\u0443\u043f\u0438\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u043c\u043e\u0449\u043d\u044b\u0445 \u0441\u0435\u0440\u0432\u0435\u0440\u043e\u0432 \u0438\u043b\u0438 \u0437\u0430\u0434\u0435\u0439\u0441\u0442\u0432\u043e\u0432\u0430\u0442\u044c \u0434\u0435\u0441\u044f\u0442\u043e\u043a \u0441\u0442\u0430\u0440\u044b\u0445 \u043c\u0430\u0448\u0438\u043d \u0432 \u0432\u0430\u0448\u0435\u043c \u0434\u0430\u0442\u0430-\u0446\u0435\u043d\u0442\u0440\u0435? \u0410 \u0432 \u043e\u0431\u043b\u0430\u043a\u0435 \u043b\u0443\u0447\u0448\u0435 \u0432\u0437\u044f\u0442\u044c \u0432\u043e\u0441\u0435\u043c\u044c \u043e\u0434\u043d\u043e\u044f\u0434\u0435\u0440\u043d\u044b\u0445 \u0438\u043b\u0438 \u0434\u0432\u0430 \u0447\u0435\u0442\u044b\u0440\u0435\u0445\u044a\u044f\u0434\u0435\u0440\u043d\u044b\u0445 \u0438\u043d\u0441\u0442\u0430\u043d\u0441\u0430? \u041e\u0442\u0432\u0435\u0442\u044b \u043d\u0430 \u044d\u0442\u0438 \u0432\u043e\u043f\u0440\u043e\u0441\u044b \u2014 \u0432 \u0441\u0442\u0430\u0442\u044c\u0435 \u0414\u0430\u043d\u0438\u044d\u043b\u044f \u0412\u0430\u0439\u0431\u0435\u043b\u044f, \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u0430-\u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043c\u0438\u0441\u0442\u0430 \u0438 \u043f\u0440\u0435\u043f\u043e\u0434\u0430\u0432\u0430\u0442\u0435\u043b\u044f \u043e\u0431\u0443\u0447\u0430\u044e\u0449\u0435\u0433\u043e […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-56259","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47Kubernetes Working Nodes: many small ones or several large ones? | ProHoster","description":"When creating a cluster.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/rabochie-uzly-kubernetes-mnogo-malenkih-ili-neskolko-bolshih","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0420\u0430\u0431\u043e\u0447\u0438\u0435 \u0443\u0437\u043b\u044b Kubernetes: \u043c\u043d\u043e\u0433\u043e \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u0445 \u0438\u043b\u0438 \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0431\u043e\u043b\u044c\u0448\u0438\u0445? | ProHoster","og:description":"\u041f\u0440\u0438 \u0441\u043e\u0437\u0434\u0430\u043d\u0438\u0438 \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0430.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/rabochie-uzly-kubernetes-mnogo-malenkih-ili-neskolko-bolshih","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-02-07T21:00:00+00:00","article:modified_time":"2020-02-18T11:04:30+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"56259","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 19:28:23","updated":"2022-10-03 08:08:38","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/56259","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=56259"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/56259\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=56259"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=56259"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=56259"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}