{"id":39203,"date":"2019-10-31T22:28:25","date_gmt":"2019-10-31T19:28:25","guid":{"rendered":"https:\/\/prohoster.info\/blog\/zadelyvaem-dyry-v-klastere-kubernetes-doklad-i-rasshifrovka-s-devopsconf\/"},"modified":"2019-10-31T22:28:25","modified_gmt":"2019-10-31T19:28:25","slug":"zadelyvaem-dyry-v-klastere-kubernetes-doklad-i-rasshifrovka-s-devopsconf","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/zadelyvaem-dyry-v-klastere-kubernetes-doklad-i-rasshifrovka-s-devopsconf","title":{"rendered":"Filling the gaps in the Kubernetes cluster. Presentation and transcript from DevOpsConf","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Pavel Selivanov, Solutions Architect at Southbridge and instructor at Slyorm, presented at DevOpsConf 2019. This presentation is part of an in-depth course theme on Kubernetes called \"Slyorm Mega.\"<\/p>\n

<\/p>\n

Slyorm Basic: Introduction to Kubernetes<\/a><\/noindex> taking place in Moscow from November 18-20.
\nSlyorm Mega: Looking under the hood of Kubernetes<\/a><\/noindex> \u2014 Moscow, November 22-24.
\nSlyorm Online: Both Kubernetes courses<\/a><\/noindex> are available at any time.<\/p>\n

\n

\r\n \"Play\r\n
<\/div>\r\n <\/div><\/center><\/p>\n

Below is the transcript of the presentation.<\/p>\n

<\/a><\/noindex><\/p>\n

Good day, colleagues and sympathizers. Today I will be talking about security.<\/p>\n

<\/p>\n

I see that there are many security specialists in the room today. I apologize in advance if I use some terms from the security world that might not align with your usual conventions. <\/p>\n

<\/p>\n

It so happened that about six months ago I got my hands on a public Kubernetes cluster. Public means that there are a number of namespaces, and within these namespaces are users, isolated in their own namespace. All these users belong to different companies. Well, the assumption was that this cluster was to be used as a CDN. So, they provide you with a cluster, give you a user, and you go into your namespace to deploy your fronts. <\/p>\n

<\/p>\n

My previous company tried to sell such a service. They asked me to poke around the cluster to determine if such a solution was suitable. <\/p>\n

<\/p>\n

I entered this cluster. I was given limited rights and a restricted namespace. The guys there understood what safety meant. They read about Role-Based Access Control (RBAC) in Kubernetes and configured it so that I couldn't launch pods separately from deployments. I don\u2019t remember the issue I was trying to solve by launching a pod without a deployment, but I really wanted to simply run a pod. I decided to check what rights I had in the cluster, what I could and couldn't do, and what they had configured. By the way, I\u2019ll share how their RBAC was set up incorrectly. <\/p>\n

<\/p>\n

It turned out that within two minutes, I gained admin access to their cluster, looked into all the neighboring namespaces, and saw production fronts from companies that had already purchased the service and deployed. I barely stopped myself from visiting someone\u2019s front and posting some profane word on the homepage. <\/p>\n

<\/p>\n

I will explain through examples how I did this and how to protect against it. <\/p>\n

<\/p>\n

Let me introduce myself first. My name is Pavel Selivanov. I am the architect of Southbridge. I am knowledgeable in Kubernetes, DevOps, and various trendy tools. My team and I at Southbridge build all of this, while I provide consultation. <\/p>\n

<\/p>\n

In addition to our main activities, we recently launched projects called Slurms. We are trying to bring our expertise in working with Kubernetes to the masses, teaching others to also work with K8s. <\/p>\n

<\/p>\n

What I will be discussing today. The topic of the presentation is clear \u2014 cluster Kubernetes security. However, I want to immediately clarify that this is a vast subject \u2014 and therefore, I want to specify what I will not cover. I will not talk about overused terms that have been churned on the internet a hundred times, such as RBAC and certificates. <\/p>\n

<\/p>\n

I will be discussing what my colleagues and I are concerned about regarding security in the Kubernetes cluster. We see these issues both with providers who offer Kubernetes clusters and with clients who come to us. Even clients who come to us from other consulting and admin companies. So, the scale of the tragedy is quite large, actually. <\/p>\n

<\/p>\n

Just three points I will cover today: <\/p>\n

<\/p>\n

    \n
  1. User rights vs pod rights. User rights and pod rights are not the same. <\/li>\n
  2. Information gathering about the cluster. I will show what information can be gathered from the cluster without having special rights within it. <\/li>\n
  3. DoS attack on the cluster. If we cannot gather information, we can still bring down the cluster. I will talk about DoS attacks on the cluster management elements. <\/li>\n<\/ol>\n

    <\/p>\n

    Another general topic I will mention \u2014 what I used to test all this, on which I can confidently say that it all works.<\/p>\n

    <\/p>\n

    We base our installation of the Kubernetes cluster on Kubespray. For those who may not know, it is essentially a set of roles for Ansible. We use it regularly in our work. It's great because it can be deployed anywhere \u2014 on hardware or in the cloud. One installation method generally works for everything. <\/p>\n

    <\/p>\n

    In this cluster, I will have Kubernetes v1.14.5. The entire Kubernetes cluster we will be considering is divided into namespaces, and each namespace belongs to a separate team, with access granted only to members of that team. They cannot access other namespaces, only their own. However, there is an admin account that has rights to the entire cluster. <\/p>\n

    <\/p>\n

    \"Filling<\/p>\n

    <\/p>\n

    I promised that the first step would be obtaining admin rights to the cluster. We need a specially prepared pod that will breach the Kubernetes cluster. All we need to do is apply it to the Kubernetes cluster. <\/p>\n

    <\/p>\n

    kubectl apply -f pod.yaml<\/code><\/pre>\n
    <\/p>\n
    This pod will arrive on one of the master nodes of the Kubernetes cluster. After this, the cluster will happily return a file called admin.conf. In Kubernetes, this file stores all the admin certificates and configures the cluster API accordingly. That's how easily admin access can be obtained; I believe this applies to 98% of Kubernetes clusters. <\/p>\n
    <\/p>\n
    I reiterate, this pod was created by a developer in your cluster who had permissions to deploy their proposals in one small namespace, tightly constrained by RBAC. They had no privileges whatsoever. Nonetheless, the certificate was returned. <\/p>\n
    <\/p>\n
    Now about the specially prepared pod. We will run it on any image. For example, let's take debian:jessie. <\/p>\n
    <\/p>\n
    We have the following: <\/p>\n
    <\/p>\n
    tolerations:\n-   effect: NoSchedule \n    operator: Exists \nnodeSelector: \n    node-role.kubernetes.io\/master: \"\" <\/code><\/pre>\n
    <\/p>\n
    What is a toleration? Masters in a Kubernetes cluster are usually marked with something called a taint. The essence of this taint is that pods cannot be scheduled on master nodes. However, nothing prevents any pod from specifying that it is tolerant to this taint. The Toleration section states that if a node has NoSchedule, then our pod is tolerant to that taint\u2014there will be no issues. <\/p>\n
    <\/p>\n
    Next, we indicate that our pod is not just tolerant but also wants to specifically land on a master. This is because the most valuable resources we need\u2014 all the certificates\u2014are on the masters. Therefore, we use nodeSelector, and we have a standard label on masters that allows us to choose from all the cluster nodes specifically those nodes that are masters. <\/p>\n
    <\/p>\n
    With these two sections, the pod is guaranteed to land on a master and will be permitted to reside there. <\/p>\n
    <\/p>\n
    But simply arriving on a master is not enough. It won\u2019t provide us with anything. So next, we have two additional requirements:<\/p>\n
    <\/p>\n
    hostNetwork: true \nhostPID: true <\/code><\/pre>\n
    <\/p>\n
    We specify that our pod, which we are launching, will reside in the kernel namespace, in the network namespace, and in the PID namespace. Once the pod starts on the master, it will be able to see all the real, live interfaces of this node, listen to all traffic, and view the PIDs of all processes.<\/p>\n
    <\/p>\n
    Then it's a simple matter. Just take etcd and read what you want. <\/p>\n
    <\/p>\n
    Most interestingly, this is a feature of Kubernetes that is present by default. <\/p>\n
    <\/p>\n
    volumeMounts:\n- mountPath: \/host \n  name: host \nvolumes:\n- hostPath: \n    path: \/ \n    type: Directory \n  name: host <\/code><\/pre>\n
    <\/p>\n
    The essence of it is that in the pod we are launching, even without permissions on this cluster, we can specify that we want to create a volume of type hostPath. This means taking a path from the host on which we will run \u2014 and using it as a volume. We then name it name: host. We mount the entire hostPath inside the pod. In this example, in the directory \/host. <\/p>\n
    <\/p>\n
    I will reiterate. We instructed the pod to come to the master, obtain hostNetwork and hostPID \u2014 and to mount the entire root of the master inside this pod. <\/p>\n
    <\/p>\n
    You understand that in Debian we have bash running, and this bash is running as root. So we just obtained root access on the master without having any specific permissions in the Kubernetes cluster.<\/p>\n
    <\/p>\n
    The next task is to enter the pod in the directory \/host \/etc\/kubernetes\/pki, if I am not mistaken, and collect all the master certificates of the cluster, and thus become an admin of the cluster. <\/p>\n
    <\/p>\n
    Looking at it this way, these are some of the most dangerous permissions in pods \u2014 regardless of what permissions the user has:
    \n\"Filling<\/p>\n
    <\/p>\n
    If I have the permission to launch a pod in some namespace of the cluster, then this pod has those permissions by default. I can launch privileged pods, which means I have almost root access on the node. <\/p>\n
    <\/p>\n
    My favorite is the Root user. And Kubernetes has an option called Run As Non-Root. It's a sort of protection against hackers. Do you know what a 'Moldovan virus' is? If you are a hacker and you come into my Kubernetes cluster, us poor administrators request: 'Please specify in your pods that you will use to hack my cluster, run as non-root. Otherwise, it will turn out that you launched a process in your pod as root, and it will be very easy for you to hack me. Please protect yourself from yourself.' <\/p>\n
    <\/p>\n
    Host path volume \u2014 in my opinion, the fastest way to achieve the desired result from a Kubernetes cluster. <\/p>\n
    <\/p>\n
    But what to do with all this? <\/p>\n
    <\/p>\n
    Thoughts that should come to any normal administrator encountering Kubernetes: \"Aha, I told you, Kubernetes doesn't work. It's full of holes. And the whole thing is garbage.\" In reality, there is such a thing as documentation, and if you look there, you'll find a section. Pod Security Policy<\/a><\/noindex>. <\/p>\n
    <\/p>\n
    This is a YAML object that we can create in the Kubernetes cluster, which controls security aspects specifically in pod definitions. In other words, it effectively controls the permissions for utilizing various hostNetwork, hostPID, and certain types of volumes that exist in pods upon startup. With Pod Security Policy, all of this can be described. <\/p>\n
    <\/p>\n
    The most interesting thing about Pod Security Policy is that in a Kubernetes cluster, all PSP implementations are not just unwritten; they are simply turned off by default. Pod Security Policy is enabled through an admission plugin.<\/p>\n
    <\/p>\n
    Okay, let's deploy the Pod Security Policy to the cluster, stating that we have certain service pods in a namespace that only admins can access. Let's say all other pods have limited rights. Because developers likely don't need to run privileged pods in your cluster. <\/p>\n
    <\/p>\n
    And everything seems fine. Our Kubernetes cluster can't be hacked in two minutes. <\/p>\n
    <\/p>\n
    There's a problem. If you have a Kubernetes cluster, it's likely that monitoring is set up in your cluster. I even dare to predict that if there's monitoring in your cluster, it's called Prometheus. <\/p>\n
    <\/p>\n
    What I'm about to explain will be valid for both the Prometheus operator and for Prometheus installed in its pure form. The question is, if I can't quickly get an admin in the cluster, it means I need to search more. And I can search using your monitoring.<\/p>\n
    <\/p>\n
    Probably, everyone has read the same articles on Habr, and the monitoring is located in the monitoring namespace. The Helm chart for everyone is named roughly the same. I assume that if you do 'helm install stable\/prometheus', you'll get roughly the same names. It's even likely that I won't have to guess the DNS name in your cluster, because it\u2019s standard. <\/p>\n
    <\/p>\n
    \"Filling<\/p>\n
    <\/p>\n
    Next, we have a dev namespace, where one can run a pod. And it's very easy to do it like this from that pod: <\/p>\n
    <\/p>\n
    $ curl http:\/\/prometheus-kube-state-metrics.monitoring <\/code><\/pre>\n
    <\/p>\n
    prometheus-kube-state-metrics is one of Prometheus's exporters that collects metrics from the Kubernetes API itself. It gathers a wealth of data about what is running in your cluster, its status, and any issues you may have with it. <\/p>\n
    <\/p>\n
    As a simple example: <\/p>\n
    <\/p>\n
    kube_pod_container_info{namespace=\"kube-system\",pod=\"kube-apiserver-k8s-1\",container=\"kube-apiserver\",image= <\/p>\n
    <\/p>\n
    \"gcr.io\/google-containers\/kube-apiserver:v1.14.5\" <\/strong><\/p>\n
    <\/p>\n
    ,image_id=\"docker-pullable:\/\/gcr.io\/google-containers\/kube-apiserver@sha256:e29561119a52adad9edc72bfe0e7fcab308501313b09bf99df4a9638ee634989\",container_id=\"docker:\/\/7cbe7b1fea33f811fdd8f7e0e079191110268f2853397d7daf08e72c22d3cf8b\"} 1 <\/p>\n
    <\/p>\n
    By making a simple curl request from a non-privileged pod, you can obtain such information. If you\u2019re not sure which version of Kubernetes you are running, it can easily tell you. <\/p>\n
    <\/p>\n
    Interestingly, in addition to accessing kube-state-metrics, you can just as easily query Prometheus directly. You can collect metrics from there. You might even construct metrics from it. Theoretically, you could create a query from the cluster to Prometheus that simply shuts it down, causing your monitoring to stop functioning entirely. <\/p>\n
    <\/p>\n
    This raises the question of whether any external monitoring is watching your monitoring. Just now, I've managed to operate in the Kubernetes cluster without any consequences for myself. You wouldn\u2019t even know I was acting there since the monitoring is gone. <\/p>\n
    <\/p>\n
    Just like with PSP, it feels like the issue is that all these trendy technologies \u2014 Kubernetes, Prometheus \u2014 simply don\u2019t work and are filled with holes. But that\u2019s not the case. <\/p>\n
    <\/p>\n
    There is such a thing \u2014 Network Policy<\/a><\/noindex>.<\/p>\n
    <\/p>\n
    If you\u2019re a decent admin, you likely know that Network Policy is just another yaml file among many in the cluster. And some Network Policies are definitely unnecessary. Even if you have read about what Network Policy is \u2014 a yaml firewall for Kubernetes that allows you to restrict access between namespaces and pods \u2014 you\u2019re probably thinking that a yaml firewall in Kubernetes is just another abstraction... No, this is definitely not needed. <\/p>\n
    <\/p>\n
    Even if your security team hasn't been informed that you can easily and simply build a very granular firewall using your Kubernetes, if they don't know this yet and aren't bothering you with requests like, \"Well, give us this...\", you still need Network Policy to restrict access to certain service endpoints that can be queried from your cluster without any authorization. <\/p>\n
    <\/p>\n
    As in the example I mentioned, you can query kube state metrics from any namespace in the Kubernetes cluster without having any permissions for it. Network policies restricted access from all other namespaces to the monitoring namespace, so that\u2019s it: no access, no problems. In all the charts available, including the standard Prometheus and the one in the operator, there is just an option in the helm values to enable network policies for them. You just need to enable it, and they will work. <\/p>\n
    <\/p>\n
    There is indeed one problem here. As a normal bearded admin, you probably decided that network policies are unnecessary. And after reading various articles on resources like Habr, you concluded that flannel, especially with the host-gateway mode, is the best you can choose. <\/p>\n
    <\/p>\n
    What to do? <\/p>\n
    <\/p>\n
    You can try to redeploy the networking solution currently in your Kubernetes cluster, replacing it with something more functional, like Calico, for instance. But I want to be honest, changing the networking solution in a working Kubernetes cluster is quite a non-trivial task. I've tackled it twice (both times theoretically, though), but we even demonstrated how to do it at Slurm. For our trainees, we showed how to change the networking solution in a Kubernetes cluster. In principle, you can try to ensure that there is no downtime in the production cluster. However, you might end up with nothing working. <\/p>\n
    <\/p>\n
    And the problem can actually be solved quite simply. The cluster has certificates, and you know that your certificates will expire in a year. Well, the usual normal solution with certificates in the cluster is: why should we worry? We'll just bring up a new cluster next to it, let the old one expire, and redeploy everything. True, when it expires, we'll have a day where everything lies idle, but at least we'll have a new cluster. <\/p>\n
    <\/p>\n
    When you're setting up the new cluster, make sure to include Calico instead of flannel. <\/p>\n
    <\/p>\n
    What to do if your certificates are issued for a hundred years and you don't plan to redeploy the cluster? There's a tool called Kube-RBAC-Proxy. It's a really cool development that allows you to embed it as a sidecar container to any pod in a Kubernetes cluster. It effectively adds RBAC authorization directly to that pod through Kubernetes itself. <\/p>\n
    <\/p>\n
    There is one problem, though. Previously, this Kube-RBAC-Proxy solution was integrated into the Prometheus operator. But it\u2019s no longer there. Now, modern versions rely on the presence of a network policy and use it to secure access. Therefore, you will need to rewrite the chart a bit. In fact, if you go to this repository<\/a><\/noindex>, there are examples of how to use it as sidecars, and the charts will need minimal rewriting. <\/p>\n
    <\/p>\n
    There\u2019s also one more minor issue. Not only does Prometheus expose its metrics to anyone. All components of the Kubernetes cluster can also expose their own metrics. <\/p>\n
    <\/p>\n
    But as I mentioned earlier, if you can't access the cluster and gather information, you can at least cause some damage. <\/p>\n
    <\/p>\n
    So I'll quickly show you two ways how you can ruin the health of a Kubernetes cluster. <\/p>\n
    <\/p>\n
    You will laugh when I tell you this; these are two cases from real life. <\/p>\n
    <\/p>\n
    Method one: Resource exhaustion. <\/p>\n
    <\/p>\n
    We launch another special pod. It will have a section like this. <\/p>\n
    <\/p>\n
    resources: \n    requests: \n        cpu: 4 \n        memory: 4Gi <\/code><\/pre>\n
    <\/p>\n
    As you know, requests represent the amount of CPU and memory that is reserved on the host for specific pods with requests. If we have a four-core host in the Kubernetes cluster, and a pod with a request for four CPUs arrives, then no other pod with requests can be deployed on that host. <\/p>\n
    <\/p>\n
    If I run such a pod, then execute the command: <\/p>\n
    <\/p>\n
    $ kubectl scale special-pod --replicas=...<\/code><\/pre>\n
    <\/p>\n
    Then no one will be able to deploy to the Kubernetes cluster. Because all nodes will run out of requests. This way, I can take down your Kubernetes cluster. If I do this in the evening, deployments may be halted for quite a while. <\/p>\n
    <\/p>\n
    If we take another look at the Kubernetes documentation, we will see something called a Limit Range. It sets resource limits for cluster objects. You can write a Limit Range object in YAML, apply it to specific namespaces, and then within that namespace, you can specify default, maximum, and minimum resources for pods.<\/p>\n
    <\/p>\n
    With this feature, we can restrict users in specific product namespaces from deploying undesirable settings for their pods. However, unfortunately, even if you tell a user not to run pods with requests exceeding one CPU, there is a wonderful command called scale, or they can scale through the dashboard.<\/p>\n
    <\/p>\n
    And this brings us to the second way. Let's try to launch 11,111,111,111,111 pods. That's eleven billion. It\u2019s not because I made up this number; I have actually seen it happen. <\/p>\n
    <\/p>\n
    A true story. Late one evening, I was ready to leave the office. I noticed a group of developers in the corner, frantically working on their laptops. I approached them and asked, 'What happened?'<\/p>\n
    <\/p>\n
    A little earlier, around nine in the evening, one of the developers was about to go home. He decided, 'I will scale my application to one.' He pressed one, but the internet was lagging. He pressed one again, then hit Enter, and poked at everything he could. At that moment, the internet came back to life \u2014 and everything started scaling to that number. <\/p>\n
    <\/p>\n
    However, this story took place not on Kubernetes; at that time, it was Nomad. It ended up with our attempts to stop Nomad from stubbornly trying to scale continuing for an hour, during which Nomad responded that it wouldn\u2019t stop scaling and wouldn\u2019t do anything else. 'I\u2019m tired; I\u2019m leaving.' And it shut down. <\/p>\n
    <\/p>\n
    Naturally, I tried to do the same thing on Kubernetes. Eleven billion pods didn\u2019t make Kubernetes happy; it said, 'I can\u2019t. Exceeds internal caps.' However, one billion pods worked. <\/p>\n
    <\/p>\n
    In response to one billion Pods, Kubernetes did not crash. It really started to scale. The further the process went, the more time it took to create new Pods. However, the process continued. The only problem is that if I can indefinitely launch Pods in my namespace, even without requests and limits, I can start so many Pods with certain tasks that the nodes will begin to struggle with memory and CPU. When I launch that many Pods, the information from them must reach the storage, namely etcd. And when too much information arrives there, the storage starts responding too slowly \u2014 and Kubernetes begins to lag. <\/p>\n
    <\/p>\n
    And there's another problem\u2026 As you know, the control elements of Kubernetes aren't just one central thing, but several components. There's, in particular, a controller manager, a scheduler, and so on. All these components will start doing unnecessary, pointless work simultaneously, which will gradually take more and more time. The controller manager will create new Pods. The scheduler will try to find them a new node. Your new nodes in the cluster will likely run out soon. The Kubernetes cluster will start to perform more and more slowly.<\/p>\n
    <\/p>\n
    But I decided to go even further. As you know, there is something in Kubernetes called a service. Typically, in your clusters, services likely operate with the help of IP tables. <\/p>\n
    <\/p>\n
    If you launch a billion Pods, for example, and then use a script to force Kubernetes to create new services: <\/p>\n
    <\/p>\n
    for i in {1..1111111}; do\n    kubectl expose deployment test --port 80  \n        --overrides=\"{\"apiVersion\": \"v1\", \n           \"metadata\": {\"name\": \"nginx$i\"}}\"; \ndone <\/code><\/pre>\n
    <\/p>\n
    On all nodes of the cluster, new IP tables rules will approximately simultaneously be generated. Moreover, for each service, a billion IP tables rules will be generated. <\/p>\n
    <\/p>\n
    I tested all this with several thousands, up to ten. And the problem is that already at this threshold, ssh access to the node becomes quite problematic. Because packets, passing through that many chains, start to struggle. <\/p>\n
    <\/p>\n
    And this too is resolved using Kubernetes. There is an object called ResourceQuota. It sets the number of available resources and objects for a namespace in the cluster. We can create a YAML object in each namespace of the Kubernetes cluster. With this object, we can specify that a certain number of requests and limits are allocated for this namespace, and then we can say that it's possible to create 10 services and 10 pods in this namespace. A developer can run into issues if they try to exceed this limit. Kubernetes will tell them: \"You cannot scale your pods to that number because it exceeds the resource quota.\" There, problem solved. Documentation is here<\/a><\/noindex>.<\/p>\n
    <\/p>\n
    One problematic issue arises in this regard. You can feel how difficult it becomes to create a namespace in Kubernetes. To create it, we need to consider a lot of factors.<\/p>\n
    <\/p>\n
    ResourceQuota + LimitRange + RBAC
    \n\u2022 Create a namespace
    \n\u2022 Create a LimitRange within it
    \n\u2022 Create a ResourceQuota within it
    \n\u2022 Create a service account for CI
    \n\u2022 Create a role binding for CI and users
    \n\u2022 Optionally, start the necessary utility pods <\/p>\n
    <\/p>\n
    So, taking the opportunity, I would like to share my developments. There is a thing called Operator SDK. It's a way to write operators for the Kubernetes cluster. You can write operators using Ansible.<\/p>\n
    <\/p>\n
    Initially, we wrote in Ansible, but then I saw that there is an Operator SDK, so I rewrote the Ansible role into an operator. This operator allows the creation of an object in the Kubernetes cluster called a Command. Inside the Command, it allows you to describe the environment for that command in YAML. And within the Command's environment, it allows you to specify how many resources we are allocating. <\/p>\n
    <\/p>\n
    A little simplification of this complex process<\/a><\/noindex>. <\/p>\n
    <\/p>\n
    And in conclusion. What to do with all this?
    \nFirst, Pod Security Policy is good. And although none of the Kubernetes installers use them to this day, they should definitely be used in your clusters. <\/p>\n
    <\/p>\n
    Network Policy is not just another unnecessary feature. It is something that is really needed in the cluster. <\/p>\n
    <\/p>\n
    LimitRange\/ResourceQuota \u2014 it\u2019s time to use them. We started using this a long time ago, and I was convinced that everyone was applying it. It turned out to be quite rare. <\/p>\n
    <\/p>\n
    In addition to what I mentioned during the presentation, there are undocumented features that allow attacks on the cluster. A large analysis of Kubernetes vulnerabilities was released recently. Some issues are so sad and frustrating. For instance, under certain conditions, the kubelets in a Kubernetes cluster can expose the contents of the warlocks directory to unauthorized users.<\/a><\/noindex>. <\/p>\n
    <\/p>\n
    Instructions on how to reproduce everything I discussed are available. There are files with production examples illustrating how ResourceQuota and Pod Security Policy look. You can interact with all of this. <\/p>\n
    <\/p>\n
    Here<\/a><\/noindex> Thank you all. <\/p>\n
    <\/p>\n
    Madmind Studio has revealed the system requirements and gameplay for the action game Succubus.<\/p>\n
    Source: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"
    \u041f\u0430\u0432\u0435\u043b \u0421\u0435\u043b\u0438\u0432\u0430\u043d\u043e\u0432, \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u043e\u0440 \u0440\u0435\u0448\u0435\u043d\u0438\u0439 Southbridge \u0438 \u043f\u0440\u0435\u043f\u043e\u0434\u0430\u0432\u0430\u0442\u0435\u043b\u044c \u0421\u043b\u0451\u0440\u043c\u0430, \u0432\u044b\u0441\u0442\u0443\u043f\u0438\u043b \u0441 \u0434\u043e\u043a\u043b\u0430\u0434\u043e\u043c \u043d\u0430 DevOpsConf 2019. \u042d\u0442\u043e\u0442 \u0434\u043e\u043a\u043b\u0430\u0434 \u2014 \u0447\u0430\u0441\u0442\u044c \u043e\u0434\u043d\u043e\u0439 \u0438\u0437 \u0442\u0435\u043c \u0443\u0433\u043b\u0443\u0431\u043b\u0435\u043d\u043d\u043e\u0433\u043e \u043a\u0443\u0440\u0441\u0430 \u043f\u043e Kubernetes \u00ab\u0421\u043b\u0451\u0440\u043c \u041c\u0435\u0433\u0430\u00bb. \u0421\u043b\u0451\u0440\u043c \u0411\u0430\u0437\u043e\u0432\u044b\u0439: \u0432\u0432\u0435\u0434\u0435\u043d\u0438\u0435 \u0432 Kubernetes \u043f\u0440\u043e\u0445\u043e\u0434\u0438\u0442 \u0432 \u041c\u043e\u0441\u043a\u0432\u0435 18-20 \u043d\u043e\u044f\u0431\u0440\u044f. \u0421\u043b\u0451\u0440\u043c \u041c\u0435\u0433\u0430: \u0437\u0430\u0433\u043b\u044f\u0434\u044b\u0432\u0430\u0435\u043c \u043f\u043e\u0434 \u043a\u0430\u043f\u043e\u0442 Kubernetes \u2014 \u041c\u043e\u0441\u043a\u0432\u0430, 22-24 \u043d\u043e\u044f\u0431\u0440\u044f. \u0421\u043b\u0451\u0440\u043c \u041e\u043d\u043b\u0430\u0439\u043d: \u043e\u0431\u0430 \u043a\u0443\u0440\u0441\u0430 \u043f\u043e Kubernetes \u0434\u043e\u0441\u0442\u0443\u043f\u0435\u043d \u0432\u0441\u0435\u0433\u0434\u0430. […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":29423,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-39203","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd47","description":"We are plugging holes in the Kubernetes cluster. Presentation and transcript from DevOpsConf | ProHoster.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/zadelyvaem-dyry-v-klastere-kubernetes-doklad-i-rasshifrovka-s-devopsconf","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0417\u0430\u0434\u0435\u043b\u044b\u0432\u0430\u0435\u043c \u0434\u044b\u0440\u044b \u0432 \u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0435 Kubernetes. \u0414\u043e\u043a\u043b\u0430\u0434 \u0438 \u0440\u0430\u0441\u0448\u0438\u0444\u0440\u043e\u0432\u043a\u0430 \u0441 DevOpsConf | ProHoster","og:description":"\u041f\u0430\u0432\u0435\u043b \u0421\u0435\u043b\u0438\u0432\u0430\u043d\u043e\u0432, \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u043e\u0440 \u0440\u0435\u0448\u0435\u043d\u0438\u0439 Southbridge \u0438 \u043f\u0440\u0435\u043f\u043e\u0434\u0430\u0432\u0430\u0442\u0435\u043b\u044c \u0421\u043b\u0451\u0440\u043c\u0430, \u0432\u044b\u0441\u0442\u0443\u043f\u0438\u043b \u0441 \u0434\u043e\u043a\u043b\u0430\u0434\u043e\u043c \u043d\u0430 DevOpsConf 2019.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/zadelyvaem-dyry-v-klastere-kubernetes-doklad-i-rasshifrovka-s-devopsconf","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-10-31T19:28:25+00:00","article:modified_time":"2019-10-31T19:28:25+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"39203","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 01:15:20","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-03-01 00:54:43","updated":"2026-01-24 01:15:20","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/39203","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=39203"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/39203\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media\/29423"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=39203"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=39203"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=39203"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}