Ngokushesha noma kamuva, ekusebenzeni kwanoma yiluphi uhlelo, kuvela udaba lokuphepha: ukuqinisekisa ukuqinisekiswa, ukuhlukaniswa kwamalungelo, ukuhlolwa kwamabhuku kanye neminye imisebenzi. Sekuvele kudalelwe i-Kubernetes
Ukufakazela ubuqiniso
Kunezinhlobo ezimbili zabasebenzisi e-Kubernetes:
- Ama-Akhawunti Wesevisi - ama-akhawunti aphethwe i-Kubernetes API;
- Abasebenzisi — Abasebenzisi “abavamile” abaphethwe izinsiza zangaphandle, ezizimele.
Umehluko omkhulu phakathi kwalezi zinhlobo ukuthi kuma-Akhawunti Wesevisi kukhona izinto ezikhethekile ku-Kubernetes API (zibizwa ngokuthi - ServiceAccounts
), eziboshelwe endaweni yamagama kanye nesethi yedatha yokugunyazwa egcinwe kuqoqo ezintweni zohlobo Lwezimfihlo. Abasebenzisi abanjalo (Ama-Akhawunti Wesevisi) bahloselwe ngokuyinhloko ukuphatha amalungelo okufinyelela ku-Kubernetes API yezinqubo ezisebenza kuqoqo le-Kubernetes.
Abasebenzisi Abajwayelekile abanakho okufakiwe ku-Kubernetes API: kumele baphathwe izindlela zangaphandle. Zenzelwe abantu noma izinqubo ezihlala ngaphandle kweqoqo.
Isicelo ngasinye se-API sihlotshaniswa ne-Akhawunti Yesevisi, Umsebenzisi, noma sithathwa njengengaziwa.
Idatha yokuqinisekisa yomsebenzisi ihlanganisa:
- Igama lomsebenzisi - igama lomsebenzisi (icala elibucayi!);
- I-UID - iyunithi yezinhlamvu ehlonza umsebenzisi efundeka ngomshini “engaguquguquki futhi ehlukile kunegama lomsebenzisi”;
- Amaqembu - uhlu lwamaqembu umsebenzisi ayingxenye yawo;
- Extra - izinkambu ezengeziwe ezingasetshenziswa indlela yokugunyazwa.
I-Kubernetes ingasebenzisa inombolo enkulu yezindlela zokuqinisekisa: Izitifiketi ze-X509, amathokheni e-Bearer, ummeleli wokuqinisekisa, i-HTTP Basic Auth. Usebenzisa lezi zindlela, ungasebenzisa inani elikhulu lezikimu zokugunyaza: ukusuka efayeleni elimile elinamagama ayimfihlo ukuya ku-OpenID OAuth2.
Ngaphezu kwalokho, kungenzeka ukusebenzisa izikimu zokugunyaza eziningana ngasikhathi sinye. Ngokuzenzakalelayo, iqoqo lisebenzisa:
- amathokheni e-akhawunti yesevisi - yama-Akhawunti Wesevisi;
- I-X509 - Yabasebenzisi.
Umbuzo mayelana nokuphatha i-ServiceAccounts ungaphezu kwalesi sihloko, kodwa kulabo abafuna ukujwayelana nalolu daba ngokuningiliziwe, ngincoma ukuqala
Izitifiketi zabasebenzisi (X.509)
Indlela yakudala yokusebenza nezitifiketi ibandakanya:
- ukukhiqiza ukhiye:
mkdir -p ~/mynewuser/.certs/ openssl genrsa -out ~/.certs/mynewuser.key 2048
- ukukhiqiza isicelo sesitifiketi:
openssl req -new -key ~/.certs/mynewuser.key -out ~/.certs/mynewuser.csr -subj "/CN=mynewuser/O=company"
- ukucubungula isicelo sesitifiketi usebenzisa okhiye be-CA beqoqo le-Kubernetes, ukuthola isitifiketi somsebenzisi (ukuthola isitifiketi, kufanele usebenzise i-akhawunti ekwazi ukufinyelela ukhiye we-CA we-Kubernetes cluster, otholakala ngokuzenzakalelayo
/etc/kubernetes/pki/ca.key
):openssl x509 -req -in ~/.certs/mynewuser.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out ~/.certs/mynewuser.crt -days 500
- ukudala ifayela lokumisa:
- incazelo yeqoqo (chaza ikheli nendawo yefayela lesitifiketi se-CA ukuze kufakwe iqoqo elithile):
kubectl config set-cluster kubernetes --certificate-authority=/etc/kubernetes/pki/ca.crt --server=https://192.168.100.200:6443
- noma kanjani hhayiinketho enconyiwe - akudingeki ukuthi ucacise isitifiketi sempande (khona-ke i-kubectl ngeke ihlole ukulunga kwe-api-server yeqoqo):
kubectl config set-cluster kubernetes --insecure-skip-tls-verify=true --server=https://192.168.100.200:6443
- ukwengeza umsebenzisi kufayela lokumisa:
kubectl config set-credentials mynewuser --client-certificate=.certs/mynewuser.crt --client-key=.certs/mynewuser.key
- ukwengeza umongo:
kubectl config set-context mynewuser-context --cluster=kubernetes --namespace=target-namespace --user=mynewuser
- umsebenzi ozokwenziwa wokuqukethwe okuzenzakalelayo:
kubectl config use-context mynewuser-context
- incazelo yeqoqo (chaza ikheli nendawo yefayela lesitifiketi se-CA ukuze kufakwe iqoqo elithile):
Ngemuva kokuguqulwa okungenhla, kufayela .kube/config
ukulungiselelwa okufana nalokhu kuzokwenziwa:
apiVersion: v1
clusters:
- cluster:
certificate-authority: /etc/kubernetes/pki/ca.crt
server: https://192.168.100.200:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
namespace: target-namespace
user: mynewuser
name: mynewuser-context
current-context: mynewuser-context
kind: Config
preferences: {}
users:
- name: mynewuser
user:
client-certificate: /home/mynewuser/.certs/mynewuser.crt
client-key: /home/mynewuser/.certs/mynewuser.key
Ukwenza kube lula ukudlulisa ukucushwa phakathi kwama-akhawunti namaseva, kuyasiza ukuhlela amanani okhiye abalandelayo:
-
certificate-authority
-
client-certificate
-
client-key
Ukuze wenze lokhu, ungakwazi ukufaka ikhodi kumafayela ashiwo kuwo usebenzisa i-base64 futhi uwabhalise ku-config, wengeze isijobelelo egameni lezinkinobho. -data
, i.e. esethole certificate-authority-data
nokunye okunjalo.
Izitifiketi ezine-kubeadm
Ngokukhululwa
kubeadm alpha kubeconfig user --client-name=mynewuser --apiserver-advertise-address 192.168.100.200
NB: Iyadingeka khangisa ikheli ingatholakala ku-api-server config, etholakala ngokuzenzakalelayo /etc/kubernetes/manifests/kube-apiserver.yaml
.
Ukulungiselelwa okuwumphumela kuzophuma ku-stdout. Idinga ukugcinwa ngaphakathi ~/.kube/config
i-akhawunti yomsebenzisi noma ifayela elicaciswe kokuguquguquka kwemvelo KUBECONFIG
.
Gebha Ujule
Kulabo abafuna ukuqonda izindaba ezichazwe kabanzi:
-
isihloko esihlukile ekusebenzeni nezitifiketi kumadokhumenti asemthethweni akwa-Kubernetes; -
isihloko esihle esivela ku-Bitnami , lapho ukukhishwa kwezitifiketi kuthintwa khona ngokombono ongokoqobo. -
imibhalo ejwayelekile ekuqinisekisweni ku-Kubernetes.
Ngena
I-akhawunti egunyaziwe ezenzakalelayo ayinawo amalungelo okusebenza kuqoqo. Ukuze kunikezwe izimvume, i-Kubernetes isebenzisa indlela yokugunyaza.
Ngaphambi kwenguqulo 1.6, u-Kubernetes wasebenzisa uhlobo lokugunyaza olubizwa ngokuthi I-ABAC (Ukulawula ukufinyelela okusekelwe kusibaluli). Imininingwane ngakho ingatholakala ku
Indlela yamanje (futhi evumelana nezimo) yokuhlukanisa amalungelo okufinyelela kuqoqo ibizwa I-RBAC (
Ukuze unike amandla i-RBAC, udinga ukuqala i-Kubernetes api-server ngepharamitha --authorization-mode=RBAC
. Imingcele isethwe ku-manifest ngokucushwa kwe-api-server, okuthi ngokuzenzakalelayo ibekwe eceleni kwendlela. /etc/kubernetes/manifests/kube-apiserver.yaml
, esigabeni command
. Nokho, i-RBAC isivele inikwe amandla ngokuzenzakalela, ngakho-ke cishe akufanele ukhathazeke ngayo: ungakuqinisekisa lokhu ngevelu. authorization-mode
(kulokho osekushiwo kube-apiserver.yaml
). Ngendlela, phakathi kwezincazelo zayo kungase kube nezinye izinhlobo zokugunyazwa (node
, webhook
, always allow
), kodwa sizokushiya ukucatshangelwa kwabo ngaphandle kobubanzi bendaba.
By the way, sesivele sishicilele
Amabhizinisi alandelayo e-API asetshenziselwa ukulawula ukufinyelela ku-Kubernetes nge-RBAC:
-
Role
иClusterRole
- izindima ezichaza amalungelo okufinyelela: -
Role
ikuvumela ukuthi uchaze amalungelo ngaphakathi kwendawo yamagama; -
ClusterRole
- ngaphakathi kweqoqo, okuhlanganisa ukuya kuzinto ezithize eziqoqiwe njengama-node, ama-url okungezona izisetshenziswa (okungukuthi angahlobene nezinsiza ze-Kubernetes - isibonelo,/version
,/logs
,/api*
); -
RoleBinding
иClusterRoleBinding
- esetshenziselwa ukubophaRole
иClusterRole
kumsebenzisi, iqembu lomsebenzisi noma i-ServiceAccount.
Amabhizinisi we-Role kanye ne-RoleBinding anqunyelwe indawo yamagama, i.e. kufanele ibe phakathi kwesikhala samagama esifanayo. Nokho, i-RoleBinding ingabhekisela ku-ClusterRole, ekuvumela ukuthi udale isethi yezimvume ezijwayelekile futhi ulawule ukufinyelela uzisebenzisa.
Izindima zichaza amalungelo zisebenzisa amasethi emithetho equkethe:
- Amaqembu e-API - bona
imibhalo esemthethweni nge-apiGroups nokuphumayokubectl api-resources
; - izinsiza (izinsiza:
pod
,namespace
,deployment
njalo njalo.); - Izenzo (tento:
set
,update
njalo njalo.). - amagama wezinsiza (
resourceNames
) - esimweni lapho udinga ukunikeza ukufinyelela esisetshenziswa esithile, hhayi kuzo zonke izinsiza zalolu hlobo.
Ukuhlaziywa okuningiliziwe kokugunyazwa ku-Kubernetes kungatholakala ekhasini
Izibonelo zamabhizinisi e-RBAC
Kulula Role
, okukuvumela ukuthi uthole uhlu nesimo sama-pods futhi uwaqaphe endaweni yamagama target-namespace
:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: target-namespace
name: pod-reader
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
Isibonelo: ClusterRole
, okukuvumela ukuthi uthole uhlu nesimo sama-pods futhi uwaqaphe kulo lonke iqoqo:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
# секции "namespace" нет, так как ClusterRole задействует весь кластер
name: secret-reader
rules:
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "watch", "list"]
Isibonelo: RoleBinding
, okuvumela umsebenzisi mynewuser
"funda" ama-pods endaweni yamagama my-namespace
:
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-pods
namespace: target-namespace
subjects:
- kind: User
name: mynewuser # имя пользователя зависимо от регистра!
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: Role # здесь должно быть “Role” или “ClusterRole”
name: pod-reader # имя Role, что находится в том же namespace,
# или имя ClusterRole, использование которой
# хотим разрешить пользователю
apiGroup: rbac.authorization.k8s.io
Ukuhlolwa komcimbi
Ngokuhlelekile, i-architecture ye-Kubernetes ingamelwa kanje:
Ingxenye eyinhloko ye-Kubernetes enesibopho sokucubungula izicelo yile i-api-server. Yonke imisebenzi eku-cluster idlula kuyo. Ungafunda kabanzi mayelana nalezi zindlela zangaphakathi esihlokweni esithi “
Ukuhlolwa kwesistimu kuyisici esithandekayo ku-Kubernetes, evinjwa ngokuzenzakalelayo. Ikuvumela ukuthi ungene kuwo wonke amakholi ku-Kubernetes API. Njengoba ungase uqagele, zonke izenzo ezihlobene nokuqapha kanye nokushintsha isimo seqoqo zenziwa ngale API. Incazelo enhle yamakhono ayo (njengokuvamile) ingatholakala ku
Ngakho-ke, ukuze unike amandla ukuhlola, sidinga ukudlula amapharamitha amathathu adingekayo esitsheni esiseseva ye-api, echazwe kabanzi ngezansi:
-
--audit-policy-file=/etc/kubernetes/policies/audit-policy.yaml
-
--audit-log-path=/var/log/kube-audit/audit.log
-
--audit-log-format=json
Ngokungeziwe kulawa mapharamitha amathathu adingekayo, kunezilungiselelo eziningi ezengeziwe ezihlobene nokucwaninga: kusukela ekuzungezweni kwelogi kuya ezincazelweni ze-webhook. Isibonelo samapharamitha wokuzungezisa ilogu:
-
--audit-log-maxbackup=10
-
--audit-log-maxsize=100
-
--audit-log-maxage=7
Kodwa ngeke sigxile kuzo ngokuningiliziwe - ungathola yonke imininingwane kuyo
Njengoba sekushiwo, wonke amapharamitha asethwe ku-manifest ngokulungiselelwa kwe-api-server (ngokuzenzakalelayo /etc/kubernetes/manifests/kube-apiserver.yaml
), esigabeni command
. Ake sibuyele kumapharamitha angu-3 adingekayo futhi siwahlaziye:
-
audit-policy-file
— indlela eya kufayela le-YAML elichaza inqubomgomo yocwaningo. Sizobuyela kokuqukethwe kwayo kamuva, kodwa okwamanje ngizoqaphela ukuthi ifayela kufanele lifundeke ngenqubo ye-api-server. Ngakho-ke, kuyadingeka ukuyifaka ngaphakathi kwesiqukathi, ongangeza ikhodi elandelayo ezigabeni ezifanele zokucushwa:volumeMounts: - mountPath: /etc/kubernetes/policies name: policies readOnly: true volumes: - hostPath: path: /etc/kubernetes/policies type: DirectoryOrCreate name: policies
-
audit-log-path
- indlela eya kufayela lokungena. Indlela kufanele futhi ifinyeleleke kunqubo ye-api-server, ngakho-ke sichaza ukukhuphuka kwayo ngendlela efanayo:volumeMounts: - mountPath: /var/log/kube-audit name: logs readOnly: false volumes: - hostPath: path: /var/log/kube-audit type: DirectoryOrCreate name: logs
-
audit-log-format
- Ifomethi yelogi yokuhlola. Okuzenzakalelayo ngujson
, kodwa ifomethi yombhalo wefa iyatholakala (legacy
).
Inqubomgomo Yokucwaninga
Manje mayelana nefayela elishiwo elichaza inqubomgomo yokugawula. Umqondo wokuqala wenqubomgomo yokucwaningwa kwamabhuku uthi level
, izinga lokugawula. Zimi kanje:
-
None
- ungangeni; -
Metadata
- Imethadatha yesicelo sokungena: umsebenzisi, isikhathi sokucela, insiza okuqondiswe kuyo (i-pod, indawo yamagama, njll.), uhlobo lwesenzo (isenzo), njll.; -
Request
- log imethadatha kanye nomzimba wesicelo; -
RequestResponse
- log imethadatha, umzimba wesicelo kanye nomzimba wokuphendula.
Amaleveli amabili okugcina (Request
и RequestResponse
) ungafaki izicelo ezingazange zifinyelele izinsiza (ukufinyelela kulokho okubizwa ngokuthi ama-url okungewona wezinsiza).
Futhi zonke izicelo ziyadlula izigaba eziningana:
-
RequestReceived
- isiteji lapho isicelo sitholwa umprosesa futhi asikakadluliswa ngokuqhubekayo ochungechungeni lwamaphrosesa; -
ResponseStarted
— izihloko zempendulo zithunyelwa, kodwa ngaphambi kokuba umzimba wokuphendula uthunyelwe. Kwenzelwe imibuzo ehlala isikhathi eside (isibonelo,watch
); -
ResponseComplete
- indikimba yempendulo ithunyelwe, alukho olunye ulwazi oluzothunyelwa; -
Panic
- izehlakalo zenziwa lapho kutholwa isimo esingavamile.
Ukweqa noma yiziphi izinyathelo ongazisebenzisa omitStages
.
Efayeleni lenqubomgomo, singachaza izigaba ezimbalwa ezinamaleveli okugawula ahlukene. Umthetho wokuqala ofanayo otholakala encazelweni yenqubomgomo uzosetshenziswa.
I-kubelet daemon monitors iyashintsha ku-manifest ngokulungiselelwa kwe-api-server futhi, uma kukhona okutholiwe, iqala kabusha isiqukathi nge-api-server. Kodwa kukhona imininingwane ebalulekile: izinguquko kufayela lenqubomgomo zizozitshwa yiyo. Ngemva kokwenza izinguquko kufayela lenqubomgomo, uzodinga ukuqala kabusha i-api-server mathupha. Njengoba i-api-server iqalwa njenge kubectl delete
ngeke ibangele ukuthi iqale kabusha. Kuzodingeka ukwenze mathupha docker stop
ku-kube-masters, lapho inqubomgomo yokucwaningwa kwamabhuku ishintshiwe:
docker stop $(docker ps | grep k8s_kube-apiserver | awk '{print $1}')
Lapho unika amandla ukucwaningwa kwamabhuku, kubalulekile ukukhumbula lokho umthwalo ku-kube-apiserver uyakhula. Ikakhulukazi, ukusetshenziswa kwenkumbulo yokugcina okuqukethwe kwesicelo kuyenyuka. Ukungena ngemvume kuqala kuphela ngemva kokuthunyelwa kwesihloko sempendulo. Umthwalo nawo uncike ekucushweni kwenqubomgomo yocwaningo.
Izibonelo zezinqubomgomo
Ake sibheke ukwakheka kwamafayela enqubomgomo sisebenzisa izibonelo.
Nali ifayela elilula policy
ukuze ungene yonke into ezingeni Metadata
:
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
Kunqubomgomo ungacacisa uhlu lwabasebenzisi (Users
и ServiceAccounts
) namaqembu abasebenzisi. Isibonelo, yile ndlela esizobaziba ngayo abasebenzisi besistimu, kodwa sibhale yonke enye into ezingeni Request
:
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: None
userGroups:
- "system:serviceaccounts"
- "system:nodes"
users:
- "system:anonymous"
- "system:apiserver"
- "system:kube-controller-manager"
- "system:kube-scheduler"
- level: Request
Kungenzeka futhi ukuchaza okuhlosiwe:
- izikhala zamagama (
namespaces
); - Izenzo (tento:
get
,update
,delete
nabanye); - izinsiza (izinsiza, okungukuthi:
pod
,configmaps
njll.) kanye namaqembu ezinsiza (apiGroups
).
Nakani! Izinsiza namaqembu wezinsiza (amaqembu e-API, okungukuthi i-apiGroups), kanye nezinguqulo zawo ezifakwe kuqoqo, zingatholwa kusetshenziswa imiyalo:
kubectl api-resources
kubectl api-versions
Le nqubomgomo yocwaningomabhuku elandelayo inikezwa njengendlela yokubonisa imikhuba emihle kakhulu
apiVersion: audit.k8s.io/v1beta1
kind: Policy
# Не логировать стадию RequestReceived
omitStages:
- "RequestReceived"
rules:
# Не логировать события, считающиеся малозначительными и не опасными:
- level: None
users: ["system:kube-proxy"]
verbs: ["watch"]
resources:
- group: "" # это api group с пустым именем, к которому относятся
# базовые ресурсы Kubernetes, называемые “core”
resources: ["endpoints", "services"]
- level: None
users: ["system:unsecured"]
namespaces: ["kube-system"]
verbs: ["get"]
resources:
- group: "" # core
resources: ["configmaps"]
- level: None
users: ["kubelet"]
verbs: ["get"]
resources:
- group: "" # core
resources: ["nodes"]
- level: None
userGroups: ["system:nodes"]
verbs: ["get"]
resources:
- group: "" # core
resources: ["nodes"]
- level: None
users:
- system:kube-controller-manager
- system:kube-scheduler
- system:serviceaccount:kube-system:endpoint-controller
verbs: ["get", "update"]
namespaces: ["kube-system"]
resources:
- group: "" # core
resources: ["endpoints"]
- level: None
users: ["system:apiserver"]
verbs: ["get"]
resources:
- group: "" # core
resources: ["namespaces"]
# Не логировать обращения к read-only URLs:
- level: None
nonResourceURLs:
- /healthz*
- /version
- /swagger*
# Не логировать сообщения, относящиеся к типу ресурсов “события”:
- level: None
resources:
- group: "" # core
resources: ["events"]
# Ресурсы типа Secret, ConfigMap и TokenReview могут содержать секретные данные,
# поэтому логируем только метаданные связанных с ними запросов
- level: Metadata
resources:
- group: "" # core
resources: ["secrets", "configmaps"]
- group: authentication.k8s.io
resources: ["tokenreviews"]
# Действия типа get, list и watch могут быть ресурсоёмкими; не логируем их
- level: Request
verbs: ["get", "list", "watch"]
resources:
- group: "" # core
- group: "admissionregistration.k8s.io"
- group: "apps"
- group: "authentication.k8s.io"
- group: "authorization.k8s.io"
- group: "autoscaling"
- group: "batch"
- group: "certificates.k8s.io"
- group: "extensions"
- group: "networking.k8s.io"
- group: "policy"
- group: "rbac.authorization.k8s.io"
- group: "settings.k8s.io"
- group: "storage.k8s.io"
# Уровень логирования по умолчанию для стандартных ресурсов API
- level: RequestResponse
resources:
- group: "" # core
- group: "admissionregistration.k8s.io"
- group: "apps"
- group: "authentication.k8s.io"
- group: "authorization.k8s.io"
- group: "autoscaling"
- group: "batch"
- group: "certificates.k8s.io"
- group: "extensions"
- group: "networking.k8s.io"
- group: "policy"
- group: "rbac.authorization.k8s.io"
- group: "settings.k8s.io"
- group: "storage.k8s.io"
# Уровень логирования по умолчанию для всех остальных запросов
- level: Metadata
Esinye isibonelo esihle senqubomgomo yocwaningomabhuku
Ukuze uphendule ngokushesha emicimbini yocwaningo, kungenzeka chaza i-webhook. Lolu daba lufakwe ngaphakathi
Imiphumela
I-athikili ihlinzeka ngesifinyezo sezindlela zokuphepha eziyisisekelo kumaqoqo e-Kubernetes, akuvumela ukuthi udale ama-akhawunti omsebenzisi aqondene nawe, uhlukanise amalungelo abo, futhi urekhode izenzo zabo. Ngithemba ukuthi kuzoba usizo kulabo ababhekene nezinkinga ezinjalo ngombono noma ekusebenzeni. Ngiphinde ngincoma ukuthi ufunde uhlu lwezinye izinto eziphathelene nokuphepha ku-Kubernetes, olunikezwa ku- "PS" - mhlawumbe phakathi kwazo uzothola imininingwane edingekayo ngezinkinga ezihambisana nawe.
PS
Funda futhi kubhulogi yethu:
- «
33+ amathuluzi okuphepha e-Kubernetes "; - «
Isingeniso se-Kubernetes Network Policies for Security Professionals "; - «
Ukuqonda i-RBAC ku-Kubernetes "; - «
9 Izindlela Ezinhle Kakhulu Zokuphepha ze-Kubernetes "; - «
Izindlela Eziyi-11 Zokuba (Hhayi) Ukuba yisisulu Se-Kubernetes Hack ".
Source: www.habr.com