Kubernetes 1.16: ืกืงื™ืจื” ื›ืœืœื™ืช ืฉืœ ื”ื—ื™ื“ื•ืฉื™ื ื”ืขื™ืงืจื™ื™ื

Kubernetes 1.16: ืกืงื™ืจื” ื›ืœืœื™ืช ืฉืœ ื”ื—ื™ื“ื•ืฉื™ื ื”ืขื™ืงืจื™ื™ื

ื”ื™ื•ื, ื™ื•ื ืจื‘ื™ืขื™, ืžืชืจื—ืฉ ื”ืžื”ื“ื•ืจื” ื”ื‘ืื” ืฉืœ Kubernetes - 1.16. ืขืœ ืคื™ ื”ืžืกื•ืจืช ืฉื”ืชืคืชื—ื” ืœื‘ืœื•ื’ ืฉืœื ื•, ื–ื• ื”ืคืขื ื”ืขืฉื™ืจื™ืช ืฉืื ื• ืžื“ื‘ืจื™ื ืขืœ ื”ืฉื™ื ื•ื™ื™ื ื”ืžืฉืžืขื•ืชื™ื™ื ื‘ื™ื•ืชืจ ื‘ื’ืจืกื” ื”ื—ื“ืฉื”.

ื”ืžื™ื“ืข ื”ืžืฉืžืฉ ืœื”ื›ื ืช ื—ื•ืžืจ ื–ื” ื ืœืงื— ืž ื˜ื‘ืœืื•ืช ืžืขืงื‘ ืœืฉื™ืคื•ืจื™ Kubernetes, CHANGELOG-1.16 ื•ื‘ืขื™ื•ืช ืงืฉื•ืจื•ืช, ื‘ืงืฉื•ืช ืžืฉื™ื›ื” ื•ื”ืฆืขื•ืช ืฉื™ืคื•ืจ ืฉืœ Kubernetes (KEP). ืื– ื‘ื•ื ื ืœืš!..

ืฆืžืชื™ื

ืžืกืคืจ ื’ื“ื•ืœ ื‘ืืžืช ืฉืœ ื—ื™ื“ื•ืฉื™ื ื‘ื•ืœื˜ื™ื (ื‘ืกื˜ื˜ื•ืก ื’ืจืกืช ืืœืคื) ืžื•ืฆื’ื™ื ื‘ืฆื“ ืฉืœ ืฆืžืชื™ ื”ืืฉื›ื•ืœ K8s (Kubelet).

ืจืืฉื™ืช, ืžื” ืฉื ืงืจื ยซืžื™ื›ืœื™ื ืืจืขื™ื™ืยป (ืžื›ื•ืœื•ืช ืืจืขื™ื•ืช), ืฉื ื•ืขื“ ืœืคืฉื˜ ืชื”ืœื™ื›ื™ ื ื™ืคื•ื™ ื‘ืื’ื™ื ื‘-pods. ื”ืžื ื’ื ื•ืŸ ื”ื—ื“ืฉ ืžืืคืฉืจ ืœื”ืฉื™ืง ืงื•ื ื˜ื™ื™ื ืจื™ื ืžื™ื•ื—ื“ื™ื ืฉืžืชื—ื™ืœื™ื ื‘ืžืจื—ื‘ ื”ืฉืžื•ืช ืฉืœ ืคื•ื“ื™ื ืงื™ื™ืžื™ื ื•ื—ื™ื™ื ืœื–ืžืŸ ืงืฆืจ. ื”ืžื˜ืจื” ืฉืœื”ื ื”ื™ื ืœื™ืฆื•ืจ ืื™ื ื˜ืจืืงืฆื™ื” ืขื ืคื•ื“ื™ื ื•ืžื™ื›ืœื™ื ืื—ืจื™ื ืขืœ ืžื ืช ืœืคืชื•ืจ ื‘ืขื™ื•ืช ื›ืœืฉื”ืŸ ื•ืœืคืชื•ืจ ื‘ืื’ื™ื. ืคืงื•ื“ื” ื—ื“ืฉื” ื™ื•ืฉืžื” ืขื‘ื•ืจ ืชื›ื•ื ื” ื–ื• kubectl debug, ื“ื•ืžื” ื‘ืžื”ื•ืชื• ืœ kubectl exec: ืจืง ื‘ืžืงื•ื ืœื”ืคืขื™ืœ ืชื”ืœื™ืš ื‘ืžื™ื›ืœ (ื›ืžื• ื‘ exec) ื”ื•ื ืžืฉื’ืจ ืžื™ื›ืœ ื‘ืชืจืžื™ืœ. ืœื“ื•ื’ืžื”, ืคืงื•ื“ื” ื–ื• ืชื—ื‘ืจ ืžื™ื›ืœ ื—ื“ืฉ ืœืชืจืžื™ืœ:

kubectl debug -c debug-shell --image=debian target-pod -- bash

ืคืจื˜ื™ื ืขืœ ืžื™ื›ืœื™ื ืืจืขื™ื™ื (ื•ื“ื•ื’ืžืื•ืช ืœืฉื™ืžื•ืฉ ื‘ื”ื) ื ื™ืชืŸ ืœืžืฆื•ื ื‘ KEP ื”ืžืงื‘ื™ืœ. ื”ื™ื™ืฉื•ื ื”ื ื•ื›ื—ื™ (ื‘-K8s 1.16) ื”ื•ื ื’ืจืกืช ืืœืคื, ื•ื‘ื™ืŸ ื”ืงืจื™ื˜ืจื™ื•ื ื™ื ืœื”ืขื‘ืจืชื• ืœื’ืจืกืช ื‘ื˜ื ื”ื•ื "ื‘ื“ื™ืงืช ื”-API ืฉืœ Ephemeral Containers ืขื‘ื•ืจ ืœืคื—ื•ืช 2 ืžื”ื“ื•ืจื•ืช ืฉืœ [Kubernetes]."

NB: ื‘ืžื”ื•ืชื• ื•ืืคื™ืœื• ื‘ืฉืžื”, ื”ืชื›ื•ื ื” ื“ื•ืžื” ืœืชื•ืกืฃ ืฉื›ื‘ืจ ืงื™ื™ื kubectl-debugืฉืขืœื™ื• ืื ื—ื ื• ื›ื‘ืจ ื›ืชื‘. ืฆืคื•ื™ ืฉืขื ื›ื ื™ืกืชื ืฉืœ ืงื•ื ื˜ื™ื™ื ืจื™ื ืืจืขื™ื™ื ื™ื™ืคืกืง ืคื™ืชื•ื—ื• ืฉืœ ืชื•ืกืฃ ื—ื™ืฆื•ื ื™ ื ืคืจื“.

ื—ื™ื“ื•ืฉ ื ื•ืกืฃ - PodOverhead - ื ื•ืขื“ ืœืกืคืง ืžื ื’ื ื•ืŸ ืœื—ื™ืฉื•ื‘ ืขืœื•ื™ื•ืช ืชืงื•ืจื” ืœืชืจืžื™ืœื™ื, ืฉื™ื›ื•ืœ ืœื”ืฉืชื ื•ืช ืžืื•ื“ ื‘ื”ืชืื ืœื–ืžืŸ ื”ืจื™ืฆื” ื‘ืฉื™ืžื•ืฉ. ื›ื“ื•ื’ืžื”, ื”ืžื—ื‘ืจื™ื ื”-KEP ื”ื–ื” ืชื•ืฆืื” ื‘-Kata Containers, ื”ื“ื•ืจืฉื™ื ื”ืคืขืœืช ืœื™ื‘ืช ื”ืื•ืจื—, ืกื•ื›ืŸ ื”ืงืื˜ื”, ืžืขืจื›ืช init ื•ื›ื•'. ื›ืืฉืจ ื”ืชืงื•ืจื” ื”ื•ืคื›ืช ื›ืœ ื›ืš ื’ื“ื•ืœื”, ืื™ ืืคืฉืจ ืœื”ืชืขืœื ืžืžื ื”, ืžื” ืฉืื•ืžืจ ืฉืฆืจื™ื›ื” ืœื”ื™ื•ืช ื“ืจืš ืœืงื—ืช ืื•ืชื” ื‘ื—ืฉื‘ื•ืŸ ืœื”ืžืฉืš ืžื›ืกื•ืช, ืชื›ื ื•ืŸ ื•ื›ื•'. ืœื™ื™ืฉื ืืช ื–ื” ื‘ PodSpec ืฉื“ื” ื ื•ืกืฃ Overhead *ResourceList (ืžืฉื•ื•ื” ืœื ืชื•ื ื™ื ื‘ RuntimeClass, ืื ื ืขืฉื” ืฉื™ืžื•ืฉ ื›ื–ื”).

ื—ื™ื“ื•ืฉ ื‘ื•ืœื˜ ื ื•ืกืฃ ื”ื•ื ืžื ื”ืœ ื˜ื•ืคื•ืœื•ื’ื™ื™ืช ืฆืžืชื™ื (ืžื ื”ืœ ื˜ื•ืคื•ืœื•ื’ื™ื™ืช ื”ืฆืžืชื™ื), ืฉื ื•ืขื“ ืœืื—ื“ ืืช ื”ื’ื™ืฉื” ืœื›ื•ื•ื ื•ืŸ ืขื“ื™ืŸ ืฉืœ ื”ืงืฆืืช ืžืฉืื‘ื™ ื”ื—ื•ืžืจื” ืขื‘ื•ืจ ืจื›ื™ื‘ื™ื ืฉื•ื ื™ื ื‘-Kubernetes. ื™ื•ื–ืžื” ื–ื• ืžื•ื ืขืช ืžื”ืฆื•ืจืš ื”ื”ื•ืœืš ื•ื’ื•ื‘ืจ ืฉืœ ืžืขืจื›ื•ืช ืžื•ื“ืจื ื™ื•ืช ืฉื•ื ื•ืช (ืžืชื—ื•ื ื”ื˜ืœืงื•ืžื•ื ื™ืงืฆื™ื”, ืœืžื™ื“ืช ืžื›ื•ื ื”, ืฉื™ืจื•ืชื™ื ืคื™ื ื ืกื™ื™ื ื•ื›ื•') ืœืžื—ืฉื•ื‘ ืžืงื‘ื™ืœ ื‘ืขืœ ื‘ื™ืฆื•ืขื™ื ื’ื‘ื•ื”ื™ื ื•ืœืžื–ืขื•ืจ ืขื™ื›ื•ื‘ื™ื ื‘ื‘ื™ืฆื•ืข ื”ืคืขื•ืœื•ืช, ืขื‘ื•ืจืŸ ื”ืŸ ืžืฉืชืžืฉื•ืช ื‘ืžืขื‘ื“ ืžืชืงื“ื ื™ื›ื•ืœื•ืช ื”ืืฆืช ื—ื•ืžืจื”. ืื•ืคื˜ื™ืžื™ื–ืฆื™ื•ืช ื›ืืœื” ื‘-Kubernetes ื”ื•ืฉื’ื• ืขื“ ื›ื” ื”ื•ื“ื•ืช ืœืจื›ื™ื‘ื™ื ืฉื•ื ื™ื (ืžื ื”ืœ ืžืขื‘ื“, ืžื ื”ืœ ื”ืชืงื ื™ื, CNI), ื•ื›ืขืช ื™ืชื•ื•ืกืฃ ืœื”ื ืžืžืฉืง ืคื ื™ืžื™ ื™ื—ื™ื“ ืฉืžืื—ื“ ืืช ื”ื’ื™ืฉื” ื•ืžืคืฉื˜ ืืช ื”ื—ื™ื‘ื•ืจ ืฉืœ ื“ื•ืžื™ื ื—ื“ืฉื™ื - ืžื” ืฉื ืงืจื ื˜ื•ืคื•ืœื•ื’ื™ื”- ืžื•ื“ืข - ืจื›ื™ื‘ื™ื ื‘ืฆื“ Kubelet. ืคืจื˜ื™ื - ื‘ KEP ื”ืžืงื‘ื™ืœ.

Kubernetes 1.16: ืกืงื™ืจื” ื›ืœืœื™ืช ืฉืœ ื”ื—ื™ื“ื•ืฉื™ื ื”ืขื™ืงืจื™ื™ื
ืชืจืฉื™ื ืจื›ื™ื‘ื™ ืžื ื”ืœ ื˜ื•ืคื•ืœื•ื’ื™ื”

ื”ืชื›ื•ื ื” ื”ื‘ืื” - ื‘ื“ื™ืงืช ืžื™ื›ืœื™ื ื‘ื–ืžืŸ ืฉื”ื ืคื•ืขืœื™ื (ื‘ื“ื™ืงืช ื”ืคืขืœื”). ื›ื™ื“ื•ืข, ืขื‘ื•ืจ ืžื›ื•ืœื•ืช ืฉืœื•ืงื— ื–ืžืŸ ืœื”ืฉืงืชืŸ, ืงืฉื” ืœืงื‘ืœ ืกื˜ื˜ื•ืก ืขื“ื›ื ื™: ืื• ืฉื”ืŸ "ื ื”ืจื’ื•ืช" ืœืคื ื™ ืฉื”ืŸ ืžืชื—ื™ืœื•ืช ืœืคืขื•ืœ ื‘ืคื•ืขืœ, ืื• ืฉื”ืŸ ืžืกืชื™ื™ืžื•ืช ื‘ืžื‘ื•ื™ ืกืชื•ื ืœืžืฉืš ื–ืžืŸ ืจื‘. ื‘ื“ื™ืงื” ื—ื“ืฉื” (ืžื•ืคืขืœ ื“ืจืš ืฉืขืจ ืชื›ื•ื ื” ื ืงืจื StartupProbeEnabled) ืžื‘ื˜ืœ - ืื• ื™ื•ืชืจ ื ื›ื•ืŸ, ื“ื•ื—ื” - ืืช ื”ืฉืคืขืช ื›ืœ ื‘ื“ื™ืงื” ืื—ืจืช ืขื“ ืœืจื’ืข ืฉื”ืคื•ื“ ืกื™ื™ื ืœืคืขื•ืœ. ืžืกื™ื‘ื” ื–ื•, ื”ืชื›ื•ื ื” ื ืงืจืื” ื‘ืžืงื•ืจ pod-startup liveness-probe holdoff. ืขื‘ื•ืจ ืคื•ื“ื™ื ืฉืœื•ืงื— ืœื”ื ื–ืžืŸ ืจื‘ ืœื”ืชื—ื™ืœ, ืืชื” ื™ื›ื•ืœ ืœืกืงื•ืจ ืืช ื”ืžืฆื‘ ื‘ืžืจื•ื•ื—ื™ ื–ืžืŸ ืงืฆืจื™ื ื™ื—ืกื™ืช.

ื‘ื ื•ืกืฃ, ืฉื™ืคื•ืจ ืขื‘ื•ืจ RuntimeClass ื–ืžื™ืŸ ื‘ืื•ืคืŸ ืžื™ื™ื“ื™ ื‘ืกื˜ื˜ื•ืก ื‘ื˜ื, ื•ืžื•ืกื™ืฃ ืชืžื™ื›ื” ื‘"ืืฉื›ื•ืœื•ืช ื”ื˜ืจื•ื’ื ื™ื™ื". ื’ ืชื–ืžื•ืŸ RuntimeClass ื›ืขืช ืื™ืŸ ืฆื•ืจืš ื›ืœืœ ืฉืœื›ืœ ืฆื•ืžืช ืชื”ื™ื” ืชืžื™ื›ื” ืขื‘ื•ืจ ื›ืœ RuntimeClass: ืขื‘ื•ืจ ืคื•ื“ื™ื ืืชื” ื™ื›ื•ืœ ืœื‘ื—ื•ืจ RuntimeClass ืžื‘ืœื™ ืœื—ืฉื•ื‘ ืขืœ ื˜ื•ืคื•ืœื•ื’ื™ื™ืช ื”ืืฉื›ื•ืœื•ืช. ื‘ืขื‘ืจ, ื›ื“ื™ ืœื”ืฉื™ื’ ื–ืืช - ื›ื“ื™ ืฉืชืจืžื™ืœื™ื ื™ื’ื™ืขื• ืœืฆืžืชื™ื ืขื ืชืžื™ื›ื” ื‘ื›ืœ ืžื” ืฉื”ื ืฆืจื™ื›ื™ื - ื”ื™ื” ืฆื•ืจืš ืœื”ืงืฆื•ืช ื›ืœืœื™ื ืžืชืื™ืžื™ื ืœ-NodeSelector ื•ืกื•ื‘ืœื ื•ืช. IN ืงืค ื”ื•ื ืžื“ื‘ืจ ืขืœ ื“ื•ื’ืžืื•ืช ืœืฉื™ืžื•ืฉ ื•ื›ืžื•ื‘ืŸ ืคืจื˜ื™ ื™ื™ืฉื•ื.

ืจืฉืช

ืฉืชื™ ืชื›ื•ื ื•ืช ืจืฉืช ืžืฉืžืขื•ืชื™ื•ืช ืฉื”ื•ืคื™ืขื• ืœืจืืฉื•ื ื” (ื‘ื’ืจืกืช ืืœืคื) ื‘-Kubernetes 1.16 ื”ืŸ:

  • ืชืžื™ื›ื” ืขืจื™ืžืช ืจืฉืช ื›ืคื•ืœื” - IPv4/IPv6 - ื•ื”"ื”ื‘ื ื”" ื”ืžืงื‘ื™ืœื” ืฉืœื• ื‘ืจืžืช ื”ืชืจืžื™ืœื™ื, ื”ืฆืžืชื™ื, ื”ืฉื™ืจื•ืชื™ื. ื–ื” ื›ื•ืœืœ ื™ื›ื•ืœืช ืคืขื•ืœื” ื”ื“ื“ื™ืช ืฉืœ IPv4-to-IPv4 ื•-IPv6-to-IPv6 ื‘ื™ืŸ ืคื•ื“ื™ื, ืž-pods ื•ืขื“ ืฉื™ืจื•ืชื™ื ื—ื™ืฆื•ื ื™ื™ื, ื”ื˜ืžืขื•ืช ื”ืชื™ื™ื—ืกื•ืช (ื‘ืชื•ืš ื”ืชื•ืกืคื™ื Bridge CNI, PTP CNI ื•- Host-Local IPAM), ื•ื›ืŸ ืชื•ืื ืœืื—ื•ืจ ืขื ืืฉื›ื•ืœื•ืช Kubernetes ื”ืคื•ืขืœื™ื IPv4 ืื• IPv6 ื‘ืœื‘ื“. ืคืจื˜ื™ ื”ื™ื™ืฉื•ื ื ืžืฆืื™ื ืงืค.

    ื“ื•ื’ืžื” ืœื”ืฆื’ืช ื›ืชื•ื‘ื•ืช IP ืžืฉื ื™ ืกื•ื’ื™ื (IPv4 ื•-IPv6) ื‘ืจืฉื™ืžืช ื”ืคื•ื“ื™ื:

    kube-master# kubectl get pods -o wide
    NAME               READY     STATUS    RESTARTS   AGE       IP                          NODE
    nginx-controller   1/1       Running   0          20m       fd00:db8:1::2,192.168.1.3   kube-minion-1
    kube-master#

  • API ื—ื“ืฉ ืขื‘ื•ืจ ื ืงื•ื“ืช ืงืฆื” - EndpointSlice API. ื–ื” ืคื•ืชืจ ืืช ื‘ืขื™ื•ืช ื”ื‘ื™ืฆื•ืขื™ื/ืกืงืœื‘ื™ืœื™ื•ืช ืฉืœ ื”-Endpoint API ื”ืงื™ื™ื ืฉืžืฉืคื™ืขื•ืช ืขืœ ืจื›ื™ื‘ื™ื ืฉื•ื ื™ื ื‘ืžื™ืฉื•ืจ ื”ื‘ืงืจื” (apiserver, ื•ื›ื•', ืงืฆื”-ื‘ืงืจ, kube-proxy). ื”-API ื”ื—ื“ืฉ ื™ืชื•ื•ืกืฃ ืœืงื‘ื•ืฆืช Discovery API ื•ื™ื•ื›ืœ ืœืฉืจืช ืขืฉืจื•ืช ืืœืคื™ ื ืงื•ื“ื•ืช ืงืฆื” ืขื•ืจืคื™ื•ืช ื‘ื›ืœ ืฉื™ืจื•ืช ื‘ืืฉื›ื•ืœ ื”ืžื•ืจื›ื‘ ืžืืœืคื™ ืฆืžืชื™ื. ืœืฉื ื›ืš, ื›ืœ ืฉื™ืจื•ืช ืžืžื•ืคื” ืœ-N ืื•ื‘ื™ื™ืงื˜ื™ื EndpointSlice, ืฉืœื›ืœ ืื—ืช ืžื”ืŸ ื›ื‘ืจื™ืจืช ืžื—ื“ืœ ืื™ืŸ ื™ื•ืชืจ ืž-100 ื ืงื•ื“ื•ืช ืงืฆื” (ื”ืขืจืš ื ื™ืชืŸ ืœื”ื’ื“ืจื”). ื”-API ืฉืœ EndpointSlice ื™ืกืคืง ื’ื ื”ื–ื“ืžื ื•ื™ื•ืช ืœืคื™ืชื•ื— ืขืชื™ื“ื™ ืฉืœื•: ืชืžื™ื›ื” ื‘ืžืกืคืจ ื›ืชื•ื‘ื•ืช IP ืขื‘ื•ืจ ื›ืœ ืคื•ื“, ืžืฆื‘ื™ื ื—ื“ืฉื™ื ืขื‘ื•ืจ ื ืงื•ื“ื•ืช ืงืฆื” (ืœื ืจืง Ready ะธ NotReady), ื”ื’ื“ืจืช ืžืฉื ื” ื“ื™ื ืžื™ืช ืขื‘ื•ืจ ื ืงื•ื“ื•ืช ืงืฆื”.

ื–ื” ืฉื”ื•ืฆื’ ื‘ืžื”ื“ื•ืจื” ื”ืื—ืจื•ื ื” ื”ื’ื™ืข ืœื’ืจืกืช ื”ื‘ื˜ื ื’ืžืจ, ื‘ืฉื service.kubernetes.io/load-balancer-cleanup ื•ืžืฆื•ืจืคืช ืœื›ืœ ืฉื™ืจื•ืช ืขื ืกื•ื’ LoadBalancer. ื‘ื–ืžืŸ ืžื—ื™ืงืช ืฉื™ืจื•ืช ื›ื–ื”, ื”ื•ื ืžื•ื ืข ืืช ื”ืžื—ื™ืงื” ื‘ืคื•ืขืœ ืฉืœ ื”ืžืฉืื‘ ืขื“ ืœื”ืฉืœืžืช ื”"ื ื™ืงื•ื™" ืฉืœ ื›ืœ ืžืฉืื‘ื™ ื”ืื™ื–ื•ืŸ ื”ืจืœื•ื•ื ื˜ื™ื™ื.

ืžื›ื•ื ื•ืช API

"ืื‘ืŸ ื”ื“ืจืš ืœื™ื™ืฆื•ื‘" ื”ืืžื™ืชื™ืช ื”ื™ื ื‘ืื–ื•ืจ ืฉืจืช ื”-API ืฉืœ Kubernetes ื•ื”ืื™ื ื˜ืจืืงืฆื™ื” ืื™ืชื•. ื–ื” ืงืจื” ื‘ืขื™ืงืจ ื‘ื–ื›ื•ืช ื”ืขื‘ืจืช ืœืžืขืžื“ ื™ืฆื™ื‘ ืืช ืžื™ ืฉืœื ืฆืจื™ืš ื”ื™ื›ืจื•ืช ืžื™ื•ื—ื“ืช CustomResourceDefinitions (CRD), ื‘ืขืœื™ ืกื˜ื˜ื•ืก ื‘ื˜ื ืžืื– ื”ื™ืžื™ื ื”ืจื—ื•ืงื™ื ืฉืœ Kubernetes 1.7 (ื•ื–ื” ื™ื•ื ื™ 2017!). ืื•ืชื• ื™ื™ืฆื•ื‘ ื”ื’ื™ืข ืœืชื›ื•ื ื•ืช ื”ืงืฉื•ืจื•ืช:

  • "ืžืฉืื‘ื™ ืžืฉื ื”" ืขื /status ะธ /scale ืขื‘ื•ืจ CustomResources;
  • ื˜ืจื ืกืคื•ืจืžืฆื™ื” ื’ืจืกืื•ืช ืขื‘ื•ืจ CRD, ื”ืžื‘ื•ืกืกื•ืช ืขืœ webhook ื—ื™ืฆื•ื ื™;
  • ื”ื•ืฆื’ ืœืื—ืจื•ื ื” (ื‘-K8s 1.15) ืขืจื›ื™ ื‘ืจื™ืจืช ืžื—ื“ืœ (ื‘ืจื™ืจืช ืžื—ื“ืœ) ื•ื”ืกืจืช ืฉื“ื•ืช ืื•ื˜ื•ืžื˜ื™ืช (ืงึดืฆื•ึผืฅ) ืขื‘ื•ืจ CustomResources;
  • ื”ื–ื“ืžื ื•ืช ืฉื™ืžื•ืฉ ื‘ืกื›ื™ืžืช OpenAPI v3 ื›ื“ื™ ืœื™ืฆื•ืจ ื•ืœืคืจืกื ืชื™ืขื•ื“ OpenAPI ื”ืžืฉืžืฉ ืœืื™ืžื•ืช ืžืฉืื‘ื™ CRD ื‘ืฆื“ ื”ืฉืจืช.

ืžื ื’ื ื•ืŸ ื ื•ืกืฃ ืฉื”ืคืš ืžื•ื›ืจ ืžื–ืžืŸ ืœืžื ื”ืœื™ Kubernetes: ื”ื•ืง ืžืงื•ื•ืŸ - ื’ื ื ืฉืืจ ื‘ืกื˜ื˜ื•ืก ื‘ื˜ื ื‘ืžืฉืš ื–ืžืŸ ืจื‘ (ืžืื– K8s 1.9) ื•ื›ืขืช ื”ื•ื ืžื•ื›ืจื– ื™ืฆื™ื‘.

ืฉืชื™ ืชื›ื•ื ื•ืช ื ื•ืกืคื•ืช ื”ื’ื™ืขื• ืœื‘ื˜ื: ื—ืœื™ื ื‘ืฆื“ ื”ืฉืจืช ะธ ืœืฆืคื•ืช ื‘ืกื™ืžื ื™ื•ืช.

ื•ื”ื—ื™ื“ื•ืฉ ื”ืžืฉืžืขื•ืชื™ ื”ื™ื—ื™ื“ ื‘ื’ืจืกืช ื”ืืœืคื ื”ื™ื” ื›ื™ืฉืœื•ืŸ ืž SelfLink - URI ืžื™ื•ื—ื“ ื”ืžื™ื™ืฆื’ ืืช ื”ืื•ื‘ื™ื™ืงื˜ ืฉืฆื•ื™ืŸ ื•ืžื”ื•ื•ื” ื—ืœืง ืžืžื ื• ObjectMeta ะธ ListMeta (ื›ืœื•ืžืจ ื—ืœืง ืžื›ืœ ืื•ื‘ื™ื™ืงื˜ ื‘-Kubernetes). ืœืžื” ื”ื ื ื•ื˜ืฉื™ื ืืช ื–ื”? ืžื•ื˜ื™ื‘ืฆื™ื” ื‘ืฆื•ืจื” ืคืฉื•ื˜ื” ืงื•ืœื•ืช ื›ืžื• ื”ื™ืขื“ืจ ืกื™ื‘ื•ืช ืืžื™ืชื™ื•ืช (ืžื›ืจื™ืขื•ืช) ืœื›ืš ืฉืชื—ื•ื ื–ื” ืขื“ื™ื™ืŸ ืงื™ื™ื. ืกื™ื‘ื•ืช ืจืฉืžื™ื•ืช ื™ื•ืชืจ ื”ืŸ ืœื™ื™ืขืœ ืืช ื”ื‘ื™ืฆื•ืขื™ื (ืขืœ ื™ื“ื™ ื”ืกืจืช ืฉื“ื” ืžื™ื•ืชืจ) ื•ืœืคืฉื˜ ืืช ื”ืขื‘ื•ื“ื” ืฉืœ ื”-apiserver ื”ื’ื ืจื™, ืฉื ืืœืฅ ืœื˜ืคืœ ื‘ืฉื“ื” ื›ื–ื” ื‘ืฆื•ืจื” ืžื™ื•ื—ื“ืช (ื–ื”ื• ื”ืฉื“ื” ื”ื™ื—ื™ื“ ืฉืžื•ื’ื“ืจ ืžืžืฉ ืœืคื ื™ ื”ืื•ื‘ื™ื™ืงื˜ ืžืกื•ื“ืจืช). ื”ืชื™ื™ืฉื ื•ืช ืืžื™ืชื™ืช (ื‘ืชื•ืš ื‘ื˜ื) SelfLink ื™ืชืจื—ืฉ ืขืœ ื™ื“ื™ Kubernetes ื’ืจืกื” 1.20, ื•ืกื•ืคื™ - 1.21.

ืื—ืกื•ืŸ ื ืชื•ื ื™ื

ื”ืขื‘ื•ื“ื” ื”ืขื™ืงืจื™ืช ื‘ืชื—ื•ื ื”ืื—ืกื•ืŸ, ื›ืžื• ื‘ืžื”ื“ื•ืจื•ืช ืงื•ื“ืžื•ืช, ื ืฆืคืชื” ื‘ืื–ื•ืจ ืชืžื™ื›ื” ื‘-CSI. ื”ืฉื™ื ื•ื™ื™ื ื”ืขื™ืงืจื™ื™ื ื›ืืŸ ื”ื™ื•:

  • ื‘ืคืขื ื”ืจืืฉื•ื ื” (ื‘ื’ืจืกืช ืืœืคื) ื”ื•ืคื™ืข ืชืžื™ื›ื” ื‘ืคืœืื’ื™ืŸ CSI ืœืฆืžืชื™ ืขื‘ื•ื“ื” ืฉืœ Windows: ื“ืจืš ื”ืขื‘ื•ื“ื” ื”ื ื•ื›ื—ื™ืช ืขื ืื—ืกื•ืŸ ืชื—ืœื™ืฃ ื’ื ืชื•ืกืคื™ื ื‘ืชื•ืš ื”ืขืฅ ื‘ืœื™ื‘ืช Kubernetes ื•ืชื•ืกืคื™ FlexVolume ืžื‘ื™ืช ืžื™ืงืจื•ืกื•ืคื˜ ื”ืžื‘ื•ืกืกื™ื ืขืœ Powershell;

    Kubernetes 1.16: ืกืงื™ืจื” ื›ืœืœื™ืช ืฉืœ ื”ื—ื™ื“ื•ืฉื™ื ื”ืขื™ืงืจื™ื™ื
    ืชื›ื ื™ืช ืœื”ื˜ืžืขืช ืชื•ืกืคื™ื ืฉืœ CSI ื‘-Kubernetes ืขื‘ื•ืจ Windows

  • ื”ื–ื“ืžื ื•ืช ืฉื™ื ื•ื™ ื’ื•ื“ืœ ื ืคื—ื™ CSI, ืฉื”ื•ืฆื’ ื‘ื—ื–ืจื” ื‘-K8s 1.12, ื’ื“ืœ ืœื’ืจืกืช ื‘ื˜ื;
  • "ืงื™ื“ื•ื" ื“ื•ืžื” (ืžืืœืคื ืœื‘ื˜ื) ื”ื•ืฉื’ ืขืœ ื™ื“ื™ ื”ื™ื›ื•ืœืช ืœื”ืฉืชืžืฉ ื‘-CSI ืœื™ืฆื™ืจืช ื ืคื—ื™ื ืืจืขื™ื™ื ืžืงื•ืžื™ื™ื (ืชืžื™ื›ื” ื‘ื ืคื— ืžื•ื˜ื‘ืข ื‘-CSI).

ื”ื•ืฆื’ ื‘ื’ืจืกื” ื”ืงื•ื“ืžืช ืฉืœ Kubernetes ืคื•ื ืงืฆื™ื™ืช ืฉื™ื‘ื•ื˜ ื ืคื— (ื‘ืืžืฆืขื•ืช PVC ืงื™ื™ื ื› DataSource ื›ื“ื™ ืœื™ืฆื•ืจ PVC ื—ื“ืฉ) ืงื™ื‘ืœื” ื’ื ื›ืขืช ืกื˜ื˜ื•ืก ื‘ื˜ื.

ืžืชื–ืžืŸ

ืฉื ื™ ืฉื™ื ื•ื™ื™ื ื‘ื•ืœื˜ื™ื ื‘ืชื–ืžื•ืŸ (ืฉื ื™ื”ื ื‘ืืœืคื):

  • EvenPodsSpreading - ื”ื–ื“ืžื ื•ืช ื”ืฉืชืžืฉ ื‘ืชืจืžื™ืœื™ื ื‘ืžืงื•ื ื‘ื™ื—ื™ื“ื•ืช ื™ื™ืฉื•ื ืœื•ื’ื™ื•ืช ืœ"ื—ืœื•ืงื” ื”ื•ื’ื ืช" ืฉืœ ืขื•ืžืกื™ื (ื›ืžื• Deployment ื•- ReplicaSet) ื•ื”ืชืืžืช ื”ืคืฆื” ื–ื• (ื›ื“ืจื™ืฉื” ืงืฉื” ืื• ื›ืชื ืื™ ืจืš, ื›ืœื•ืžืจ ืขื“ื™ืคื•ืช). ื”ืชื›ื•ื ื” ืชืจื—ื™ื‘ ืืช ื™ื›ื•ืœื•ืช ื”ื”ืคืฆื” ื”ืงื™ื™ืžื•ืช ืฉืœ ืคื•ื“ื™ื ืžืชื•ื›ื ื ื™ื, ื”ืžื•ื’ื‘ืœืช ื›ืขืช ืขืœ ื™ื“ื™ ืืคืฉืจื•ื™ื•ืช PodAffinity ะธ PodAntiAffinity, ื ื•ืชืŸ ืœืžื ื”ืœื™ ืžืขืจื›ืช ืฉืœื™ื˜ื” ืขื“ื™ื ื” ื™ื•ืชืจ ื‘ืขื ื™ื™ืŸ ื–ื”, ืžื” ืฉืื•ืžืจ ื–ืžื™ื ื•ืช ื’ื‘ื•ื”ื” ื™ื•ืชืจ ื•ืฆืจื™ื›ืช ืžืฉืื‘ื™ื ืื•ืคื˜ื™ืžืœื™ืช. ืคืจื˜ื™ื - ื‘ ืงืค.
  • ืœื”ืฉืชืžืฉ ืžื“ื™ื ื™ื•ืช BestFit ะฒ RequestedToCapacityRatio Priority Function ื‘ืžื”ืœืš ืชื›ื ื•ืŸ ื”ืชืจืžื™ืœ, ืžื” ืฉื™ืืคืฉืจ ืœื”ื—ื™ืœ ืืจื™ื–ืช ืคื—ื™ื ("ืืจื™ื–ื” ื‘ืžื›ื•ืœื•ืช") ื”ืŸ ืขื‘ื•ืจ ืžืฉืื‘ื™ื ื‘ืกื™ืกื™ื™ื (ืžืขื‘ื“, ื–ื™ื›ืจื•ืŸ) ื•ื”ืŸ ืขื‘ื•ืจ ืžืฉืื‘ื™ื ืžื•ืจื—ื‘ื™ื (ื›ืžื• GPU). ืœืคืจื˜ื™ื ื ื•ืกืคื™ื, ืจืื” ืงืค.

    Kubernetes 1.16: ืกืงื™ืจื” ื›ืœืœื™ืช ืฉืœ ื”ื—ื™ื“ื•ืฉื™ื ื”ืขื™ืงืจื™ื™ื
    ืชื–ืžื•ืŸ ืชืจืžื™ืœื™ื: ืœืคื ื™ ื”ืฉื™ืžื•ืฉ ื‘ืžื“ื™ื ื™ื•ืช ื”ื”ืชืืžื” ื”ื˜ื•ื‘ื” ื‘ื™ื•ืชืจ (ื™ืฉื™ืจื•ืช ื‘ืืžืฆืขื•ืช ืžืชื–ืžืŸ ื‘ืจื™ืจืช ื”ืžื—ื“ืœ) ื•ืขื ื”ืฉื™ืžื•ืฉ ื‘ื” (ื‘ืืžืฆืขื•ืช ืžืจื—ื™ื‘ ืžืชื–ืžืŸ)

ื‘ื ื•ืกืฃ, ืžื™ื•ืฆื’ ืขืœ ื™ื“ื™ ื”ื™ื›ื•ืœืช ืœื™ืฆื•ืจ ืชื•ืกืคื™ ืžืชื–ืžืŸ ืžืฉืœืš ืžื—ื•ืฅ ืœืขืฅ ื”ืคื™ืชื•ื— ื”ืจืืฉื™ ืฉืœ Kubernetes (ืžื—ื•ืฅ ืœืขืฅ).

ืฉื™ื ื•ื™ื™ื ืื—ืจื™ื

ื’ื ื‘ืžื”ื“ื•ืจืช Kubernetes 1.16 ื ื™ืชืŸ ืœืฆื™ื™ืŸ ื–ืืช ื™ื•ื–ืžื” ืขื‘ื•ืจ ืžื‘ื™ืื” ืžื“ื“ื™ื ื–ืžื™ื ื™ื ื‘ืกื“ืจ ืžืœื, ืื• ืœื™ืชืจ ื“ื™ื•ืง, ื‘ื”ืชืื ืชืงื ื•ืช ืจืฉืžื™ื•ืช ืœืžื›ืฉื•ืจ K8s. ื”ื ืžืกืชืžื›ื™ื ื‘ืžื™ื“ื” ืจื‘ื” ืขืœ ื”ืžืงื‘ื™ืœ ืชื™ืขื•ื“ ืคืจื•ืžืชืื•ืก. ื—ื•ืกืจ ืขืงื‘ื™ื•ืช ื”ืชืขื•ืจืจ ืžืกื™ื‘ื•ืช ืฉื•ื ื•ืช (ืœื“ื•ื’ืžื”, ื›ืžื” ืžื“ื“ื™ื ืคืฉื•ื˜ ื ื•ืฆืจื• ืœืคื ื™ ืฉื”ื•ืคื™ืขื• ื”ื”ื•ืจืื•ืช ื”ื ื•ื›ื—ื™ื•ืช), ื•ื”ืžืคืชื—ื™ื ื”ื—ืœื™ื˜ื• ืฉื”ื’ื™ืข ื”ื–ืžืŸ ืœื”ื‘ื™ื ื”ื›ืœ ืœืกื˜ื ื“ืจื˜ ืื—ื“, "ื‘ื”ืชืืžื” ืœืฉืืจ ื”ืžืขืจื›ืช ื”ืืงื•ืœื•ื’ื™ืช ืฉืœ ืคืจื•ืžืชืื•ืก". ื”ื™ื™ืฉื•ื ื”ื ื•ื›ื—ื™ ืฉืœ ื™ื•ื–ืžื” ื–ื• ื ืžืฆื ื‘ืกื˜ื˜ื•ืก ืืœืคื, ืืฉืจ ื™ืงื•ื“ื ื‘ื”ื“ืจื’ื” ื‘ื’ืจืกืื•ืช ื”ื‘ืื•ืช ืฉืœ Kubernetes ืœื‘ื˜ื (1.17) ื•ื™ืฆื™ื‘ (1.18).

ื‘ื ื•ืกืฃ, ื ื™ืชืŸ ืœืฆื™ื™ืŸ ืืช ื”ืฉื™ื ื•ื™ื™ื ื”ื‘ืื™ื:

  • ืชืžื™ื›ื” ื‘ืคื™ืชื•ื— Windows ั ืžืจืื” ื—ื™ืฆื•ื ื™ ื›ืœื™ ืฉื™ืจื•ืช Kubeadm ืขื‘ื•ืจ ืžืขืจื›ืช ื”ืคืขืœื” ื–ื• (ื’ืจืกืช ืืœืคื), ื”ึดื–ื“ึทืžึฐื ื•ึผืช RunAsUserName ืขื‘ื•ืจ ืžื™ื›ืœื™ Windows (ื’ืจืกืช ืืœืคื), ื”ึทืฉืื‘ึผึธื—ึธื” ืชืžื™ื›ื” ื‘ื—ืฉื‘ื•ืŸ ืฉื™ืจื•ืช ืžื ื•ื”ืœ ื‘ืงื‘ื•ืฆื” (gMSA) ืขื“ ื’ืจืกืช ื‘ื˜ื, ืชืžื™ื›ื” ื”ืจื›ื‘/ืžืฆืจืฃ ืขื‘ื•ืจ ืืžืฆืขื™ ืื—ืกื•ืŸ ืฉืœ vSphere.
  • ืžืžื•ื—ื–ืจ ืžื ื’ื ื•ืŸ ื“ื—ื™ืกืช ื ืชื•ื ื™ื ื‘ืชื’ื•ื‘ื•ืช API. ื‘ืขื‘ืจ, ื ืขืฉื” ืฉื™ืžื•ืฉ ื‘ืžืกื ืŸ HTTP ืœืžื˜ืจื•ืช ืืœื•, ืืฉืจ ื”ื˜ื™ืœ ืžืกืคืจ ื”ื’ื‘ืœื•ืช ืฉืžื ืขื• ืืช ื”ืคืขืœืชื• ื›ื‘ืจื™ืจืช ืžื—ื“ืœ. "ื“ื—ื™ืกื” ืฉืงื•ืคื” ืฉืœ ื‘ืงืฉื”" ืขื•ื‘ื“ืช ื›ืขืช: ืœืงื•ื—ื•ืช ืฉื•ืœื—ื™ื Accept-Encoding: gzip ื‘ื›ื•ืชืจืช, ื”ื ืžืงื‘ืœื™ื ืชื’ื•ื‘ื” ื“ื—ื•ืกื” ืฉืœ GZIP ืื ื”ื’ื•ื“ืœ ืฉืœื” ืขื•ืœื” ืขืœ 128 KB. ืœืงื•ื—ื•ืช Go ืชื•ืžื›ื™ื ื‘ืื•ืคืŸ ืื•ื˜ื•ืžื˜ื™ ื‘ื“ื—ื™ืกื” (ืฉืœื™ื—ืช ื”ื›ื•ืชืจืช ื”ื ื“ืจืฉืช), ื›ืš ืฉื”ื ื™ื‘ื—ื™ื ื• ืžื™ื“ ื‘ื™ืจื™ื“ื” ื‘ืชืขื‘ื•ืจื”. (ื™ื™ืชื›ืŸ ืฉื™ื”ื™ื” ืฆื•ืจืš ื‘ืฉื™ื ื•ื™ื™ื ืงืœื™ื ืขื‘ื•ืจ ืฉืคื•ืช ืื—ืจื•ืช.)
  • ื”ืคืš ืืคืฉืจื™ ืฉื™ื ื•ื™ ืงื ื” ื”ืžื™ื“ื” ืฉืœ HPA ืž/ืœืืคืก ืชืจืžื™ืœื™ื ื‘ื”ืชื‘ืกืก ืขืœ ืžื“ื“ื™ื ื—ื™ืฆื•ื ื™ื™ื. ืื ืืชื” ืžืฉื ื” ืงื ื” ืžื™ื“ื” ืขืœ ืกืžืš ืื•ื‘ื™ื™ืงื˜ื™ื/ืžื“ื“ื™ื ื—ื™ืฆื•ื ื™ื™ื, ืื– ื›ืืฉืจ ืขื•ืžืกื™ ื”ืขื‘ื•ื“ื” ืื™ื ื ืคืขื™ืœื™ื ืชื•ื›ืœ ืœืฉื ื•ืช ืืช ืงื ื” ื”ืžื™ื“ื” ืื•ื˜ื•ืžื˜ื™ืช ืœ-0 ื”ืขืชืงื™ื ื›ื“ื™ ืœื—ืกื•ืš ื‘ืžืฉืื‘ื™ื. ืชื›ื•ื ื” ื–ื• ืืžื•ืจื” ืœื”ื™ื•ืช ืฉื™ืžื•ืฉื™ืช ื‘ืžื™ื•ื—ื“ ื‘ืžืงืจื™ื ืฉื‘ื”ื ืขื•ื‘ื“ื™ื ืžื‘ืงืฉื™ื ืžืฉืื‘ื™ GPU, ื•ืžืกืคืจ ื”ืกื•ื’ื™ื ื”ืฉื•ื ื™ื ืฉืœ ืขื•ื‘ื“ื™ื ืกืจืง ืขื•ืœื” ืขืœ ืžืกืคืจ ื”-GPUs ื”ื–ืžื™ื ื™ื.
  • ืœืงื•ื— ื—ื“ืฉ - k8s.io/client-go/metadata.Client - ืœื’ื™ืฉื” "ืžื•ื›ืœืœืช" ืœืื•ื‘ื™ื™ืงื˜ื™ื. ื–ื” ื ื•ืขื“ ืœืื—ื–ืจ ื‘ืงืœื•ืช ืžื˜ื ื ืชื•ื ื™ื (ื›ืœื•ืžืจ ืกืขื™ืฃ ืžืฉื ื” metadata) ืžืžืฉืื‘ื™ ืืฉื›ื•ืœ ื•ืœื‘ืฆืข ืื™ืชื ืคืขื•ืœื•ืช ืื™ืกื•ืฃ ืืฉืคื” ื•ืžื›ืกื•ืช.
  • ื‘ื ื” Kubernetes ืขื›ืฉื™ื• ืืชื” ื™ื›ื•ืœ ืœืœื ืกืคืงื™ ืขื ืŸ ืžื“ื•ืจ ืงื•ื“ื ("ืžื•ื‘ื ื”" ื‘ืชื•ืš ื”ืขืฅ) (ื’ืจืกืช ืืœืคื).
  • ืœื›ืœื™ ื”ืฉื™ืจื•ืช kubeadm ื”ื•ืกื™ืฃ ื™ื›ื•ืœืช ื ื™ืกื™ื•ื ื™ืช (ื’ืจืกืช ืืœืคื) ืœื”ื—ื™ืœ ืชื™ืงื•ื ื™ื ืžื•ืชืืžื™ื ืื™ืฉื™ืช ื‘ืžื”ืœืš ื”ืคืขื•ืœื•ืช init, join ะธ upgrade. ืœืžื™ื“ืข ื ื•ืกืฃ ืขืœ ืื•ืคืŸ ื”ืฉื™ืžื•ืฉ ื‘ื“ื’ืœ --experimental-kustomize, ืจืื” ื‘ ืงืค.
  • ื ืงื•ื“ืช ืงืฆื” ื—ื“ืฉื” ืขื‘ื•ืจ apiserver - readyz, - ืžืืคืฉืจ ืœื™ื™ืฆื ืžื™ื“ืข ืขืœ ืžื•ื›ื ื•ืชื•. ื’ื ืœืฉืจืช ื”-API ื™ืฉ ื›ืขืช ื“ื’ืœ --maximum-startup-sequence-duration, ื”ืžืืคืฉืจ ืœืš ืœื•ื•ืกืช ืืช ื”ื”ืคืขืœื” ืžื—ื“ืฉ ืฉืœื•.
  • ืฉื ื™ ืชื›ื•ื ื•ืช ืขื‘ื•ืจ Azure ื”ื•ื›ืจื– ื™ืฆื™ื‘: ืชืžื™ื›ื” ืื–ื•ืจื™ ื–ืžื™ื ื•ืช (ืื–ื•ืจื™ ื–ืžื™ื ื•ืช) ื• ืงื‘ื•ืฆืช ืžืฉืื‘ื™ื ื—ื•ืฆื” (RG). ื‘ื ื•ืกืฃ, Azure ื”ื•ืกื™ืคื”:
  • ืœ-AWS ื™ืฉ ืขื›ืฉื™ื• ืœืชืžื•ืš ืขื‘ื•ืจ EBS ื‘-Windows ื• ืื•ืคื˜ื™ืžื™ื–ืฆื™ื” ืงืจื™ืื•ืช EC2 API DescribeInstances.
  • Kubeadm ื”ื•ื ื›ืขืช ืขืฆืžืื™ ื ื•ื“ื“ ืชืฆื•ืจืช CoreDNS ื‘ืขืช ืฉื“ืจื•ื’ ื’ืจืกืช CoreDNS.
  • ื‘ื™ื ืืจื™ื™ื ื•ื›ื• ' ื‘ืชืžื•ื ืช Docker ื”ืžืชืื™ืžื” ืขืฉื• world-executable, ื”ืžืืคืฉืจ ืœืš ืœื”ืจื™ืฅ ืชืžื•ื ื” ื–ื• ืœืœื ืฆื•ืจืš ื‘ื–ื›ื•ื™ื•ืช ืฉื•ืจืฉ. ื›ืžื• ื›ืŸ, ืชืžื•ื ืช ื”ื’ื™ืจื” ื•ื›ื•' ื—ื“ืœ ืชืžื™ื›ื” ื‘ื’ืจืกืช etcd2.
  • ะ’ Cluster Autoscaler 1.16.0 ืขื‘ืจ ืœื”ืฉืชืžืฉ ื‘-Distroless ื›ืชืžื•ื ืช ื”ื‘ืกื™ืก, ื‘ื™ืฆื•ืขื™ื ืžืฉื•ืคืจื™ื, ื ื•ืกืคื• ืกืคืงื™ ืขื ืŸ ื—ื“ืฉื™ื (DigitalOcean, Magnum, Packet).
  • ืขื“ื›ื•ื ื™ื ื‘ืชื•ื›ื ื” ืžืฉื•ืžืฉืช/ืชืœื•ื™ื”: Go 1.12.9, etcd 3.3.15, CoreDNS 1.6.2.

ื .ื‘.

ืงืจื ื’ื ื‘ื‘ืœื•ื’ ืฉืœื ื•:

ืžืงื•ืจ: www.habr.com

ื”ื•ืกืคืช ืชื’ื•ื‘ื”