ืื ืืืืขืจืืขืฆืื ื ืคืื ืืขื ืึทืจืืืงื ืืื ืืขืืืขื ืฆืืืขืืจืืื ืืืืฃ ืื ืขืจืขืื ืคืื ืื ืึธื ืืืื ืคืื ืื ืงืืจืก
ืืื ืฆื ืฉืคึผืึธืจื ืืืืฃ ืืืึธืืงื ืงืึธืก ืืืขื ืืจืืขืื ืืื Kubernetes? ืขืก ืืื ืงืืื ืืืื ืจืขืื ืืืืืื ื, ืึธืืขืจ ืืขืจ ืึทืจืืืงื ืืืฉืจืืืื ืขืืืขืืข ืืืฉืืจืื ืืืึธืก ืงืขื ืขื ืืขืืคึฟื ืืืจ ืคืืจื ืืืื ืจืขืกืืจืกื ืืขืจ ืืคืขืงืืืืืื ืืื ืจืขืืืฆืืจื ืืืื ืืืึธืืงื ืงืึทืืคึผืืืืื ื ืงืึธืก.
ืืื ืืขืฉืจืืื ืืขื ืึทืจืืืงื ืืื Kubernetes ืคึฟืึทืจ AWS ืืื ืืื ืขื, ืึธืืขืจ ืขืก ืืืขื ืึทืคึผืืืื (ืึผืืขื) ืคึผืื ืงื ืื ืืขืืืข ืืืขื ืฆื ืื ืืขืจืข ืืืึธืืงื ืคึผืจืึทืืืืืืขืจื. ืืื ืืืขืจื ืขืืขื ืืืื ืงื ืืื (s) ืฉืืื ืืึธืื ืึทืืืึธืกืงืึทืืื ื ืงืึทื ืคืืืืขืจื (
ืืขืจ ืึทืจืืืงื ืืืขื ืืขืงื:
- ืจืืื ืืงืื ื ืึทื ืืืื ืจืขืกืืจืกื (
ืงืืืข-ืกืืจืืืฉ ) - ืจืขืืืฆืืจื ืกืงืืืืื ื ืืขืฉืึทืก ื ืื-ืืจืืขืื ืฉืขื (
kube-downscaler ) - ื ืืฆื ืืึธืจืืืึธื ืืึทื ืึทืืืึธืกืงืึทืืื ื (HPA),
- ืจืขืืืงืฆืืข ืคืื โโืืืขืจืืง ืืืื ืจืขืืขืจืืืึทืฆืืข (
kube-resource-report , ืืืคึผืึท) - ื ืืฆื ืกืคึผืึธื ืื ืกืืึทื ืกืื
ืจืืื ืืงืื ื ืึทื ืืืื ืจืขืกืืจืกื
ืืจืืขืื ืืื ืึท ืฉื ืขื-ืคึผืืืกื ืกืืืืืืข ืืื ืืจืืืก. ืืืจ ืืืืื ืืขืง ืึธืจืืึทื ืืืึทืฆืืขืก
(ืืขื ื ืื ื ืืขืงืืก:
ืืฉืืื:
(ืฆืืืืจื) ืงืึธืจื ืงืืืื:
ืืืืึธืก: ืืืื AWS ืืฉืืื ืืื ืึท ืคืื ืงืฆืืข ืคืื โโโโืื ื ืืืขืจ ืคืื ืืืืขืจื ืืืจ ืืึธืื.
ืคืึทืงื: ืืืื AWS ืืขืืฉืื ืืื ืึท ืคืื ืงืฆืืข ืคืื โโโโืื ื ืืืขืจ ืคืื ืขื ืืืฉืึทื ืืจื ืืืจ ืืึธืื.
ืืืืืื ืงืืจื ืึธืกืึธืื (ืืื ืขื ืืคืขืจ):
ืคืึทืงืืืฉ ืคืึทืงื: ืืืื AWS ืืขืืฉืื ืืื ืึท ืคืื ืงืฆืืข ืคืื โโโโืื ื ืืืขืจ ืคืื ืืื ืื ืืืึธืก ืืืจ ืืึธื ืคืืจืืขืกื ืฆื ืืืกืืืืึทื / ืืืกืืขืงื.)
- ืงื ืืื-ืืจืืื ืึผืืืื ืงืขื ืขื ืืขืคืื ืืจื ืื ืืึทืงืกืืืื ืฆืืื-ืฆื-ืืขืื (ืืื) ืคึฟืึทืจ ืคึผืจ / ืคึผืจืืืืจื ืืืคึผืืืืืึทื ืฅ.
- ืืืื ืจืขืกืืจืกื ืงืขื ืขื ืืืื ืึทื ืึทืืืืืื ืืื ืกืืจืึธืืฉ / ืืื, ืืืฉื ืฆื ืืืืืึธืืึทืืืฉ ืืึทืืืึทืืืงื ืื ืกืคึผืืืง / ืคึผืจืึธืืืึทืืืืคึผ ื ืึธื 7 ืืขื.
ืึทืืืขืืืื ืข ืึผืืืื ืืขื ืขื ืืืคืืื ื ืืื ืื YAML ืืขืงืข. ืืืึทื ืืจื ืืื ืืืจืืืขืืื ืืขื ืืืจื ืืขื ืคึผืึทืจืึทืืขืืขืจ --rules-file
ืืื ืงืืืข-ืกืืจืืืฉ. ืืึธ ืืื ืึท ืืืืฉืคึผืื ืืขืจืฉื ืฆื ืืึทืืืึทืืืงื ืึทืืข ื ืึทืืขืกืคึผืึทืกืขืก ืืื -pr-
ืืื ื ืึธืืขื ื ืึธื ืฆืืืื ืืขื:
- id: cleanup-resources-from-pull-requests
resources:
- namespaces
jmespath: "contains(metadata.name, '-pr-')"
ttl: 2d
ืื ืคืืืืขื ืืข ืืืืฉืคึผืื ืจืขืืืึทืืืืฅ ืื ื ืืฆื ืคืื ืื ืึทืคึผืืึทืงืืืฉืึทื ืคืืจืืข ืืืืฃ ืื ืืืคึผืืืืืึทื ื ืืื ืกืืขืขืคืืืกืขื ืคึผืึธืืก ืคึฟืึทืจ ืึทืืข ื ืืึทืข ืืืคึผืืืืืึทื ืฅ / ืกืืืืืคืืืกืขืฅ ืืื 2020, ืึธืืขืจ ืืื ืืขืจ ืืขืืืืงืขืจ ืฆืืื ืึทืืึทืื ืื ืืืจืืคืืจืื ื ืคืื ืืขืกืฅ ืึธื ืืขื ืคืืจืืข ืคึฟืึทืจ ืึท ืืืึธื:
- id: require-application-label
# ัะดะฐะปะธัั deployments ะธ statefulsets ะฑะตะท ะผะตัะบะธ "application"
resources:
- deployments
- statefulsets
# ัะผ. http://jmespath.org/specification.html
jmespath: "!(spec.template.metadata.labels.application) && metadata.creationTimestamp > '2020-01-01'"
ttl: 7d
ืืืืคื ืึท ืฆืืื-ืืืืืืขื ืืขืืึธ ืคึฟืึทืจ 30 ืืื ืื ืืืืฃ ืึท ืงื ืืื ืคืืืกื ืืืง ืงืืืข-ืกืืจืืืฉ:
kubectl run nginx-demo --image=nginx
kubectl annotate deploy nginx-demo janitor/ttl=30m
ืื ืื ืืขืจ ืืงืืจ ืคืื ืื ืงืจืืกืื ื ืงืึธืก ืืื ืคึผืขืจืกืืกืืขื ื ืืืึทืืืืื (AWS EBS). ืืืืืืื ื ืึท Kubernetes StatefulSet ืืื ื ืืฉื ืืืกืืขืงื ืื ืคึผืขืจืกืืกืืขื ื ืืืึทืืืืื (PVC - PersistentVolumeClaim). ืึทื ืืืื EBS ืืืึทืืืืื ืงืขื ืขื ืืืืื ืจืขืืืืืึทื ืืื ืงืึธืก ืคืื ืืื ืืขืจืืขืจ ืคืื ืืึธืืืึทืจืก ืคึผืขืจ ืืืืฉ. Kubernetes Janitor ืืื ืึท ืฉืืจืื ืฆื ืจืืื ืึทืจืืืฃ ืึทื ืืืื ืคึผืืืงืก. ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืขื ืืขืจืฉื ืืืขื ืืึทืืืึทืืืงื ืึทืืข ืคึผืืืงืก ืืืึธืก ืืขื ืขื ื ืืฉื ืืึธืื ืืขื ืืืจื ืึท ืืึธืืืืข ืืื ืืืึธืก ืืขื ืขื ื ืืฉื ืจืขืคืขืจืจืขื ืืืจื ืึท StatefulSet ืึธืืขืจ CronJob:
# ัะดะฐะปะธัั ะฒัะต PVC, ะบะพัะพััะต ะฝะต ัะผะพะฝัะธัะพะฒะฐะฝั ะธ ะฝะฐ ะบะพัะพััะต ะฝะต ัััะปะฐัััั StatefulSets
- id: remove-unused-pvcs
resources:
- persistentvolumeclaims
jmespath: "_context.pvc_is_not_mounted && _context.pvc_is_not_referenced"
ttl: 24h
Kubernetes Janitor ืงืขื ืขื ืืขืืคึฟื ืืืจ ืืึทืืื ืืืื ืงื ืืื ืจืืื ืืื ืคืึทืจืืืึทืื ืืืึธืืงื ืงืึทืืคึผืืืืื ื ืงืึธืก ืคืื ืกืืึธืืื ืคึผืืืื ื ืืื. ืคึฟืึทืจ ืืืคึผืืืืืึทื ื ืืื ืงืึทื ืคืืืืขืจืืืฉืึทื ืื ืกืืจืึทืงืฉืึทื ื, ื ืึธืืืืื
ืจืขืืืฆืืจื ืกืงืืืืื ื ืืขืฉืึทืก ื ืื-ืืจืืขืื ืฉืขื
ืืขืกื ืืื ืกืืึทืืื ื ืกืืกืืขืืขื ืืขื ืขื ืืืคึผืืงืื ืคืืจืืื ืื ืฆื ืึทืจืืขืื ืืืืื ืืขืฉืึทืก ืืขืฉืขืคื ืฉืขื. ืขืืืขืืข ืคึผืจืึธืืืงืฆืืข ืึทืคึผืืึทืงืืืฉืึทื ื, ืึทืืึท ืืื ืฆืืจืืง ืึธืคืืก / ืึทืืืื ืืืฉืืจืื, ืืืื ืืึทืจืคื ืืืืื ืืืืืืขื ืึทืืืืืืึทืืืืึทืื ืืื ืงืขื ืืืื ืคืึทืจืงืจืืคึผืื ืืืขืจื ืึทืืืืง.
image: hjacobs/kube-downscaler:20.4.3
args:
- --interval=30
# ะฝะต ะพัะบะปััะฐัั ะบะพะผะฟะพะฝะตะฝัั ะธะฝััะฐััััะบัััั
- --exclude-namespaces=kube-system,infra
# ะฝะต ะพัะบะปััะฐัั kube-downscaler, ะฐ ัะฐะบะถะต ะพััะฐะฒะธัั Postgres Operator, ััะพะฑั ะธัะบะปััะตะฝะฝัะผะธ ะะ ะผะพะถะฝะพ ะฑัะปะพ ัะฟัะฐะฒะปััั
- --exclude-deployments=kube-downscaler,postgres-operator
- --default-uptime=Mon-Fri 08:00-20:00 Europe/Berlin
- --include-resources=deployments,statefulsets,stacks,cronjobs
- --deployment-time-annotation=deployment-time
ืืึธ ืืื ืึท ืืจืึทืคืืง ืคึฟืึทืจ ืกืงืืืืื ื ืงื ืืื ืึทืจืืขืืขืจ ื ืึธืืื ืืืืฃ ืืืืงืขื ืื:
ืกืงืึทืืื ื ืึทืจืึธืคึผ ืคืื ~ 13 ืฆื 4 ืึทืจืืขืืขืจ ื ืึธืืื ืึทืืืึทืืข ืืืื ืึท ืืืืขืจืงื ืืืืืง ืืื ืืืื AWS ืจืขืืขื ืื ื.
ืึธืืขืจ ืืืึธืก ืืืื ืืื ืืึทืจืคึฟื ืฆื ืึทืจืืขืื ืืขืฉืึทืก ืงื ืืื "ืืึทืื ืืืื"? ืืืืขืจ ืืืคึผืืืืืึทื ืฅ ืงืขื ืขื ืืืื ืคึผืขืจืืึทื ืึทื ืืื ืืงืกืงืืืืื ืคืื ืกืงืืืืื ื ืืืจื ืึทืืื ื ืื ืืึทืื ืกืงืึทืืขืจ / ืืืกืฉืืืกื: ืืืช ืึทื ืึธืืึทืฆืืข. ืืืคึผืืืืืึทื ืฅ ืงืขื ืขื ืืืื ืืขืืคึผืขืจืขืจืึทืื ืืงืกืงืืืืื ืืื ืื ืืึทืื ืกืงืึทืืขืจ / ืืืกืฉืืืกื-ืืื ืึทื ืึธืืึทืฆืืข ืืื ืึทื ืึทืืกืึธืืื ืฆืืื ืฉืืขืืคึผื ืืื ืื ืคึฟืึธืจืืึทื YYYY-MM-DD HH:MM (UTC). ืืืื ื ืืืืืง, ืื ืืื ืฆืข ืงื ืืื ืงืขื ืขื ืืืื ืกืงืืืื ืฆืืจืืง ืืืจื ืืืคึผืืืืื ื ืึท ืคึผืึธื ืืื ืื ืึทื ืึธืืึทืฆืืข downscaler/force-uptime
, ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืืจื ืืึธื ืืฉืื ื nginx ืืืืืืง:
kubectl run scale-up --image=nginx
kubectl annotate deploy scale-up janitor/ttl=1h # ัะดะฐะปะธัั ัะฐะทะฒะตัััะฒะฐะฝะธะต ัะตัะตะท ัะฐั
kubectl annotate pod $(kubectl get pod -l run=scale-up -o jsonpath="{.items[0].metadata.name}") downscaler/force-uptime=true
ืืขื
ื ืืฆื ืืึธืจืืืึธื ืืึทื ืึทืืืึธืกืงืึทืืื ื
ืคืืืข ืึทืคึผืืึทืงืืืฉืึทื ื / ืืึทืืื ืื ืืก ืืึทื ืืืขื ืืื ืึท ืืื ืึทืืืฉ ืืึธืืืื ื ืืืกืืขืจ: ืืื ืืืืขืจ ืืึทืืืฉืืื ืืขื ืขื ืืืืืืง, ืืื ืืื ืืื ืึทืจืืขืื ืืื ืคืื ืงืึทืคึผืึทืฆืืืขื. ืึทืคึผืขืจืืืืื ื ืึท ืฉืืขื ืืืง ืคืืื ืคืื ืคึผืึธืืก ืฆื ืงืึธืคึผืข ืืื ืืึทืงืกืืืื ืฉืคึผืืฅ ืืึทืกืข ืืื ื ืืฉื ืฉืคึผืึธืจืขืืืืืง. Kubernetes ืฉืืืฆื ืืึธืจืืืึธื ืืึทื ืึทืืืึธ-ืกืงืืืืื ื ืึทืจืืืขืจ ืึท ืืืื
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: my-app
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: my-app
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
averageUtilization: 100
type: Utilization
Zalando ืืื ืืืฉืืคื ืึท ืงืึธืืคึผืึธื ืขื ื ืฆื ืืืืื ืคืึทืจืืื ืื ืื ืื ืืขืืจืืงืก ืคึฟืึทืจ ืกืงืืืืื ื:
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
annotations:
# metric-config.<metricType>.<metricName>.<collectorName>/<configKey>
metric-config.pods.requests-per-second.json-path/json-key: "$.http_server.rps"
metric-config.pods.requests-per-second.json-path/path: /metrics
metric-config.pods.requests-per-second.json-path/port: "9090"
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 1
maxReplicas: 10
metrics:
- type: Pods
pods:
metric:
name: requests-per-second
target:
averageValue: 1k
type: AverageValue
ืงืึทื ืคืืืืขืจืื ื ืืึธืจืืืึธื ืืึทื ืึทืืืึธืกืงืึทืืื ื ืืื HPA ืืึธื ืืืื ืืืื ืขืจ ืคืื ืื ืคืขืืืงืืึทื ืึทืงืฉืึทื ื ืฆื ืคึฟืึทืจืืขืกืขืจื ืขืคืขืงืืืืืงืืึทื ืคึฟืึทืจ ืกืืืืืืขืกืก ืืึทืืื ืื ืืก. Spotify ืืื ืึท ืคึผืจืขืืขื ืืืจืื ื ืืื ืืืืขืจ ืืขืจืคืึทืจืื ื ืืื ืจืขืงืึทืืึทื ืืืืฉืึทื ื ืคึฟืึทืจ HPA:
ืจืขืืืฆืืจื ืืืื ืึธืืืืขืจืืืงืื ื
ืงืืืขืจื ืขืืขืก ืืืขืจืงืืึธืืื ืืึทืฉืืืืขื ืืืืขืจ ืงืคึผื / ืืืงืึธืจื ืืึทืจืฃ ืืืจื "ืจืืกืึธืจืก ืจืืงืืืขืก." ืงืคึผื ืจืขืกืืจืกื ืืขื ืขื ืืขืืืกืื ืืื ืืืืจืืืึทื ืงืึธืจืขืก ืึธืืขืจ ืืขืจ ืงืึทืืึทื ืื ืืื "ืืืืืงืึธืจืขืก", ืืืฉื 500 ื ืืืคึผืืืื 50% ืืืงืคึผื. ืืึผืจืื ืจืขืกืืจืกื ืืขื ืขื ืืขืืืกืื ืืื ืืืืขืก, ืืื ืคึผืจืึธืกื ืกืึทืคืืงืกืื ืงืขื ืขื ืืืื ืืขืืืืื ื, ืึทืืึท ืืื 500ืื, ืืืึธืก ืืืื 500 ืืขืืืืืื. ืืืื ืจืืงืืืขืก "ืฉืืึธืก" ืงืึทืคึผืึทืฆืืืขื ืืืืฃ ืึทืจืืขื ื ืึธืืื, ืืืึทืืฉ ืึท ืคึผืึธื ืืื ืึท 1000 ื ืงืคึผื ืืขืื ืืืืฃ ืึท ื ืึธืืข ืืื 4 ืืืงืคึผืืก ืืืขื ืืึธืื ืืืืื 3 ืืืงืคึผืืก ืื ืืืฆื ืฆื ืื ืืขืจืข ืคึผืึธืืก.
ืกืืึทืง (ืืืืขืคืืืง ืจืขืืขืจืื) ืืื ืืขืจ ืืืืืง ืฆืืืืฉื ืืขืืขืื ืจืขืกืืจืกื ืืื ืคืึทืงืืืฉ ืืึทื ืืฅ. ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืึท ืคึผืึธื ืืืึธืก ืจืืงืืืขืก 2 GiB ืคืื ืืึผืจืื ืึธืืขืจ ืืืืื ื ืืฆื 200 MiB ืืื ~ 1,8 GiB ืคืื "ืืืืขืคืืืง" ืืึผืจืื. ืืืืขืคืืืง ืงืึธืก ืืขืื. ืืขื ืงืขื ืืขืขืจืขื ืึธืคึผืฉืึทืฆื ืึทื 1 GiB ืคืื ืืืขืจืืง ืืึผืจืื ืงืึธืก ~ $ 10 ืคึผืขืจ ืืืืฉ.
ืจืืืืกืื ื ืื ืงืคึผื ืืขืื ืคืื 3000 ื ืฆื ~ 400 ื ืคืจืื ืจืขืกืืจืกื ืคึฟืึทืจ ืื ืืขืจืข ืืืขืจืงืืึธืืื ืืื ืึทืืึทืื ืื ืงื ืืื ืฆื ืืืื ืงืืขื ืขืจืขืจ.
"ืืืจืืฉื ืืืืขื ืงืคึผื ืืึทื ืืฅ ืคืื EC2 ืื ืกืืึทื ืกืื ืึธืคื ืืึทืืื ืืื ืื ืืืื-ืฆืืคึฟืขืจ ืคึผืจืึธืฆืขื ื ืงืืื,"
ืึธืืขืจ ืืึธื ืืืจ ืืึทืงืข ืืืขืื ืึทื ืืขื ืืฉื ืืฉืึทื ืืื ื ืืืึทืืืขืก ืืื YAML ืืขืงืขืก? ื ืืื, ืืืฉืื ืขื ืงืขื ืขื ืขืก ืคืื ืืขืกืขืจ! Kubernetes
ืืื ืืขืฉืจืืื ืึท ืงืืืื
ื ืืฆื EC2 ืกืคึผืึธื ืื ืกืืึทื ืกืื
ืืขืฆืืข ืึธืืขืจ ื ืื ืงืืขื ืกืืขืจ, AWS EC2 ืงืึธืก ืงืขื ืขื ืืืื ืจืืืืกื ืืืจื ื ืืฆื ืกืคึผืึธื ืื ืกืืึทื ืกืื ืืื Kubernetes ืึทืจืืขื ื ืึธืืื
ืืื ืฆื ืืืืคื Kubernetes ืืืืฃ EC2 Spot? ืขืก ืืขื ืขื ืขืืืขืืข ืึธืคึผืฆืืขืก: ื ืืฆื ืึท ืืจืื ืคึผืึทืจืืื ืืื ืกื ืืื SpotInst (ืืืฆื ืืขืจืืคื "ืกืคึผืึธื", ืืึธื ื ืื ืคืจืขืื ืืืจ ืืืึธืก), ืึธืืขืจ ืคืฉืื ืืืืื ืึท Spot AutoScalingGroup (ASG) ืฆื ืืืื ืงื ืืื. ืคึฟืึทืจ ืืืึทืฉืคึผืื, ืืึธ ืก ืึท ืงืืึธืืืคืึธืจืืึทืืืึธื ืกื ืืคึผืึทื ืคึฟืึทืจ ืึท "ืงืึทืคึผืึทืฆืืืขื-ืึธืคึผืืืืืืขื" ืกืคึผืึธื ASG ืืื ืงืืืคื ืืืึทืฉืคึผืื ืืืืคึผืก:
MySpotAutoScalingGroup:
Properties:
HealthCheckGracePeriod: 300
HealthCheckType: EC2
MixedInstancesPolicy:
InstancesDistribution:
OnDemandPercentageAboveBaseCapacity: 0
SpotAllocationStrategy: capacity-optimized
LaunchTemplate:
LaunchTemplateSpecification:
LaunchTemplateId: !Ref LaunchTemplate
Version: !GetAtt LaunchTemplate.LatestVersionNumber
Overrides:
- InstanceType: "m4.2xlarge"
- InstanceType: "m4.4xlarge"
- InstanceType: "m5.2xlarge"
- InstanceType: "m5.4xlarge"
- InstanceType: "r4.2xlarge"
- InstanceType: "r4.4xlarge"
LaunchTemplate:
LaunchTemplateId: !Ref LaunchTemplate
Version: !GetAtt LaunchTemplate.LatestVersionNumber
MinSize: 0
MaxSize: 100
Tags:
- Key: k8s.io/cluster-autoscaler/node-template/label/aws.amazon.com/spot
PropagateAtLaunch: true
Value: "true"
ืขืืืขืืข ืืขืจืืช ืืืขืื ื ืืฆื Spot ืืื Kubernetes:
- ืืืจ ืืึทืจืคึฟื ืฆื ืฉืขืคึผื ืกืคึผืึธื ืืขืจืืึทื ืืืฉืึทื ื, ืืืฉื ืืืจื ืฆืื ืืืคืืืกื ืื ื ืึธืืข ืืืขื ืืขืจ ืืืึทืฉืคึผืื ืืื ืกืืึทืคึผื
- ืืึทืืึทื ืืึธ ื ืืฆื
ืืึธืคึผื ืืึทืึทืืืขืจ ืงื ืืื ืึทืืืึธืกืงืึทืืื ื ืืื ื ืึธืืข ืืขืงื ืคึผืจืืืึธืจืึทืืื - ืึธืจื ื ืึธืืื
ืงืขื ืขื ืืืื ืืขืฆืืืื ืืขื ืึธื ื ืขืืขื "ืจืขืืืกืืจืึทืืืึธื ืก" ืคืื ืืืขืจืงืืึธืืื ืฆื ืืืืคื ืืื ืกืคึผืึธื
ืงืืฆืขืจ
ืืื ืืึธืคึฟื ืืืจ ืืขืคึฟืื ืขื ืขืืืขืืข ืคืื โโืื ืืขืจืืื ืื ืืืฉืืจืื ื ืืฆืืง ืืื ืจืืืืกืื ื ืืืื ืืืึธืืงื ืจืขืืขื ืื ื. ืืืจ ืงืขื ื ืืืื ืืขืคึฟืื ืขื ืจืืึฟ ืคืื ืื ืืื ืืึทืื ืคืื ืืขื ืึทืจืืืงื ืืื
ืืืึธืก ืืขื ืขื ืืืื ืืขืกืืขืจ ืคึผืจืึทืงืืืกืื ืคึฟืึทืจ ืฉืคึผืึธืจื ืืืึธืืงื ืงืึธืก ืืืืฃ Kubernetes? ืืืืข ืืึธืื ืืืจ ืืืืกื ืืืึท
ืืงืืจ: www.habr.com