🥇Yuav nkag mus rau Kubernetes Pod cov peev txheej

The Reward by Tohad

Thaum pib nrog Kubernetes, nws yog ib qho uas tsis nco qab txog kev teeb tsa cov khoom ntim khoom. Lub sijhawm no, nws txaus los xyuas kom meej tias Docker duab ua haujlwm thiab tuaj yeem xa mus rau Kubernetes pawg.

Tab sis tom qab daim ntawv thov yuav tsum tau muab tso rau hauv ib pawg ntau lawm nrog rau lwm yam kev siv. Txhawm rau ua qhov no, koj yuav tsum faib cov peev txheej rau lub thawv thiab xyuas kom meej tias muaj txaus ntawm lawv kom tau txais daim ntawv thov nce thiab khiav, thiab lwm cov ntawv thov khiav yuav tsis muaj teeb meem.

pab neeg Kubernetes aaS los ntawm Mail.ru txhais ib tsab xov xwm hais txog cov peev txheej ntim khoom (CPU & MEM), kev thov thiab cov peev txheej txwv. Koj yuav kawm tau cov txiaj ntsig ntawm cov chaw no thiab yuav ua li cas yog tias koj tsis teeb lawv.

xam cov peev txheej

Peb muaj ob hom kev pab nrog cov nram qab no units:

Central processing unit (CPU) - cores;
Memory (MEM) - bytes.

Cov peev txheej tau teev tseg rau txhua lub thawv. Hauv cov ntaub ntawv Pod YAML hauv qab no, koj yuav pom cov khoom siv uas muaj cov ntawv thov thiab txwv cov peev txheej:

Thov Pod Resources = sum of requested resources of all containers;
Pod Resource Limit = Sum ntawm tag nrho Pod Resource Limits.

apiVersion: v1
kind: Pod
metadata:
  name: backend-pod-name
  labels:
    application: backend
spec:
  containers:
    — name: main-container
      image: my-backend
      tag: v1
      ports:
      — containerPort: 8080
      resources:
        requests:
          cpu: 0.2 # REQUESTED CPU: 200m cores
          memory: "1Gi" # REQUESTED MEM: 1Gi
        limits:
          cpu: 1 # MAX CPU USAGE: 1 core
          memory: "1Gi" # MAX MEM USAGE:  1Gi
    — name: other-container
      image: other-app
      tag: v1
      ports:
      — containerPort: 8000
      resources:
        requests:
          cpu: "200m" # REQUESTED CPU: 200m cores
          memory: "0.5Gi" # REQUESTED MEM: 0.5Gi
        limits:
          cpu: 1 # MAX CPU USAGE: 1 core
          memory: "1Gi" # MAX MEM USAGE:  1Gi

Piv txwv ntawm Kev Thov thiab Txwv Kev Pabcuam

teb resources.requested los ntawm cov lus qhia Pod yog ib qho ntawm cov ntsiab lus uas siv los nrhiav qhov xav tau. Koj tuaj yeem npaj Pod xa mus rau nws. Yuav ua li cas koj thiaj nrhiav tau ib tug haum node?

Kubernetes muaj ntau yam khoom, suav nrog tus tswv ntawm lossis tus tswv ntawm (Kubernetes Control Plane). Tus tswv node muaj ntau yam txheej txheem: kube-apiserver, kube-controller-manager thiab kube-scheduler.

Cov txheej txheem kube-scheduler yog lub luag haujlwm los tshuaj xyuas cov pods tsim tshiab thiab nrhiav cov neeg ua haujlwm tau zoo uas haum rau txhua qhov kev thov pods, suav nrog cov peev txheej thov. Cov npe ntawm cov nodes pom los ntawm kube-scheduler yog qeb duas. Lub pod yog teem rau ntawm lub node nrog cov qhab nia siab tshaj plaws.

Lub Pod liab yuav muab tso rau qhov twg?

Hauv daim duab koj tuaj yeem pom tias kube-scheduler yuav tsum teem caij lub Pod liab liab tshiab. Cov pab pawg Kubernetes muaj ob lub nodes: A thiab B. Raws li koj tuaj yeem pom, kube-scheduler tsis tuaj yeem teem caij lub Pod ntawm node A - cov peev txheej muaj (tsis tau thov) tsis phim cov kev thov ntawm cov ntshav Pod. Yog li, 1 GB ntawm lub cim xeeb thov los ntawm lub paj liab Pod yuav tsis haum ntawm node A, txij li lub cim xeeb muaj yog 0,5 GB. Tab sis node B muaj peev xwm txaus. Raws li qhov tshwm sim, kube-scheduler txiav txim siab tias qhov chaw ntawm cov ntshav Pod yog ntawm B.

Tam sim no peb paub yuav ua li cas cov ntaub ntawv thov cuam tshuam rau kev xaiv ntawm node los khiav Pod. Tab sis dab tsi yog qhov cuam tshuam ntawm cov peev txheej marginal?

Cov peev txheej txwv yog ib thaj tsam uas CPU / MEM tsis tuaj yeem hla. Txawm li cas los xij, cov peev txheej CPU hloov pauv tau, yog li cov thawv uas ncav cuag lawv cov kev txwv CPU yuav tsis ua rau Pod tawm. Hloov chaw, CPU throttling yuav pib. Yog tias MEM siv txwv tsis pub dhau, lub thawv yuav tsum nres vim OOM-Killer thiab rov pib dua yog tias tso cai los ntawm RestartPolicy teeb tsa.

Thov thiab cov peev txheej siab tshaj plaws hauv kev nthuav dav

Kev sib txuas lus ntawm Docker thiab Kubernetes

Txoj hauv kev zoo tshaj plaws los piav qhia txog kev thov cov peev txheej thiab cov peev txheej ua haujlwm li cas yog los qhia txog kev sib raug zoo ntawm Kubernetes thiab Docker. Hauv daim duab saum toj no koj tuaj yeem pom yuav ua li cas Kubernetes teb thiab Docker startup chij muaj feem xyuam.

Nco: thov thiab txwv

containers:
...
 resources:
   requests:
     memory: "0.5Gi"
   limits:
     memory: "1Gi"

Raws li tau hais los saum no, lub cim xeeb yog ntsuas hauv bytes. Raws li Kubernetes cov ntaub ntawv, peb tuaj yeem qhia qhov cim xeeb ua tus lej. Feem ntau nws yog tus lej, piv txwv li 2678 - uas yog, 2678 bytes. Koj tseem tuaj yeem siv cov lus xaus G и Gi, qhov tseem ceeb tshaj plaws yog kom nco ntsoov tias lawv tsis sib npaug. Qhov thib ib yog kaum thiab qhov thib ob yog binary. Zoo li cov piv txwv hais hauv cov ntaub ntawv k8s: 128974848, 129e6, 129M, 123Mi - lawv xyaum sib npaug.

Kubernetes kev xaiv limits.memory phim tus chij --memory los ntawm Docker. Yog li cas request.memory Tsis muaj xub rau Docker vim Docker tsis siv daim teb no. Tej zaum koj yuav nug, qhov no puas tseem ceeb? Yog xav tau. Raws li kuv tau hais ua ntej, thaj chaw tseem ceeb rau Kubernetes. Raws li cov ntaub ntawv los ntawm nws, kube-scheduler txiav txim siab ntawm qhov twg los teem caij lub Pod.

Yuav ua li cas yog tias koj teem lub cim xeeb tsis txaus rau kev thov?

Yog tias lub thawv tau mus txog qhov txwv ntawm qhov kev thov nco, ces Pods tau muab tso rau hauv ib pab pawg Pods uas nres thaum tsis muaj lub cim xeeb txaus hauv lub pob.

Yuav ua li cas yog tias koj teem lub cim xeeb tsawg dhau lawm?

Yog tias lub thawv ntim ntau tshaj qhov kev nco, nws yuav raug txiav tawm vim OOM-Killed. Thiab yuav rov pib dua yog tias ua tau raws li RestartPolicy qhov twg tus nqi pib Always.

Yuav ua li cas yog tias koj tsis qhia lub cim xeeb thov?

Kubernetes yuav coj tus nqi txwv thiab teem nws ua tus nqi qub.

Yuav ua li cas tshwm sim yog tias koj tsis qhia meej txog qhov txwv kev nco?

Lub thawv tsis muaj kev txwv; nws tuaj yeem siv lub cim xeeb ntau npaum li nws xav tau. Yog tias nws pib siv tag nrho cov cim xeeb ntawm lub pob, ces OOM yuav tua nws. Lub thawv yuav rov pib dua yog tias ua tau raws li RestartPolicy.

Yuav ua li cas yog tias koj tsis qhia txog kev txwv kev nco?

Qhov no yog qhov xwm txheej phem tshaj plaws: tus neeg teem sijhawm tsis paub ntau npaum li cas cov peev txheej hauv lub thawv xav tau, thiab qhov no tuaj yeem ua rau muaj teeb meem loj ntawm lub pob. Hauv qhov no, nws yuav zoo kom muaj qhov txwv tsis pub dhau ntawm lub npe (teev los ntawm LimitRange). Tsis muaj qhov txwv tsis pub dhau - Pod tsis muaj kev txwv, nws tuaj yeem siv lub cim xeeb ntau npaum li nws xav tau.

Yog hais tias qhov kev thov nco ntau tshaj qhov node tuaj yeem muab tau, Pod yuav tsis teem sijhawm. Nws tseem ceeb heev uas yuav tsum nco ntsoov qhov ntawd Requests.memory - tsis yog tus nqi tsawg kawg nkaus. Qhov no yog ib qho kev piav qhia txog qhov muaj peev xwm ntawm lub cim xeeb txaus kom lub thawv khiav tsis tu ncua.

Nws yog feem ntau pom zoo kom teem tib tus nqi rau request.memory и limit.memory. Qhov no ua kom ntseeg tau tias Kubernetes yuav tsis teem sijhawm Pod ntawm lub pob uas muaj lub cim xeeb txaus los khiav Pod tab sis tsis txaus los khiav nws. Nco ntsoov: Kubernetes Pod kev npaj tsuas yog siv rau hauv tus account requests.memorythiab limits.memory tsis coj mus rau hauv tus account.

CPU: thov thiab txwv

containers:
...
 resources:
   requests:
     cpu: 1
   limits:
     cpu: "1200m"

Nrog CPU txhua yam yog qhov nyuaj me ntsis. Rov qab mus rau daim duab ntawm kev sib raug zoo ntawm Kubernetes thiab Docker, koj tuaj yeem pom qhov ntawd request.cpu соответствует --cpu-shares, whereas limit.cpu phim tus chij cpus hauv Docker.

CPU uas Kubernetes thov yog muab faib ua 1024, qhov feem ntawm CPU mus. Yog hais tias koj xav thov 1 tag nrho core, koj yuav tsum tau ntxiv cpu: 1raws li qhia saum toj no.

Thov kom tag nrho cov ntsiav (proportion = 1024) tsis txhais hais tias koj lub thawv yuav tau txais nws. Yog tias koj lub tshuab hostess tsuas muaj ib qho tseem ceeb thiab koj tab tom khiav ntau tshaj ib lub thawv, ces txhua lub ntim yuav tsum muab cov CPU muaj nyob nruab nrab ntawm lawv. Qhov no tshwm sim li cas? Cia peb saib daim duab.

Kev thov CPU - Ib qho Core System

Cia peb xav txog tias koj muaj ib tug tub ntxhais-core host system khiav ntim. Niam (Kubernetes) ci ncuav qab zib (CPU) thiab xav faib nws ntawm cov me nyuam (cov thawv). Peb tus menyuam xav tau ib lub ncuav tag nrho (feem ntau = 1024), lwm tus menyuam xav tau ib nrab lub ncuav (512). Niam xav kom ncaj ncees thiab ua kom yooj yim xam.

# Сколько пирогов хотят дети?
# 3 ребенка хотят по целому пирогу и еще один хочет половину пирога
cakesNumberKidsWant = (3 * 1) + (1 * 0.5) = 3.5
# Выражение получается так:
3 (ребенка/контейнера) * 1 (целый пирог/полное ядро) + 1 (ребенок/контейнер) * 0.5 (половина пирога/половина ядра)
# Сколько пирогов испечено?
availableCakesNumber = 1
# Сколько пирога (максимально) дети реально могут получить?
newMaxRequest = 1 / 3.5 =~ 28%

Raws li kev suav, peb tus menyuam yuav tau txais 28% ntawm cov tub ntxhais kawm, thiab tsis yog tag nrho cov tub ntxhais kawm. Tus menyuam thib plaub yuav tau txais 14% ntawm tag nrho cov ntsiav, tsis yog ib nrab. Tab sis tej yam yuav txawv yog hais tias koj muaj ib tug multi-core system.

CPU Thov - Multi-Core (4) System

Hauv daim duab saum toj no koj tuaj yeem pom tias peb tus menyuam xav tau lub ncuav qab zib tag nrho, thiab ib tus xav tau ib nrab. Txij li thaum niam tau ci plaub lub ncuav, txhua tus ntawm nws cov menyuam yuav tau txais ntau npaum li lawv xav tau. Nyob rau hauv ib tug multi-core system, processor cov peev txheej raug faib thoob plaws tag nrho cov processor cores. Yog tias lub thawv tau txwv tsawg dua ib qho tseem ceeb ntawm CPU, nws tseem tuaj yeem siv nws ntawm 100%.

Cov kev suav saum toj no tau yooj yim kom nkag siab tias CPU faib li cas hauv cov thawv. Tau kawg, dhau li ntawm cov ntim lawv tus kheej, muaj lwm cov txheej txheem uas tseem siv CPU cov peev txheej. Thaum cov txheej txheem hauv ib lub thawv tsis ua haujlwm, lwm tus tuaj yeem siv nws cov peev txheej. CPU: "200m" соответствует CPU: 0,2, uas txhais tau hais tias kwv yees li 20% ntawm ib qho tseem ceeb.

Tam sim no cia peb tham txog limit.cpu. Lub CPU uas Kubernetes txwv tau muab faib ua 100. Qhov tshwm sim yog lub sijhawm lub thawv tuaj yeem siv txhua 100 µs (cpu-period).

limit.cpu phim Docker chij --cpus. Nov yog kev sib xyaw tshiab ntawm qub --cpu-period и --cpu-quota. Los ntawm kev teeb tsa nws, peb qhia tias muaj pes tsawg CPU cov peev txheej uas lub thawv tuaj yeem siv tau ntau tshaj plaws ua ntej throttling pib:

cpus - kev sib xyaw cpu-period и cpu-quota. cpus = 1.5 sib npaug rau kev teeb tsa cpu-period = 100000 и cpu-quota = 150000;
CPU-lub sijhawm - lub sij hawm CPU CFS teem sijhawm, default 100 microseconds;
cpu-cov - tus naj npawb microseconds sab hauv cpu-period, uas yog bounded los ntawm lub thawv.

Yuav ua li cas yog tias koj nruab tsis txaus thov CPU?

Yog tias lub thawv xav tau ntau dua li nws tau teeb tsa, nws yuav nyiag CPU los ntawm lwm cov txheej txheem.

Yuav ua li cas yog tias koj teem CPU txwv tsawg dhau lawm?

Txij li thaum cov peev txheej CPU tuaj yeem kho tau, throttling yuav qhib.

Yuav ua li cas yog tias koj tsis qhia qhov kev thov CPU?

Raws li nrog kev nco, tus nqi thov yog sib npaug rau qhov txwv.

Yuav ua li cas yog tias koj tsis qhia txog CPU txwv?

Lub thawv yuav siv CPU ntau npaum li nws xav tau. Yog hais tias lub neej ntawd CPU txoj cai (LimitRange) tau txhais nyob rau hauv lub namespace, ces qhov kev txwv no kuj siv rau lub thawv.

Yuav ua li cas yog tias koj tsis qhia ib qho kev thov lossis CPU txwv?

Raws li nrog kev nco, qhov no yog qhov xwm txheej phem tshaj plaws. Tus neeg teem sijhawm tsis paub ntau npaum li cas cov peev txheej uas koj lub thawv xav tau, thiab qhov no tuaj yeem ua rau muaj teeb meem loj rau ntawm node. Txhawm rau zam qhov no, koj yuav tsum teeb tsa lub neej ntawd txwv rau namespaces (LimitRange).

Nco ntsoov: yog tias koj thov CPU ntau dua li cov nodes tuaj yeem muab tau, Pod yuav tsis teem sijhawm. Requests.cpu - tsis yog qhov tsawg kawg nkaus tus nqi, tab sis tus nqi txaus los pib lub Pod thiab ua haujlwm yam tsis muaj kev poob haujlwm. Yog tias daim ntawv thov tsis ua cov kev suav ua nyuaj, qhov kev xaiv zoo tshaj plaws yog rau nruab request.cpu <= 1 thiab tso tawm ntau yam replicas raws li xav tau.

Qhov zoo tshaj plaws ntawm kev thov cov peev txheej lossis cov peev txheej txwv

Peb kawm txog qhov txwv ntawm kev siv nyiaj txiag. Tam sim no nws yog lub sijhawm los teb cov lus nug: "Puas muaj pes tsawg lub peev txheej kuv Pod xav tau los khiav daim ntawv thov yam tsis muaj teeb meem? Tus nqi zoo tshaj yog dab tsi?

Hmoov tsis zoo, tsis muaj cov lus teb meej rau cov lus nug no. Yog tias koj tsis paub tias koj daim ntawv thov ua haujlwm li cas lossis ntau npaum li cas CPU lossis lub cim xeeb nws xav tau, qhov kev xaiv zoo tshaj yog muab daim ntawv thov ntau lub cim xeeb thiab CPU thiab tom qab ntawd khiav cov kev xeem ua haujlwm.

Ntxiv nrog rau kev ntsuas kev ua tau zoo, saib xyuas daim ntawv thov tus cwj pwm hauv kev saib xyuas rau ib lub lim tiam. Yog tias cov duab qhia tias koj daim ntawv thov siv tsawg dua li qhov koj thov, koj tuaj yeem txo qhov CPU lossis nco tau thov.

Ua piv txwv saib qhov no Grafana dashboard. Nws qhia qhov sib txawv ntawm cov kev thov los yog cov peev txheej txwv thiab kev siv cov peev txheej tam sim no.

xaus

Thov thiab txwv cov peev txheej pab ua kom koj pawg Kubernetes noj qab nyob zoo. Txoj kev txwv kom raug txo qis cov nqi thiab ua kom cov ntawv thov khiav txhua lub sijhawm.

Hauv ntej, muaj ob peb yam uas yuav tsum nco ntsoov:

Thov cov peev txheej yog ib qho kev teeb tsa uas tau muab coj los rau hauv tus account thaum pib lub sijhawm (thaum Kubernetes npaj los tuav daim ntawv thov). Hauv qhov sib piv, txwv cov peev txheej yog qhov tseem ceeb ntawm lub sijhawm ua haujlwm-thaum daim ntawv thov twb tau khiav ntawm node.
Piv nrog rau kev nco, CPU yog ib qho kev tswj hwm. Yog tias tsis muaj CPU txaus, koj lub Pod yuav tsis kaw thiab lub tshuab kaw lus yuav qhib.
Thov cov peev txheej thiab cov peev txheej txwv tsis yog qhov tsawg kawg nkaus thiab qhov siab tshaj plaws! Los ntawm kev txhais cov peev txheej thov, koj xyuas kom meej tias daim ntawv thov yuav khiav tsis muaj teeb meem.
Ib qho kev xyaum zoo yog los teeb tsa lub cim xeeb thov kom sib npaug ntawm lub cim xeeb txwv.
Ok install thov CPU <=1, yog hais tias daim ntawv thov tsis ua complex xam.
Yog tias koj thov cov peev txheej ntau dua li muaj nyob rau ntawm lub pob, Pod yuav tsis raug teem sijhawm rau ntawm qhov ntawd.
Txhawm rau txiav txim siab qhov tseeb ntawm qhov thov cov peev txheej / cov peev txheej txwv, siv cov khoom thauj khoom thiab saib xyuas.

Kuv vam tias tsab xov xwm no yuav pab koj nkag siab txog lub ntsiab lus tseem ceeb ntawm kev txwv cov peev txheej. Thiab koj yuav siv tau qhov kev paub no hauv koj txoj haujlwm.

Hmoov zoo!

Yuav nyeem dab tsi ntxiv:

Tau qhov twg los: www.hab.com

Yuav nkag mus rau Kubernetes Pod cov peev txheej li cas