Saib xyuas Kubernetes pab pawg

Kuv tsim Kube Eagle - Prometheus exporter. Nws tau dhau los ua qhov txias uas yuav pab kom nkag siab zoo dua cov peev txheej ntawm pawg me thiab nruab nrab. Thaum kawg, kuv tau khaws ntau pua nyiaj vim kuv xaiv hom tshuab zoo thiab teeb tsa daim ntawv thov kev txwv rau kev ua haujlwm.

Kuv mam li qhia koj txog cov txiaj ntsig Kub Eagle, tab sis ua ntej kuv yuav piav qhia dab tsi ua rau muaj kev ntxhov siab thiab vim li cas thiaj li xav tau kev saib xyuas zoo.

Kuv tswj tau ob peb pawg ntawm 4-50 nodes. Txhua pawg muaj txog 200 microservices thiab daim ntawv thov. Txhawm rau kom siv tau zoo dua ntawm cov khoom siv uas twb muaj lawm, feem ntau kev xa tawm tau teeb tsa nrog cov khoom siv RAM thiab CPU tawg. Txoj kev no, pods tuaj yeem nqa cov peev txheej yog tias tsim nyog, thiab tib lub sijhawm tsis cuam tshuam nrog lwm cov ntawv thov ntawm node. Zoo, tsis yog nws zoo?

Thiab txawm hais tias pawg tau siv CPU tsawg (8%) thiab RAM (40%), peb pheej muaj teeb meem nrog cov pods raug preempted thaum lawv sim faib ntau lub cim xeeb dua li muaj nyob rau ntawm node. Tom qab ntawd peb tsuas muaj ib lub dashboard rau saib xyuas Kubernetes cov peev txheej. Zoo li no:

Grafana dashboard nrog cAdvisor metrics nkaus xwb

Nrog lub vaj huam sib luag zoo li no, nws tsis yog qhov teeb meem pom cov nodes uas noj ntau lub cim xeeb thiab CPU. Qhov teeb meem yog xav paub seb yog vim li cas. Txhawm rau khaws cov pods nyob rau hauv qhov chaw, ib qho tuaj yeem teeb tsa cov peev txheej tau lees paub ntawm txhua lub pods (thov cov peev txheej sib npaug rau qhov txwv). Tab sis qhov no tsis yog qhov smartest siv kho vajtse. Cov pawg muaj ob peb puas gigabytes ntawm lub cim xeeb, thaum qee cov nodes tau tshaib plab, thaum lwm tus muaj 4-10 GB tshuav nyob rau hauv cia.

Nws hloov tawm tias Kubernetes tus teem sijhawm faib cov haujlwm tsis sib xws thoob plaws cov peev txheej muaj. Lub Kubernetes teem sijhawm siv rau hauv cov kev teeb tsa sib txawv: affinity, taints thiab tolerations cov cai, node selectors uas tuaj yeem txwv cov nodes muaj. Tab sis nyob rau hauv kuv rooj plaub tsis muaj dab tsi zoo li ntawd, thiab cov pods tau npaj nyob ntawm seb qhov kev thov kev pab ntawm txhua lub node.

Cov node uas muaj cov peev txheej pub dawb tshaj plaws thiab qhov ua tau raws li qhov kev thov tau raug xaiv rau lub pod. Peb pom tias cov ntaub ntawv thov ntawm cov nodes tsis phim qhov kev siv tiag tiag, thiab qhov no yog qhov twg Kube Eagle thiab nws cov peev txheej saib xyuas cov peev txheej tau los cawm.

Kuv muaj yuav luag tag nrho Kubernetes pawg saib xyuas nrog Node exporter и Kube State Metrics. Node Exporter muab kev txheeb cais ntawm I / O thiab disk, CPU, thiab RAM kev siv, thaum Kube State Metrics qhia Kubernetes cov khoom ntsuas xws li kev thov thiab CPU thiab kev txwv kev txwv.

Peb yuav tsum tau muab cov kev ntsuas kev siv nrog cov kev thov thiab txwv kev ntsuas hauv Grafana, thiab tom qab ntawd peb yuav tau txais tag nrho cov ntaub ntawv hais txog qhov teeb meem. Qhov no suab yooj yim, tab sis ob lub cuab yeej yeej npe cov ntawv sib txawv, thiab qee qhov kev ntsuas tsis muaj cov ntawv sau metadata txhua. Kube Eagle ua txhua yam nws tus kheej thiab lub vaj huam sib luag zoo li no:

Kube Eagle Dashboard

Peb tau tswj kom daws tau ntau yam teeb meem nrog cov peev txheej thiab txuag cov cuab yeej:

1. Qee tus neeg tsim khoom tsis paub ntau npaum li cas cov kev pabcuam microservices xav tau (lossis tsuas yog tsis thab). Tsis muaj txoj hauv kev rau peb nrhiav kev thov tsis raug rau cov peev txheej - rau qhov no peb yuav tsum paub txog kev noj ntxiv rau kev thov thiab kev txwv. Tam sim no lawv pom Prometheus metrics, saib xyuas kev siv tiag tiag thiab kho cov lus thov thiab txwv.
2. JVM daim ntawv thov siv RAM ntau npaum li lawv tuaj yeem ua tau. Cov khoom khib nyiab tsuas yog tso tawm lub cim xeeb thaum siv ntau dua 75%. Thiab txij li feem ntau cov kev pabcuam muaj kev nco tau tawg, nws yeej ib txwm nyob los ntawm JVM. Yog li ntawd, tag nrho cov kev pabcuam Java no tau noj ntau RAM ntau dua li qhov xav tau.
3. Qee daim ntawv thov thov nco ntau dhau, thiab Kubernetes tus teem sijhawm tsis tau muab cov nodes rau lwm daim ntawv thov, txawm tias qhov tseeb lawv tau dawb dua li lwm cov nodes. Ib tus tsim tawm yuam kev ntxiv tus lej ntxiv hauv qhov kev thov thiab rub ib daim loj ntawm RAM: 20 GB es tsis txhob 2. Tsis muaj leej twg pom. Daim ntawv thov muaj 3 replicas, yog li ntau li 3 nodes raug cuam tshuam.
4. Peb tau qhia txog kev txwv cov peev txheej, rov teem dua pods nrog qhov kev thov kom raug, thiab tau txais qhov zoo tshaj plaws ntawm kev siv kho vajtse thoob plaws txhua qhov. Ob peb lub nodes tuaj yeem raug kaw tag nrho. Thiab tom qab ntawd peb pom tias peb muaj cov tshuab tsis raug (CPU taw qhia, tsis nco qab taw qhia). Peb hloov hom thiab deleted ob peb ntxiv nodes.

Cov txiaj ntsim tau los

Nrog cov peev txheej tawg tau nyob hauv pawg, koj siv cov khoom siv muaj txiaj ntsig zoo dua, tab sis Kubernetes lub sijhawm teem sijhawm pods raws li kev thov rau cov peev txheej, thiab qhov no yog qhov tsis txaus ntseeg. Txhawm rau tua ob tus noog nrog ib lub pob zeb: kom tsis txhob muaj teeb meem thiab siv cov peev txheej kom tiav, koj yuav tsum tau saib xyuas zoo. Qhov no yog vim li cas nws yuav pab tau Kub Eagle (Prometheus exporter thiab Grafana dashboard).

Tau qhov twg los: www.hab.com