Sannu duka! Sunana Oleg Sidorenkov, Ina aiki a DomClick a matsayin shugaban kungiyar samar da ababen more rayuwa. Mun kasance muna amfani da Kubik a cikin samarwa fiye da shekaru uku, kuma a wannan lokacin mun sami lokuta daban-daban masu ban sha'awa tare da shi. A yau zan gaya muku yadda, tare da tsarin da ya dace, zaku iya matsi ƙarin aiki daga vanilla Kubernetes don tarin ku. Shirya ci gaba!
Dukanku kun san sarai cewa Kubernetes tsarin buɗaɗɗen tushe ne mai ƙima don ƙungiyar kade-kade; da kyau, ko kuma 5 binaries waɗanda ke yin sihiri ta hanyar sarrafa yanayin rayuwar microservices a cikin mahallin uwar garken. Bugu da ƙari, kayan aiki ne mai sauƙi wanda za'a iya haɗa shi kamar Lego don iyakar gyare-gyare don ayyuka daban-daban.
Kuma komai yana da kyau: jefa sabobin a cikin gungu kamar itacen wuta a cikin akwatin wuta, kuma ba za ku san wani baƙin ciki ba. Amma idan kun kasance don muhalli, za ku yi tunani: "Ta yaya zan iya ci gaba da ci gaba da ci gaba da ci gaba da ci gaba da dajin?" Ma'ana, yadda ake nemo hanyoyin inganta ababen more rayuwa da rage farashi.
1. Kula da ƙungiyar da albarkatun aikace-aikace
Ɗaya daga cikin wuraren da aka fi sani, amma hanyoyi masu tasiri shine gabatar da buƙatun / iyakoki. Rarraba aikace-aikace ta wuraren suna, da wuraren suna ta ƙungiyoyin ci gaba. Kafin turawa, saita ƙimar aikace-aikacen don amfani da lokacin sarrafawa, ƙwaƙwalwar ajiya, da ma'ajin ƙima.
resources:
requests:
memory: 2Gi
cpu: 250m
limits:
memory: 4Gi
cpu: 500mTa hanyar kwarewa, mun zo ga ƙarshe: bai kamata ku ƙaddamar da buƙatun daga iyakoki fiye da sau biyu ba. Ana ƙididdige ƙarar gungu bisa ga buƙatun, kuma idan kun ba da aikace-aikacen bambancin albarkatu, alal misali, sau 5-10, to, kuyi tunanin abin da zai faru da kumburin ku lokacin da ya cika da kwasfa kuma ba zato ba tsammani ya karɓi kaya. Babu wani abu mai kyau. A mafi ƙanƙanta, matsawa, kuma a matsakaici, za ku yi bankwana da ma'aikaci kuma ku sami nauyin hawan keke akan ragowar nodes bayan kwas ɗin sun fara motsi.
Bugu da ƙari, tare da taimako limitranges A farkon, zaku iya saita ƙimar albarkatu don akwati - mafi ƙarancin, matsakaici da tsoho:
➜ ~ kubectl describe limitranges --namespace ops
Name: limit-range
Namespace: ops
Type Resource Min Max Default Request Default Limit Max Limit/Request Ratio
---- -------- --- --- --------------- ------------- -----------------------
Container cpu 50m 10 100m 100m 2
Container ephemeral-storage 12Mi 8Gi 128Mi 4Gi -
Container memory 64Mi 40Gi 128Mi 128Mi 2Kar a manta da iyakance albarkatun sararin suna ta yadda ƙungiya ɗaya ba za ta iya karɓar duk albarkatun tarin ba:
➜ ~ kubectl describe resourcequotas --namespace ops
Name: resource-quota
Namespace: ops
Resource Used Hard
-------- ---- ----
limits.cpu 77250m 80
limits.memory 124814367488 150Gi
pods 31 45
requests.cpu 53850m 80
requests.memory 75613234944 150Gi
services 26 50
services.loadbalancers 0 0
services.nodeports 0 0Kamar yadda ake iya gani daga bayanin resourcequotas, Idan ƙungiyar ops suna son tura kwas ɗin da za su cinye wani 10 cpu, mai tsarawa ba zai ƙyale wannan ba kuma zai jefa kuskure:
Error creating: pods "nginx-proxy-9967d8d78-nh4fs" is forbidden: exceeded quota: resource-quota, requested: limits.cpu=5,requests.cpu=5, used: limits.cpu=77250m,requests.cpu=53850m, limited: limits.cpu=10,requests.cpu=10Don magance irin wannan matsala, zaka iya rubuta kayan aiki, misali, kamar , iya adanawa da aiwatar da albarkatun umarni.
2. Zaɓi mafi kyawun ajiyar fayil
Anan zan so in taɓa batun jigon juzu'i da tsarin faifai na nodes na ma'aikatan Kubernetes. Ina fatan cewa babu wanda ke amfani da "Cube" akan HDD a samarwa, amma wani lokacin SSD na yau da kullun bai isa ba. Mun ci karo da matsala inda rajistan ayyukan ke kashe faifai saboda ayyukan I/O, kuma babu mafita da yawa:
-
Yi amfani da SSDs masu girma ko canza zuwa NVMe (idan kuna sarrafa kayan aikin ku).
-
Rage matakin shiga.
-
Yi daidaita “smart” na kwas ɗin da ke lalata diski (
podAntiAffinity).
Allon da ke sama yana nuna abin da ke faruwa a ƙarƙashin nginx-ingress-controller zuwa faifai lokacin da aka kunna shiga_logs log (~ 12 dubu rajistan ayyukan / sec). Wannan yanayin, ba shakka, na iya haifar da lalacewar duk aikace-aikace akan wannan kumburi.
Amma ga PV, kash, ban gwada komai ba Juzu'i masu dagewa. Yi amfani da mafi kyawun zaɓi wanda ya dace da ku. A tarihi, ya faru a cikin ƙasarmu cewa ƙaramin ɓangaren sabis yana buƙatar kundin RWX, kuma tun da daɗewa sun fara amfani da ajiyar NFS don wannan aikin. Mai arha kuma ... isa. Tabbas, ni da shi mun ci shit - albarkace ku, amma mun koyi yadda ake gyara shi, kuma kaina ba ya jin zafi kuma. Kuma idan zai yiwu, matsa zuwa ma'ajin abu na S3.
3. Tattara ingantattun hotuna
Zai fi kyau a yi amfani da ingantattun hotuna na kwantena ta yadda Kubernetes zai iya ɗauko su da sauri kuma ya aiwatar da su yadda ya kamata.
Ingantacce yana nufin cewa hotuna:
-
ƙunshi aikace-aikace guda ɗaya ko yin aiki ɗaya kawai;
-
ƙananan girman, saboda ana watsa manyan hotuna mafi muni akan hanyar sadarwa;
-
samun lafiya da shirye-shiryen ƙarshen shirye-shiryen da ke ba da damar Kubernetes suyi aiki a yayin da ake raguwa;
-
yi amfani da tsarin aiki masu dacewa da kwantena (kamar Alpine ko CoreOS), waɗanda suka fi tsayayya da kurakuran daidaitawa;
-
yi amfani da ginin matakai da yawa ta yadda za ku iya tura aikace-aikacen da aka haɗa kawai ba hanyoyin da ke tare ba.
Akwai kayan aiki da ayyuka da yawa waɗanda ke ba ku damar dubawa da haɓaka hotuna akan tashi. Yana da mahimmanci koyaushe a sabunta su kuma a gwada su don aminci. A sakamakon haka kuna samun:
-
Rage nauyin cibiyar sadarwa a kan dukkan tari.
-
Rage lokacin farawa kwantena.
-
Karamin girman duk rajistar Docker ku.
4. Yi amfani da cache na DNS
Idan muka yi magana game da manyan lodi, to rayuwa tana da kyau ba tare da daidaita tsarin DNS na gungu ba. A wani lokaci, masu haɓaka Kubernetes sun goyi bayan maganin kube-dn su. An kuma aiwatar da ita a nan, amma wannan software ba ta kasance musamman da aka gyara ba kuma ba ta samar da aikin da ake bukata ba, ko da yake yana da alama aiki ne mai sauƙi. Sannan coredns ya bayyana, wanda muka canza zuwa kuma ba mu da baƙin ciki; daga baya ya zama tsohuwar sabis na DNS a cikin K8s. A wani lokaci, mun girma zuwa 40 dubu rps zuwa tsarin DNS, kuma wannan bayani ya zama kasa. Amma, cikin sa'a, Nodelocaldns ya fito, aka node local cache, aka .
Me yasa muke amfani da wannan? Akwai bug a cikin Linux kernel wanda, lokacin da kira da yawa ta hanyar conntrack NAT akan UDP, ya haifar da yanayin tsere don shigarwar a cikin tebur na conntrack, kuma wani ɓangare na zirga-zirga ta hanyar NAT ya ɓace (kowace tafiya ta hanyar Sabis shine NAT). Nodelocaldns yana magance wannan matsalar ta hanyar kawar da NAT da haɓaka haɗin kai zuwa TCP zuwa sama na DNS, da kuma caching na sama da tambayoyin DNS na gida (ciki har da gajeriyar 5-second korau cache).
5. Sikelin kwas ɗin a kwance da a tsaye ta atomatik
Shin za ku iya cewa da kwarin gwiwa cewa duk microservices ɗinku a shirye suke don haɓaka nauyi sau biyu zuwa uku? Yadda ake rarraba albarkatu da kyau ga aikace-aikacenku? Tsayar da kwas ɗin guda biyu suna gudana fiye da nauyin aiki na iya zama mai wuyar gaske, amma mayar da su baya yana haifar da haɗarin raguwa daga haɓakar zirga-zirgar ababen hawa zuwa sabis. Ayyuka kamar и .
VPA yana ba ku damar ɗaga buƙatun/iyakan kwantena ta atomatik a cikin kwaf ɗin ya dogara da ainihin amfani. Ta yaya zai zama da amfani? Idan kuna da kwasfa waɗanda ba za a iya ƙididdige su a kwance ba saboda wasu dalilai (wanda ba abin dogaro gabaɗaya ba ne), to kuna iya ƙoƙarin ba da sauye-sauye ga albarkatun sa zuwa VPA. Siffar sa shine tsarin shawarwarin dangane da bayanan tarihi da na yanzu daga uwar garken awo, don haka idan ba kwa son canza buƙatun / iyakoki ta atomatik, zaku iya kawai saka idanu abubuwan da aka ba da shawarar don kwantena ku kuma inganta saitunan don adana CPU da ƙwaƙwalwar ajiya a cikin gungu.
Hoton da aka ɗauka daga https://levelup.gitconnected.com/kubernetes-autoscaling-101-cluster-autoscaler-horizontal-pod-autoscaler-and-vertical-pod-2a441d9ad231
Mai tsara jadawalin a Kubernetes koyaushe yana dogara ne akan buƙatun. Kowace darajar da kuka sanya a wurin, mai tsara jadawalin zai nemo kulli mai dacewa bisa ta. Ana buƙatar ƙimar iyakoki don cubelet don fahimtar lokacin da za a murƙushe ko kashe kwaf ɗin. Kuma tunda kawai mahimman ma'aunin shine ƙimar buƙatun, VPA za ta yi aiki tare da shi. Duk lokacin da kuka daidaita aikace-aikacen a tsaye, kuna ayyana abin da buƙatun ya kamata su kasance. Me zai faru da iyaka to? Wannan ma'auni kuma za a daidaita daidai gwargwado.
Misali, ga saitunan kwas ɗin da aka saba:
resources:
requests:
memory: 250Mi
cpu: 200m
limits:
memory: 500Mi
cpu: 350mInjin shawarwarin yana ƙayyade cewa aikace-aikacenku yana buƙatar CPU 300m da 500Mi don yin aiki da kyau. Za ku sami saitunan masu zuwa:
resources:
requests:
memory: 500Mi
cpu: 300m
limits:
memory: 1000Mi
cpu: 525mKamar yadda aka ambata a sama, wannan ma'auni daidai ne dangane da buƙatun / iyakoki a cikin bayyani:
-
CPU: 200m → 300m: rabo 1: 1.75;
-
Ƙwaƙwalwar ajiya: 250Mi → 500Mi: rabo 1:2.
Game da HPA, to, tsarin aiki ya fi dacewa. Ma'auni kamar CPU da ƙwaƙwalwar ajiya suna da ƙima, kuma idan matsakaicin duk kwafi ya wuce madaidaicin, aikace-aikacen ana auna shi ta +1 sub har sai ƙimar ta faɗi ƙasa da maƙasudin ko har sai an kai matsakaicin adadin kwafi.
Hoton da aka ɗauka daga https://levelup.gitconnected.com/kubernetes-autoscaling-101-cluster-autoscaler-horizontal-pod-autoscaler-and-vertical-pod-2a441d9ad231
Baya ga ma'auni na yau da kullun kamar CPU da ƙwaƙwalwar ajiya, zaku iya saita ƙofa akan ma'aunin ku na al'ada daga Prometheus kuma kuyi aiki tare da su idan kuna tunanin hakan shine mafi daidaiton nunin lokacin da za'a haɓaka aikace-aikacenku. Da zarar aikace-aikacen ya daidaita ƙasa da ƙayyadaddun ma'aunin ma'auni, HPA za ta fara ƙaddamar da ma'auni zuwa mafi ƙarancin adadin kwafi ko har sai nauyin ya dace da ƙayyadaddun ƙofa.
6. Kar a manta game da Node Affinity da Pod Affinity
Ba duk nodes ke gudana akan kayan aiki iri ɗaya ba, kuma ba duk kwas ɗin ke buƙatar gudanar da aikace-aikacen ƙididdiga ba. Kubernetes yana ba ku damar saita ƙwarewar nodes da kwas ɗin ta amfani da su Ƙunƙwasawa и Dangantakar Pod.
Idan kuna da nodes waɗanda suka dace da ayyukan ƙididdigewa, to don mafi girman inganci yana da kyau a ɗaure aikace-aikacen zuwa nodes masu dacewa. Don yin wannan amfani nodeSelector tare da alamar kumburi.
Bari mu ce kuna da nodes biyu: ɗaya tare da CPUType=HIGHFREQ da kuma babban adadin maƙallan sauri, wani tare da MemoryType=HIGHMEMORY ƙarin ƙwaƙwalwar ajiya da saurin aiki. Hanya mafi sauƙi ita ce sanya turawa zuwa kumburi HIGHFREQta ƙara zuwa sashe spec wannan mai zabar:
…
nodeSelector:
CPUType: HIGHFREQHanya mafi tsada da takamaiman hanyar yin wannan ita ce amfani nodeAffinity a cikin filin affinity razdala spec. Akwai zaɓuɓɓuka guda biyu:
-
requiredDuringSchedulingIgnoredDuringExecution: saiti mai wuya (mai tsarawa zai tura kwasfan fayiloli kawai akan takamaiman nodes (kuma babu wani wuri)); -
preferredDuringSchedulingIgnoredDuringExecution: saiti mai laushi (mai tsarawa zai yi ƙoƙarin tura zuwa takamaiman nodes, kuma idan hakan ya kasa, zai yi ƙoƙarin turawa zuwa kumburi na gaba).
Kuna iya ƙididdige ƙayyadaddun tsarin aiki don sarrafa alamun kumburi, kamar In, NotIn, Exists, DoesNotExist, Gt ko Lt. Koyaya, ku tuna cewa hadaddun hanyoyin a cikin dogayen jerin sunayen za su rage yanke shawara a cikin mawuyacin yanayi. Ma'ana, kiyaye shi cikin sauki.
Kamar yadda aka ambata a sama, Kubernetes yana ba ku damar saita alaƙar kwas ɗin na yanzu. Wato, zaku iya tabbatar da cewa wasu kwas ɗin suna aiki tare da wasu kwas ɗin a cikin yanki ɗaya samuwa (wanda ya dace da gajimare) ko nodes.
В podAffinity filayen affinity razdala spec filayen guda ɗaya suna samuwa kamar yadda yake a cikin yanayin nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution и preferredDuringSchedulingIgnoredDuringExecution. Bambancin kawai shi ne matchExpressions zai ɗaure kwafsas ɗin zuwa kumburin da ya riga ya ke tafiyar da kwafsa mai wannan alamar.
Kubernetes kuma yana ba da filin podAntiAffinity, wanda, akasin haka, ba ya ɗaure kwafsa zuwa kumburi tare da takamaiman kwasfa.
Game da maganganu nodeAffinity Hakanan za'a iya ba da shawara iri ɗaya: yi ƙoƙarin kiyaye ƙa'idodi masu sauƙi da ma'ana, kar a yi ƙoƙarin yin obalodi ƙayyadaddun kwas ɗin tare da ƙayyadaddun ƙa'idodi. Yana da matukar sauƙi don ƙirƙirar ƙa'idar da ba za ta dace da yanayin gungu ba, ƙirƙirar nauyin da ba dole ba a kan mai tsarawa da rage yawan aiki.
7. Taints & Juriya
Akwai wata hanya don sarrafa mai tsarawa. Idan kuna da babban gungu tare da ɗaruruwan nodes da dubunnan microservices, to yana da matukar wahala a daina barin wasu kwas ɗin da za a shirya su akan wasu nodes.
Hanyar taints - haramta dokoki - yana taimakawa da wannan. Misali, a wasu yanayi za ka iya haramta wasu nodes daga gudanar da kwasfa. Don shafa taint zuwa takamaiman kumburi kuna buƙatar amfani da zaɓin taint in kubectl. Ƙayyade maɓalli da ƙimar sa'an nan kuma taint kamar NoSchedule ko NoExecute:
$ kubectl taint nodes node10 node-role.kubernetes.io/ingress=true:NoScheduleHakanan yana da mahimmanci a lura cewa injin taint yana goyan bayan manyan sakamako guda uku: NoSchedule, NoExecute и PreferNoSchedule.
-
NoScheduleyana nufin cewa a yanzu ba za a sami shigar da ta dace a cikin ƙayyadaddun kwas ɗin batolerations, ba za a iya tura shi a kan kumburi ba (a cikin wannan misalinode10). -
PreferNoSchedule- Saukake sigarNoSchedule. A wannan yanayin, mai tsara jadawalin zai yi ƙoƙarin kada ya ware kwas ɗin da ba su da shigar da ta dacetolerationskowane kumburi, amma wannan ba ƙayyadaddun ƙaya ba ne. Idan babu albarkatu a cikin gungu, to, kwas ɗin za su fara turawa akan wannan kumburin. -
NoExecute- wannan tasirin yana haifar da fitar da kuɗaɗen da ba su da madaidaicin shigarwatolerations.
Abin sha'awa, ana iya soke wannan ɗabi'ar ta amfani da tsarin jurewa. Wannan ya dace lokacin da akwai kumburin “haramta” kuma kawai kuna buƙatar sanya ayyukan more rayuwa akansa. Yadda za a yi? Bada izini kawai waɗancan kwas ɗin waɗanda ke da dacewa da haƙuri.
Ga yadda ƙayyadaddun kwas ɗin zai yi kama:
spec:
tolerations:
- key: "node-role.kubernetes.io/ingress"
operator: "Equal"
value: "true"
effect: "NoSchedule"Wannan ba yana nufin cewa sake turawa na gaba zai faɗi akan wannan kulli na musamman ba, wannan ba tsarin Node Affinity bane kuma nodeSelector. Amma ta hanyar haɗa abubuwa da yawa, za ku iya cimma saitunan tsarawa masu sassauƙa.
8. Saita Tufafin Tufafi
Domin kawai kuna da kwas ɗin da aka sanya wa nodes ba yana nufin cewa dole ne a kula da dukkan kwas ɗin da fifiko iri ɗaya ba. Misali, kuna iya tura wasu kwas ɗin kafin wasu.
Kubernetes yana ba da hanyoyi daban-daban don saita fifikon Pod da Preemption. Saitin ya ƙunshi sassa da yawa: abu PriorityClass da bayanin filin priorityClassName a cikin kwafsa bayani. Bari mu kalli misali:
apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
name: high-priority
value: 99999
globalDefault: false
description: "This priority class should be used for very important pods only"Mun halitta PriorityClass, ba shi suna, kwatanci da ƙima. Mafi girma value, mafi girman fifiko. Ƙimar na iya zama kowane lamba 32-bit ƙasa da ko daidai da 1. An keɓance maɗaukaki mafi girma don madaidaicin tsarin kwas ɗin manufa wanda gabaɗaya ba za a iya rigaya ba. Matsala zai faru ne kawai idan wani babban fasfo mai mahimmanci ba shi da wurin juyawa, sannan za a kwashe wasu daga cikin kwas ɗin daga wani kumburi. Idan wannan tsarin ya kasance mai tsauri a gare ku, kuna iya ƙara zaɓin preemptionPolicy: Never, sa'an nan kuma ba za a yi preemption, kwafsa zai tsaya a farko a cikin jerin gwano kuma jira mai tsarawa don nemo albarkatun kyauta don shi.
Na gaba, mun ƙirƙiri wani kwasfa wanda a ciki muke nuna sunan priorityClassName:
apiVersion: v1
kind: Pod
metadata:
name: static-web
labels:
role: myrole
spec:
containers:
- name: web
image: nginx
ports:
- name: web
containerPort: 80
protocol: TCP
priorityClassName: high-priority
Kuna iya ƙirƙirar azuzuwan fifiko kamar yadda kuke so, kodayake ana ba da shawarar kada ku tafi da wannan (ce, iyakance kanku ga ƙananan, matsakaici da fifiko).
Don haka, idan ya cancanta, zaku iya haɓaka haɓakar ƙaddamar da ayyuka masu mahimmanci kamar nginx-ingress-controller, coredns, da sauransu.
9. Inganta gungu na ETCD
ETCD ana iya kiransa kwakwalwar duka tari. Yana da matukar muhimmanci a kula da aikin wannan bayanan a babban matakin, tun da saurin aiki a cikin Cube ya dogara da shi. Daidaitaccen ma'auni, kuma a lokaci guda, mafita mai kyau shine a kiyaye gungu na ETCD akan manyan nodes don samun ɗan jinkiri ga kube-apiserver. Idan ba za ku iya yin wannan ba, to, sanya ETCD a matsayin kusa da zai yiwu, tare da bandwidth mai kyau tsakanin mahalarta. Hakanan kula da yawan nodes daga ETCD zasu iya faɗuwa ba tare da lahani ga gungu ba
Ka tuna cewa ƙara yawan mambobi a cikin tari na iya ƙara yawan haƙuri a cikin kuɗin aiki, komai ya kamata ya kasance cikin matsakaici.
Idan muka yi magana game da kafa sabis ɗin, akwai ƴan shawarwari:
-
Samun kayan aiki masu kyau, dangane da girman gungu (zaka iya karantawa ).
-
Tweak ƴan sigogi idan kun yada tari tsakanin guda biyu na DCs ko cibiyar sadarwar ku da fayafai suna barin abin da ake so (zaka iya karantawa. ).
ƙarshe
Wannan labarin yana bayyana abubuwan da ƙungiyarmu ke ƙoƙarin yin aiki da su. Wannan ba bayanin mataki-mataki ba ne na ayyuka, amma zaɓuɓɓuka waɗanda za su iya zama masu amfani don haɓaka tari sama da ƙasa. A bayyane yake cewa kowane gungu na musamman ne ta hanyarsa, kuma hanyoyin daidaitawa na iya bambanta sosai, don haka zai zama mai ban sha'awa don samun ra'ayin ku kan yadda kuke saka idanu kan gungu na Kubernetes da yadda kuke haɓaka ayyukansa. Raba kwarewar ku a cikin sharhi, zai zama mai ban sha'awa don sanin.
source: www.habr.com