Olu luhlaziyo lwam , esebenza ngoku kwi-Kubernetes 1.14 kunye nenguqulo ye-CNI yamva nje ngo-Epreli ka-2019.
Okokuqala, ndifuna ukubulela iqela leCilium: abafana bandinceda ukuba ndihlolisise kwaye ndilungise izikripthi zokubeka iliso kwiimetriki.
Yintoni etshintshileyo ukusukela ngoNovemba ka-2018
Nantsi into etshintshileyo ukusukela ngoko (ukuba unomdla):
I-Flannel ihlala iyeyona ikhawulezayo kwaye ilula i-interface ye-CNI, kodwa ayixhasi imigaqo-nkqubo yenethiwekhi kunye noguqulelo oluntsonkothileyo.
I-Romana ayisaxhaswa, ngoko siyisusile kumda wokulinganisa.
I-WeaveNet ngoku ixhasa imigaqo-nkqubo yenethiwekhi ye-Ingress kunye ne-Egress! Kodwa imveliso ihlile.
KwiCalico, kusafuneka uqwalasele ubungakanani bepakethe enkulu (MTU) ukuze usebenze kakuhle. I-Calico inikezela ngeendlela ezimbini zokufakela i-CNI, ngoko unokwenza ngaphandle kwendawo yokugcina i-ETCD eyahlukileyo:
- ukugcina i-state kwi-Kubernetes API njengendawo yokugcina idatha (ubungakanani beqela <50 nodes);
- ukugcina imeko kwi-Kubernetes API njengedatha yedatha kunye ne-Typha proxy ukukhulula umthwalo kwi-K8S API (ubungakanani beqoqo> i-50 nodes).
I-Calico ibhengeze inkxaso phezu kwe-Istio yokhuseleko lwenqanaba lesicelo.
I-Cilium ngoku ixhasa i-encryption! I-Cilium ibonelela ngoguqulelo oluntsonkothileyo ngeetonela ze-IPSec kwaye ibonelela ngenye indlela kuthungelwano olufihliweyo lwe-WeaveNet. Kodwa i-WeaveNet ikhawuleza kune-Cilium enoguqulelo oluntsonkothileyo.
I-Cilium ngoku kulula ukuyihambisa ngokubonga kumqhubi we-ETCD owakhelwe ngaphakathi.
Iqela leCilium liye lazama ukunciphisa ubunzima kwi-CNI yayo ngokunciphisa ukusetyenziswa kwememori kunye neendleko ze-CPU, kodwa abakhuphisana nabo basalula.
Umxholo womlinganiselo
Ibhentshimakhi iqhutywa kwiiseva ezintathu zeSupermicro ezingezizo ezoqobo ezine-10 Gb Supermicro iswitshi. Iiseva ziqhagamshelwe ngokuthe ngqo kutshintshi ngokugqitha kwi-DAC SFP + iintambo kwaye ziqwalaselwe kwi-VLAN efanayo kunye nezakhelo ze-jumbo (MTU 9000).
I-Kubernetes 1.14.0 efakwe kwi-Ubuntu 18.04 LTS ene-Docker 18.09.2 (uguqulelo lwe-Docker olungagqibekanga kolu kukhutshwa).
Ukuphucula ukuveliswa kwakhona, sagqiba ekubeni sihlale siqwalasela i-master kwi-node yokuqala, sibeke inxalenye yomncedisi we-benchmark kwi-server yesibini, kunye nenxalenye yomxhasi kwisithathu. Ukwenza oku, sisebenzisa iNodeSelector kwi-Kubernetes deployments.
Siza kuchaza iziphumo zebenchmark kwesi sikali silandelayo:
Ukukhetha i-CNI yomlinganiselo
Olu luphawu lwebenchmark yeCNI kuphela kuluhlu olukwicandelo Bona amaxwebhu asemthethweni e-Kubernetes. Kwi-9 CNIs, siya kuthatha i-6 kuphela: siya kukhupha ezo zinzima ukuzifaka kunye / okanye zingasebenzi ngaphandle kokucwangciswa ngokuhambelana namaxwebhu (i-Romana, Contiv-VPP kunye neJuniperContrail / TungstenFabric).
Siza kuthelekisa ezi CNI zilandelayo:
- ICalico v3.6
- Umsele v3.6 (ngokusisiseko iFlaneli yothungelwano + iCalico njengodonga lomlilo)
- I-Cilium 1.4.2
- I-Flaneli 0.11.0
- Kube-router 0.2.5
- I-WeaveNet 2.5.1
isicwangciso
Okukhona kulula ukuyifaka i-CNI, kokukhona imbonakalo yethu yokuqala iya kuba ngcono. Zonke ii-CNIs ukusuka kwibenchmark zilula kakhulu ukuzifaka (ngomyalelo omnye okanye emibini).
Njengoko besitshilo, iiseva kunye nokutshintsha ziqwalaselwe kunye nezakhelo ze-jumbo ezinikwe amandla (sibeka i-MTU kwi-9000). Singavuya ukuba i-CNI izimisele ngokuzenzekelayo i-MTU ngokusekwe kuqwalaselo lweadaptha. Nangona kunjalo, nguCilium kunye noFlannel kuphela abalawula oku. Ezinye ii-CNIs zinezicelo kwi-GitHub zokongeza ukufunyanwa kwe-MTU okuzenzekelayo, kodwa siya kuyiqwalasela ngesandla ngokutshintsha iConfigMap yeCalico, iCanal kunye neKube-router, okanye sigqithise imo eguquguqukayo yeWeaveNet.
Yintoni ingxaki nge-MTU engalunganga? Lo mzobo ubonisa umahluko phakathi kwe-WeaveNet ene-MTU engagqibekanga kunye nezakhelo ezinkulu ezenziweyo:
I-MTU iyichaphazela njani i-throughput?
Sibonile ukuba ibaluleke kangakanani i-MTU ekusebenzeni, ngoku makhe sibone ukuba i-CNIs yethu iyimisela njani ngokuzenzekelayo:
I-CNI ibhaqa i-MTU ngokuzenzekelayo
Igrafu ibonisa ukuba kufuneka uqwalasele iMTU yeCalico, Canal, Kube-router kunye neWeaveNet ukwenzela ukusebenza kakuhle. I-Cilium kunye neFlannel bakwazi ukumisela ngokuchanekileyo i-MTU ngokwabo ngaphandle kwezicwangciso.
Khu seleko
Siza kuthelekisa ukhuseleko lwe-CNI kwiinkalo ezimbini: ukukwazi ukubethela idatha edlulisiweyo kunye nokuphunyezwa kwemigaqo-nkqubo yenethiwekhi ye-Kubernetes (esekelwe kwiimvavanyo zangempela, kungekhona amaxwebhu).
Zimbini kuphela iinkcukacha ezifihliweyo ze-CNIs: iCilium kunye neWeaveNet. Uguqulelo oluntsonkothileyo WeaveNet yenziwe ngokucwangcisa igama eligqithisiweyo loguqulelo oluntsonkothileyo njengotshintsho lwemeko-bume ye-CNI. IN I-WeaveNet iyichaza ngendlela enzima, kodwa yonke into yenziwa ngokulula. Uguqulelo oluntsonkothileyo icilium iqwalaselwe yimiyalelo, ngokwenza i Kubernetes iimfihlo, kwaye ngokuguqulwa kwe daemonSet (intsokothe ββngakumbi kuneWeaveNet, kodwa iCilium inenyathelo-ne-nyathelo. ).
Ngokuphathelele ukuphunyezwa komgaqo-nkqubo womnatha, baye baphumelela ICalico, iCanal, iCilium kunye neWeaveNet, apho unokuqwalasela imithetho ye-Ingress kunye ne-Egress. Kuba Kube-router kukho imigaqo kuphela ye-Ingress, kwaye Flannel Akukho migaqo-nkqubo yenethiwekhi kwaphela.
Nazi iziphumo zizonke:
Iziphumo zeNdlela yokuSebenza ngoKhuseleko
Imveliso
Olu phawu lubonisa umndilili wokutyhubela ubuncinci imitsi emithathu yovavanyo ngalunye. Sivavanya ukusebenza kwe-TCP kunye ne-UDP (usebenzisa iperf3), izicelo zangempela ezifana ne-HTTP (nge-Nginx kunye ne-curl) okanye i-FTP (kunye ne-vsftpd kunye ne-curl) kwaye ekugqibeleni ukusebenza kwesicelo usebenzisa i-encryption esekelwe kwi-SCP (usebenzisa umxhasi kunye ne-OpenSSH yomncedisi).
Kuzo zonke iimvavanyo, senze ibhentshi yentsimbi engenanto (umgca oluhlaza) ukuthelekisa ukusebenza kwe-CNI kunye nokusebenza kwenethiwekhi yemveli. Apha sisebenzisa isikali esifanayo, kodwa ngombala:
- Mthubi = kuhle kakhulu
- I-Orenji = ilungile
- Blue = ngoko-njalo
- Ebomvu = embi
Asiyi kuthatha ii-CNIs ezicwangcisiweyo ngokungalunganga kwaye siza kubonisa kuphela iziphumo ze-CNIs kunye ne-MTU echanekileyo. (Qaphela: I-Cilium ayibali ngokuchanekileyo i-MTU ukuba wenza uguqulelo oluntsonkothileyo, ngoko ke kuya kufuneka unciphise ngesandla i-MTU ukuya ku-8900 kuguqulelo 1.4. Uguqulelo olulandelayo, 1.5, lukwenza oku ngokuzenzekelayo.)
Nazi iziphumo:
Ukusebenza kweTCP
Zonke ii-CNIs ziqhube kakuhle kwi-benchmark ye-TCP. I-CNI enoguqulelo oluntsonkothileyo isala ngasemva kakhulu kuba uguqulelo oluntsonkothileyo luyabiza.
Ukusebenza kwe-UDP
Apha, nazo zonke ii-CNIs ziqhuba kakuhle. I-CNI enoguqulelo oluntsonkothileyo ibonise phantse isiphumo esifanayo. I-Cilium incinci emva kokhuphiswano, kodwa i-2,3% kuphela yentsimbi engenanto, ngoko akusiyo isiphumo esibi. Musa ukulibala ukuba i-Cilium kunye neFlannel kuphela izimisele i-MTU ngokuchanekileyo ngokwabo, kwaye ezi ziziphumo zabo ngaphandle koqwalaselo olongezelelweyo.
Kuthekani ngesicelo sokwenyani? Njengoko ubona, ukusebenza ngokubanzi kweHTTP kuphantsi kancinci kune-TCP. Nokuba usebenzisa iHTTP nge TCP, siqwalasele iperf3 kwibenchmark ye TCP ukunqanda isiqalo esicothayo esizakuchaphazela ibhenchmark yeHTTP. Wonke umntu wenze umsebenzi omhle apha. I-Kube-router inenzuzo ecacileyo, kodwa i-WeaveNet ayizange isebenze kakuhle: malunga ne-20% embi kunentsimbi engenanto. I-Cilium kunye ne-WeaveNet ene-encryption ibonakala ilusizi ngokwenene.
Nge-FTP, enye iprotocol esekelwe kwi-TCP, iziphumo ziyahluka. I-Flannel kunye ne-Kube-router yenza umsebenzi, kodwa i-Calico, i-Canal kunye ne-Cilium zincinci ngasemva kwaye zi malunga ne-10% zicotha kunentsimbi engenanto. I-WeaveNet isemva ukuya kuthi ga kwi-17%, kodwa iWeaveNet efihliweyo ine-40% ngaphambi kwe-Cilium efihliweyo.
Nge-SCP sinokubona kwangoko ukuba yimalini na i-encryption ye-SSH ibiza kuthi. Phantse zonke ii-CNIs ziqhuba kakuhle, kodwa i-WeaveNet isala ngasemva kwakhona. I-Cilium kunye ne-WeaveNet enoguqulelo oluntsonkothileyo kulindeleke ukuba ibe yeyona imbi kakhulu ngenxa yokufihlwa kabini (SSH + CNI).
Nantsi itheyibhile yesishwankathelo eneziphumo:
Ukusetyenziswa kobutyebi
Ngoku makhe sithelekise indlela i-CNI edla ngayo izibonelelo phantsi kwemithwalo enzima (ngexesha lokudluliselwa kwe-TCP, i-10 Gbps). Kwiimvavanyo zokusebenza sithelekisa i-CNI ngentsimbi engenanto (umgca oluhlaza). Ukusetyenziswa kobutyebi, masibonise i-Kubernetes esulungekileyo (umgca omfusa) ngaphandle kwe-CNI kwaye sibone ukuba zingaphi izixhobo ezongezelelweyo ezisetyenziswa yi-CNI.
Masiqale ngenkumbulo. Nali ixabiso eliphakathi le-RAM yeendawo (ngaphandle kwe-buffers kunye ne-cache) kwi-MB ngexesha logqithiselo.
Ukusetyenziswa kwememori
I-Flannel kunye ne-Kube-router ibonise iziphumo ezigqwesileyo - kuphela i-50 MB. I-Calico kunye neCanal nganye inama-70. I-WeaveNet idla ngokucacileyo ngaphezu kwabanye - i-130 MB, kwaye i-Cilium isebenzisa kangangoko kwi-400.
Ngoku makhe sijonge ukusetyenziswa kwexesha le-CPU. Kuyaphawuleka: umzobo awubonisi iipesenti, kodwa i-ppm, oko kukuthi, i-38 ppm "yentsimbi engenanto" yi-3,8%. Nazi iziphumo:
Ukusetyenziswa kwe-CPU
I-Calico, i-Canal, i-Flannel kunye ne-Kube-router zisebenza kakhulu kwi-CPU - kuphela i-2% ngaphezu kwe-Kubernetes ngaphandle kwe-CNI. I-WeaveNet isemva kakhulu nge-5% eyongezelelweyo, ilandelwa yi-Cilium nge-7%.
Nasi isishwankathelo sokusetyenziswa kobutyebi:
Iziphumo
Itheyibhile enazo zonke iziphumo:
Iziphumo zebenchmark ngokubanzi
isiphelo
Kwinxalenye yokugqibela ndiya kuvakalisa uluvo lwam subjective kwiziphumo. Khumbula ukuba le benchmark ivavanya kuphela i-output yoqhagamshelwano olunye kwiqela elincinane kakhulu (iindawo ezi-3). Ayisebenzisi kumaqela amakhulu (<50 nodes) okanye uxhulumaniso oluhambelanayo.
Ndincoma ukusebenzisa ezi CNIs zilandelayo ngokuxhomekeke kwimeko:
- Ngaba unayo kwiqela lakho iindawo zokuhlala ezinezibonelelo ezimbalwa (ii-GB ezininzi ze-RAM, ii-cores ezininzi) kwaye awudingi iimpawu zokhuseleko - khetha Flannel. Le yenye yezona CNIs ezibiza kakhulu. Kwaye iyahambelana neentlobo ezininzi zezakhiwo (i-amd64, ingalo, i-arm64, njl.). Ukongezelela, le yenye yezimbini (enye yiCilium) CNI enokuthi inqume ngokuzenzekelayo i-MTU, ngoko akudingeki ukuba uqwalasele nantoni na. I-Kube-router nayo ifanelekile, kodwa ayikho njengomgangatho kwaye kuya kufuneka uqwalasele ngesandla i-MTU.
- Ukuba kuyafuneka ikhowudi yenethiwekhi ukhuseleko, thatha WeaveNet. Ungalibali ukukhankanya ubungakanani be MTU ukuba usebenzisa izakhelo zejumbo, kwaye wenze uguqulelo oluntsonkothileyo ngokukhankanya igama eligqithisiweyo ngokutshintsha kwemekobume. Kodwa kungcono ukulibala malunga nokusebenza - yindleko ye-encryption.
- kuba ukusetyenziswa okuqhelekileyo Ndiyacebisa Calico. Le CNI isetyenziswa ngokubanzi kwii-Kubernetes ezahlukeneyo zokusetyenziswa kwezixhobo (Kops, Kubespray, Rancher, njl.). NjengeWeaveNet, qiniseka ukuba uqwalasela iMTU kwiConfigMap ukuba usebenzisa izakhelo ezinkulu. Sisixhobo esinemisebenzi emininzi esebenzayo malunga nokusetyenziswa kobutyebi, ukusebenza kunye nokhuseleko.
Kwaye ekugqibeleni, ndikucebisa ukuba ulandele uphuhliso icilium. Le CNI ineqela elisebenzayo kakhulu elisebenza kakhulu kwimveliso yabo (iimpawu, ukugcinwa kwemithombo, ukusebenza, ukhuseleko, ukudibanisa ...) kwaye banezicwangciso ezinomdla kakhulu.
Idayagram ebonakalayo yokhetho lweCNI
umthombo: www.habr.com