Senze idizayini yenethiwekhi yesikhungo sedatha evumela ukuphakelwa kwamaqoqo ekhompuyutha amakhulu kuneziphakeli eziyinkulungwane eziyi-100 ezinomkhawulokudonsa ophindwe kabili ophakeme ongaphezu kwepetabyte eyodwa ngomzuzwana.
Kusuka embikweni ka-Dmitry Afanasyev uzofunda ngezimiso eziyisisekelo zomklamo omusha, ukukala ama-topology, izinkinga eziphakama ngalokhu, izinketho zokuzixazulula, izici zomzila kanye nokukala imisebenzi yendiza yokudlulisela yamadivayisi wenethiwekhi yesimanje "ekuxhumene kakhulu" ama-topology anenombolo enkulu yemizila ye-ECMP. Ngaphezu kwalokho, u-Dima ukhulume kafushane mayelana nenhlangano yokuxhumana kwangaphandle, ungqimba olubonakalayo, uhlelo lwe-cabling kanye nezindlela zokwandisa umthamo.
- Sawubona wonke umuntu! Igama lami ngingu-Dmitry Afanasyev, ngingumakhi wenethiwekhi kwa-Yandex futhi ngiklama amanethiwekhi esikhungo sedatha.
Indaba yami izoba mayelana nenethiwekhi ebuyekeziwe yezikhungo zedatha ye-Yandex. Kungukuguquguquka okukhulu komklamo ebesinawo, kodwa ngesikhathi esifanayo kunezinto ezintsha. Lesi isethulo sohlolojikelele ngoba bekunolwazi oluningi obekumele lupakishwe ngesikhathi esincane. Sizoqala ngokukhetha i-topology enengqondo. Khona-ke kuzoba nokubukezwa kwendiza yokulawula kanye nezinkinga nge-scalability yendiza yedatha, ukukhetha okuzokwenzeka ezingeni lomzimba, futhi sizobheka izici ezithile zamadivayisi. Ake sithinte kancane ukuthi kwenzekani esikhungweni sedatha nge-MPLS, esikhulume ngayo esikhathini esithile esidlule.
Ngakho-ke, yini i-Yandex mayelana nemithwalo nezinsizakalo? I-Yandex iyi-hyperscaler evamile. Uma sibheka abasebenzisi, ikakhulukazi sicubungula izicelo zabasebenzisi. Futhi izinsiza ezihlukahlukene zokusakaza nokudluliswa kwedatha, ngoba futhi sinezinsizakalo zokugcina. Uma kuseduze ne-backend, khona-ke imithwalo yengqalasizinda namasevisi avela lapho, njengokugcinwa kwezinto ezisabalalisiwe, ukuphindaphinda kwedatha kanye, vele, nolayini oqhubekayo. Enye yezinhlobo eziyinhloko zomthwalo wemisebenzi i-MapReduce kanye nezinhlelo ezifanayo, ukucubungula ukusakaza, ukufunda ngomshini, njll.
Injani ingqalasizinda phezu kwakho konke lokhu okwenzekayo? Futhi, siyi-hyperscaler evamile evamile, nakuba mhlawumbe siseduze kancane nohlangothi oluncane lwe-hyperscaler lwe-spectrum. Kodwa sinazo zonke izimfanelo. Sisebenzisa ihadiwe yempahla kanye nokukala okuvundlile lapho kungenzeka khona. Sinezinsizakusebenza ezigcwele zokuhlanganisa: asisebenzi ngemishini ngayinye, ama-rack angawodwana, kodwa siwahlanganisa abe yiqoqo elikhulu lezinsiza ezishintshayo namanye amasevisi angeziwe aphathelene nokuhlela nokwabiwa, futhi sisebenza nalo lonke leli chibi.
Ngakho sinezinga elilandelayo - isistimu yokusebenza ezingeni le-computing cluster. Kubaluleke kakhulu ukuthi silawule ngokugcwele isitaki sobuchwepheshe esisisebenzisayo. Silawula izindawo zokugcina (izisingathi), inethiwekhi nesitaki sesofthiwe.
Sinezikhungo ezimbalwa ezinkulu zedatha eRussia nakwamanye amazwe. Bahlanganiswe wumgogodla osebenzisa ubuchwepheshe beMPLS. Ingqalasizinda yethu yangaphakathi cishe yakhiwe ngokuphelele ku-IPv6, kodwa njengoba sidinga ukunikeza ithrafikhi yangaphandle esafika kakhulu nge-IPv4, kufanele ngandlela thize sithumele izicelo eziza nge-IPv4 kumaseva angaphambili, futhi kancane siye ku-IPv4- I-inthanethi yangaphandle - isibonelo, ukukhomba.
Ukuphindaphinda okumbalwa kokugcina kwemiklamo yenethiwekhi yesikhungo sedatha kusebenzise ama-toplogi e-Clos anezendlalelo eziningi futhi angu-L3 kuphela. Sasuka e-L2 esikhathini esidlule futhi saphefumula. Okokugcina, ingqalasizinda yethu ihlanganisa amakhulu ezinkulungwane zezenzakalo zekhompyutha (iseva). Umkhawulo kasayizi weqoqo esikhathini esithile esidlule wawungamaseva angaba yizinkulungwane eziyi-10. Lokhu kubangelwa kakhulu ukuthi lezo zinhlelo zokusebenza zeleveli yeqoqo elifanayo, abahleli, ukwabiwa kwezinsiza, njll. zingasebenza kanjani. Njengoba inqubekelaphambili yenzekile ngasohlangothini lwesoftware yengqalasizinda, usayizi ohlosiwe manje cishe uyizinkulungwane eziyi-100 zamaseva kuqoqo elilodwa lekhompyutha, futhi Sinomsebenzi - ukukwazi ukwakha izimboni zenethiwekhi ezivumela ukuhlanganisa izinsiza okuphumelelayo kuqoqo elinjalo.
Yini esiyifunayo kunethiwekhi yesikhungo sedatha? Okokuqala, kunomkhawulokudonsa omningi oshibhile futhi osatshalaliswe ngokulinganayo. Ngoba inethiwekhi ingumgogodla esingahlanganisa ngawo izinsiza. Usayizi omusha ohlosiwe ungamaseva ayizinkulungwane eziyi-100 kuqoqo elilodwa.
Siphinde, vele, sifuna indiza yokulawula eyokwehla futhi ezinzile, ngoba engqalasizinda enkulu kangaka izinhlungu zekhanda eziningi zivela ngisho nezenzakalo nje ezingahleliwe, futhi asifuni ukuthi indiza elawulayo iphinde ilethe ubuhlungu bekhanda. Ngesikhathi esifanayo, sifuna ukunciphisa isimo kuwo. Uma isimo sisincane, kuba ngcono futhi kuzinzile yonke into isebenza, futhi kulula ukuxilonga.
Yiqiniso, sidinga okuzenzakalelayo, ngoba akunakwenzeka ukuphatha ingqalasizinda enjalo ngesandla, futhi bekungenakwenzeka isikhathi esithile. Sidinga ukwesekwa kokusebenza ngangokunokwenzeka kanye nokwesekwa kwe-CI/CD ngezinga esingahlinzekwa ngalo.
Ngosayizi abanjalo bezikhungo zedatha namaqoqo, umsebenzi wokusekela ukusetshenziswa okwandayo nokunwetshwa ngaphandle kokuphazamiseka kwesevisi usube mubi kakhulu. Uma kumaqoqo osayizi wemishini eyinkulungwane, mhlawumbe eduze kwemishini eyizinkulungwane eziyishumi, isengakhishwa njengomsebenzi owodwa - okungukuthi, sihlela ukunwetshwa kwengqalasizinda, futhi imishini eyizinkulungwane ezimbalwa yengezwa njengomsebenzi owodwa, khona-ke iqoqo elilingana nemishini eyizinkulungwane eziyikhulu aliveli ngokushesha kanjena, lakhiwa isikhathi eside. Futhi kuyafiseleka ukuthi sonke lesi sikhathi lokho osekukhishiwe, ingqalasizinda esetshenzisiwe, kufanele ibe khona.
Futhi imfuneko eyodwa ebesinayo nesiyishiyile: ukusekelwa kwe-multitenancy, okungukuthi, ukwenziwa kwezinto ezibonakalayo noma ukuhlukaniswa kwenethiwekhi. Manje asikho isidingo sokwenza lokhu ezingeni lendwangu yenethiwekhi, ngoba i-sharding iye kubasingathi, futhi lokhu kwenze ukukala kube lula kakhulu kithi. Ngenxa ye-IPv6 kanye nesikhala esikhulu samakheli, asizange kudingeke ukuthi sisebenzise amakheli ayimpinda kungqalasizinda yangaphakathi; wonke amakheli abesevele ehlukile. Futhi ngenxa yokuthi sithathele ukuhlunga nokuhlukaniswa kwenethiwekhi kubasingathi, asikho isidingo sokuthi sidale amabhizinisi enethiwekhi abonakalayo kumanethiwekhi esikhungo sedatha.
Into ebaluleke kakhulu yilokho esingakudingi. Uma eminye imisebenzi ingasuswa kunethiwekhi, lokhu kwenza ukuphila kube lula kakhulu, futhi, njengomthetho, kwandisa ukukhethwa kwemishini etholakalayo kanye nesofthiwe, okwenza ukuxilonga kube lula kakhulu.
Ngakho-ke, yini esingayidingi, yini esiye sakwazi ukuyiyeka, hhayi njalo ngenjabulo ngesikhathi kwenzeka, kodwa ngokukhululeka okukhulu lapho inqubo isiqediwe?
Okokuqala, ukushiya i-L2. Asiyidingi i-L2, engeyona eyangempela noma elingiswayo. Ayisetshenziswa kakhulu ngenxa yokuthi silawula isitaki sohlelo lokusebenza. Izicelo zethu ziyakaleka ngokuvundlile, zisebenza nekheli le-L3, abakhathazeki kakhulu ngokuthi esinye isigameko siphumile, bavele bakhiphe esisha, akudingi ukuthi sikhishwe ekhelini elidala, ngoba kukhona izinga elihlukile lokutholwa kwesevisi kanye nokuqapha kwemishini etholakala kuqoqo. Asiwuthumeli lo msebenzi kunethiwekhi. Umsebenzi wenethiwekhi ukuletha amaphakethe ukusuka endaweni A ukuya endaweni B.
Futhi asinazo izimo lapho amakheli ehamba ngaphakathi kwenethiwekhi, futhi lokhu kudinga ukugadwa. Emiklamo eminingi lokhu ngokuvamile kuyadingeka ukuze kusekelwe ukuhamba kwe-VM. Asisebenzisi ukuhamba kwemishini ebonakalayo kwingqalasizinda yangaphakathi ye-Yandex enkulu, futhi, ngaphezu kwalokho, sikholelwa ukuthi ngisho noma lokhu kwenziwa, akufanele kwenzeke ngokusekelwa kwenethiwekhi. Uma udinga ngempela ukwenza lokhu, udinga ukukwenza kuleveli yosokhaya, futhi uphushe amakheli angathuthela kuzimbondela, ukuze ungathinti noma wenze izinguquko eziguquguqukayo eziningi ohlelweni lomzila lwe-underlay ngokwayo (inethiwekhi yezokuthutha) .
Obunye ubuchwepheshe esingabusebenzisi ukusakaza okuningi. Uma ufuna, ngingakutshela ngokuningiliziwe ukuthi kungani. Lokhu kwenza ukuphila kube lula kakhulu, ngoba uma othile eke wabhekana nakho futhi wabheka kahle ukuthi indiza yokulawula i-multicast ibukeka kanjani, kukho konke kodwa ukufakwa okulula, lokhu kukhanda elikhulu. Futhi ngaphezu kwalokho, kunzima ukuthola ukuqaliswa komthombo ovulekile osebenza kahle, isibonelo.
Okokugcina, siklama amanethiwekhi ethu ukuze angashintshi kakhulu. Singathembela eqinisweni lokuthi ukugeleza kwemicimbi yangaphandle ohlelweni lomzila kuncane.
Yiziphi izinkinga eziphakamayo futhi yimiphi imikhawulo okufanele icatshangelwe lapho sakha inethiwekhi yesikhungo sedatha? Izindleko, kunjalo. I-Scalability, izinga esifuna ukukhula ngalo. Isidingo sokunwetshwa ngaphandle kokumisa isevisi. Umkhawulokudonsa, ukutholakala. Ukubonakala kokwenzekayo kunethiwekhi yezinhlelo zokuqapha, emaqenjini asebenzayo. Ukusekelwa okuzenzakalelayo - futhi, ngangokunokwenzeka, njengoba imisebenzi ehlukene ingaxazululwa emazingeni ahlukene, kuhlanganise nokwethulwa kwezingqimba ezengeziwe. Hhayi-ke, [okungenzeka] kuncike kubathengisi. Nakuba ezikhathini ezihlukene zomlando, kuye ngokuthi yisiphi isigaba osibhekayo, lokhu kuzimela bekulula noma kunzima kakhulu ukukuzuza. Uma sithatha ingxenye yesiphambano sama-chips wedivayisi yenethiwekhi, khona-ke kuze kube muva nje bekunemibandela kakhulu ukukhuluma ngokuzimela kubathengisi, uma nathi sifuna ama-chips ane- throughput ephezulu.
Iyiphi i-topology enengqondo esizoyisebenzisa ukwakha inethiwekhi yethu? Lokhu kuzoba yi-Clos enamazinga amaningi. Eqinisweni, azikho ezinye izindlela zangempela okwamanje. Futhi i-Clos topology yinhle impela, ngisho noma iqhathaniswa nama-topology athuthukile asezingeni eliphakeme kakhulu endaweni yentshisekelo yezemfundo manje, uma sinokushintsha okukhulu kwe-radix.
Inethiwekhi ye-Clos enamazinga amaningi yakhiwe kanjani futhi zibizwa ngani izakhi ezihlukene kuyo? Okokuqala, umoya wasukuma, ukuze uqonde lapho enyakatho, lapho eningizimu, lapho impumalanga, lapho kukhona intshonalanga. Amanethiwekhi alolu hlobo ngokuvamile akhiwa yilabo abanethrafikhi enkulu kakhulu yasentshonalanga-mpumalanga. Ngokuqondene nezakhi ezisele, phezulu kuneswishi ebonakalayo ehlanganiswe kusuka kumaswishi amancane. Lona umqondo oyinhloko wokwakhiwa kabusha kwamanethiwekhi we-Clos. Sithatha izakhi ezinohlobo oluthile lwe-radix futhi sizixhumanise ukuze esikutholayo kubhekwe njengokushintsha nge-radix enkulu. Uma udinga ngisho nangaphezulu, inqubo ingaphinda.
Ezimweni, isibonelo, nge-Clos enamazinga amabili, lapho kungenzeka ukukhomba ngokucacile izingxenye eziqondile kumdwebo wami, ngokuvamile zibizwa ngokuthi izindiza. Uma besingakha i-Clos enamazinga amathathu okushintshwa komgogodla (konke okungewona amaswishi emingcele noma e-ToR futhi asetshenziselwa izinto zokuhamba kuphela), izindiza bezizobukeka ziyinkimbinkimbi kakhulu; ezinamaleveli amabili abukeka kanjena. Sibiza ibhulokhi ye-ToR noma amaswishi eqabunga kanye namaswishi omgogodla weleveli yokuqala ahlotshaniswa nawo nge-Pod. Amaswishi omgogodla wezinga lomgogodla-1 phezulu kwePod angaphezulu kwePod, phezulu kwePod. Amaswishi atholakala phezulu kwefekthri yonke awungqimba olungaphezulu lwefekthri, Phezulu kwendwangu.
Impela, umbuzo ophakamayo: Amanethiwekhi e-Clos akhiwe isikhathi eside; umbono ngokwawo uvame ukuvela ezikhathini zocingo lwakudala, amanethiwekhi e-TDM. Mhlawumbe kukhona okungcono okuvelile, mhlawumbe kukhona okungenziwa kangcono? Yebo futhi cha. Ngokwethiyori yebo, ekusebenzeni esikhathini esizayo esiseduze nakanjani akunjalo. Ngenxa yokuthi kunezinombolo ze-topology ezithakazelisayo, ezinye zazo zisetshenziselwa ngisho nokukhiqiza, isibonelo, i-Dragonfly isetshenziswa ezinhlelweni ze-HPC; Kukhona futhi ama-topology athokozisayo afana ne-Xpander, FatClique, Jellyfish. Uma ubheka imibiko ezingqungqutheleni ezifana ne-SIGCOMM noma i-NSDI muva nje, ungathola inani elikhulu kakhulu lemisebenzi kwezinye izindlela zokuxoxisana ezinezakhiwo ezingcono (eyodwa noma enye) kune-Clos.
Kodwa zonke lezi topology zinempahla eyodwa ethokozisayo. Ivimbela ukusetshenziswa kwawo kumanethiwekhi esikhungo sedatha, esizama ukuwakha ku-hardware yezimpahla futhi abiza imali enengqondo. Kuwo wonke lawa ma-topology ahlukile, iningi le-bandwidth ngeshwa alifinyeleleki ngezindlela ezimfishane. Ngakho-ke, silahlekelwa ngokushesha ithuba lokusebenzisa indiza yokulawula yendabuko.
Ngokwethiyori, isixazululo senkinga siyaziwa. Lokhu, ngokwesibonelo, ukuguqulwa kwesimo sokuxhumanisa kusetshenziswa indlela engu-k-shortest, kodwa, futhi, azikho izimiso ezinjalo ezizosetshenziswa ekukhiqizeni futhi zitholakale kabanzi kumishini.
Ngaphezu kwalokho, njengoba umthamo omkhulu ungafinyeleleki ngezindlela ezimfishane, sidinga ukulungisa okungaphezu nje kwendiza yokulawula ukuze sikhethe zonke lezo zindlela (futhi ngendlela, lesi simo ngokuphawulekayo endizeni yokulawula). Kusadingeka silungise indiza yokudlulisela phambili, futhi, njengomthetho, kudingeka okungenani izici ezimbili ezengeziwe. Leli yikhono lokwenza zonke izinqumo mayelana nokudlulisa iphakethe ngesikhathi esisodwa, isibonelo, kumsingathi. Eqinisweni, lokhu umzila womthombo, ngezinye izikhathi ezincwadini ezikumanethiwekhi okuxhumana lokhu kubizwa ngokuthi izinqumo zokudlulisela zonke ngasikhathi sinye. Futhi ukuqondisa okuguquguqukayo kuwumsebenzi esiwudingayo kuma-elementi enethiwekhi, abilisa phansi, isibonelo, eqinisweni lokuthi sikhetha i-hop elandelayo ngokusekelwe olwazini mayelana nomthwalo omncane kulayini. Njengesibonelo, ezinye izinketho zingenzeka.
Ngakho, isiqondiso siyathakazelisa, kodwa, maye, asikwazi ukusisebenzisa njengamanje.
Kulungile, sizinzile ku-Clos topology enengqondo. Sizosikala kanjani? Ake sibone ukuthi isebenza kanjani nokuthi yini engenziwa.
Kunethiwekhi ye-Clos kunamapharamitha amabili ayinhloko esingawashintsha ngandlela thize futhi sithole imiphumela ethile: i-radix yezinto kanye nenani lamaleveli kunethiwekhi. Nginomdwebo wohlelo wokuthi kokubili kuthinta kanjani usayizi. Ngokufanelekile, sihlanganisa kokubili.
Kungabonakala ukuthi ububanzi bokugcina benethiwekhi ye-Clos buwumkhiqizo wawo wonke amazinga okushintshwa komgogodla we-radix yaseningizimu, zingaki izixhumanisi esinazo phansi, ukuthi amagatsha kanjani. Lena yindlela esikala ngayo usayizi wenethiwekhi.
Mayelana nomthamo, ikakhulukazi ekushintsheni kwe-ToR, kunezindlela ezimbili zokukala. Kuphakathi kokuthi singakwazi, ngenkathi sigcina i-topology evamile, sisebenzise izixhumanisi ezisheshayo, noma singangeza izindiza eziningi.
Uma ubheka inguqulo enwetshiwe yenethiwekhi ye-Clos (ekhoneni elingezansi kwesokudla) bese ubuyela kulesi sithombe ngenethiwekhi ye-Clos ngezansi...
... khona-ke lena i-topology efanayo, kodwa kulesi silayidi ibhidlika ngokuminyene futhi izindiza zefekthri zibekwe phezulu komunye nomunye. Kuyafana.
Kubukeka kanjani ukukala inethiwekhi ye-Clos ngezinombolo? Lapha ngihlinzeka ngedatha yokuthi yibuphi ububanzi bobubanzi obungatholwa inethiwekhi, yiliphi inani eliphakeme lama-racks, ukushintshwa kwe-ToR noma ukushintshwa kweqabunga, uma kungekho kuma-racks, singathola kuye ngokuthi iyiphi i-radix yokushintsha esiyisebenzisayo kumazinga womgogodla, futhi mangaki amazinga esiwasebenzisayo.
Nakhu ukuthi mangaki ama-rack esingaba nawo, mangaki amaseva kanye nokuthi konke lokhu kungadla malini ngokusekelwe ku-20 kW rack ngayinye. Ngaphambilini ngike ngasho ukuthi sihlose usayizi weqoqo cishe amaseva ayizinkulungwane eziyi-100.
Kungabonakala ukuthi kulo lonke umklamo, izinketho ezimbili nengxenye zinesithakazelo. Kukhona inketho enezingqimba ezimbili ze-spines kanye nama-switch-port angu-64, awela kancane. Bese kuba nezinketho ezifaneleke ngokuphelele ze-128-port (ene-radix 128) ukushintsha komgogodla okunamazinga amabili, noma ukushintsha nge-radix 32 enamazinga amathathu. Futhi kuzo zonke izimo, lapho kukhona ama-radix amaningi kanye nezingqimba eziningi, ungenza inethiwekhi enkulu kakhulu, kodwa uma ubheka ukusetshenziswa okulindelekile, ngokuvamile kukhona ama-gigawatts. Kuyenzeka ukufaka ikhebula, kodwa mancane amathuba okuthi sithole ugesi ongaka endaweni eyodwa. Uma ubheka izibalo kanye nedatha yomphakathi ezikhungweni zedatha, ungathola izikhungo zedatha ezimbalwa kakhulu ezinomthamo olinganiselwe ongaphezu kuka-150 MW. Amakhudlwana ngokuvamile amakhampasi esikhungo sedatha, izikhungo eziningana ezinkulu zedatha eziseduze kakhulu.
Kukhona enye ipharamitha ebalulekile. Uma ubheka ikholomu engakwesokunxele, umkhawulokudonsa osebenzisekayo ubalwe lapho. Kulula ukubona ukuthi kunethiwekhi ye-Clos ingxenye ebalulekile yamachweba isetshenziselwa ukuxhuma amaswishi kwelinye. Umkhawulokudonsa osebenzisekayo, umugqa owusizo, uyinto enganikezwa ngaphandle, kumaseva. Ngokwemvelo, ngikhuluma ngamachweba anemibandela futhi ikakhulukazi ngebhendi. Njengomthetho, izixhumanisi ezingaphakathi kwenethiwekhi ziyashesha kunezixhumanisi eziya kumaseva, kodwa iyunithi ngayinye yomkhawulokudonsa, njengoba singakwazi ukuyithumela kumishini yethu yeseva, kusekhona umkhawulokudonsa ngaphakathi kwenethiwekhi ngokwayo. Futhi uma senza amaleveli engeziwe, ziba nkulu izindleko ezithile zokuhlinzeka lo mugqa ngaphandle.
Ngaphezu kwalokho, ngisho naleli bhande elingeziwe alifani ncamashi. Nakuba izikhala zizifushane, singasebenzisa into efana ne-DAC (inamathisele ngqo ikhopha, okungukuthi, izintambo ze-twinax), noma i-multimode optics, ebiza imali engaphezulu noma ephansi enengqondo. Ngokushesha lapho sithuthela ezindaweni ezinde - njengomthetho, lawa ama-optics emodi eyodwa, futhi izindleko zalo mkhawulokudonsa owengeziwe zikhuphuka ngokuphawulekayo.
Futhi futhi, sibuyela kusilayidi sangaphambilini, uma sakha inethiwekhi ye-Clos ngaphandle kokubhalisa ngokweqile, khona-ke kulula ukubuka umdwebo, ubone ukuthi inethiwekhi yakhiwe kanjani - ukwengeza izinga ngalinye lokushintshwa komgogodla, siphinda wonke umugqa owawukuyo. phansi. Ileveli yokuhlanganisa - kanye nebhendi efanayo, inombolo efanayo yezimbobo ekushintsheni njengoba bezikhona ezingeni langaphambilini, kanye nenani elifanayo lama-transceiver. Ngakho-ke, kufiseleka kakhulu ukunciphisa inani lamazinga okushintshwa komgogodla.
Ngokusekelwe kulesi sithombe, kuyacaca ukuthi sifuna ngempela ukwakha phezu kokuthile okufana nokushintsha nge-radix engu-128.
Lapha, ngokomthetho, yonke into iyafana nalokhu engisanda kukusho; lesi isilayidi esizocatshangelwa kamuva.
Yiziphi izinketho esingazikhetha njengamaswishi anjalo? Kuyizindaba ezimnandi kakhulu kithina ukuthi manje amanethiwekhi anjalo angagcina akhelwe ekushintsheni kwe-single-chip. Futhi lokhu kuhle kakhulu, banezici eziningi ezinhle. Ngokwesibonelo, cishe abanalo isakhiwo sangaphakathi. Lokhu kusho ukuthi baphuka kalula. Bagqekeza ngazo zonke izinhlobo zezindlela, kodwa ngenhlanhla baphuka ngokuphelele. Kumadivayisi we-modular kukhona inani elikhulu lamaphutha (okungathandeki kakhulu), lapho kusukela ekubukeni komakhelwane kanye nendiza yokulawula kubonakala sengathi isebenza, kodwa, isibonelo, ingxenye yendwangu ilahlekile futhi ayisebenzi. ngomthamo ogcwele. Futhi ithrafikhi eya kuyo ilinganiselwe ngokusekelwe eqinisweni lokuthi isebenza ngokugcwele, futhi singagcwala ngokweqile.
Noma, isibonelo, izinkinga ziphakama nge-backplane, ngoba ngaphakathi kwedivayisi ye-modular kukhona ne-SerDes esheshayo - iyinkimbinkimbi ngempela ngaphakathi. Kuphakathi kokuthi izimpawu phakathi kwama-elementi okudlulisela phambili ziyavumelaniswa noma azivumelaniswa. Ngokuvamile, noma iyiphi idivayisi ye-modular ekhiqizayo ehlanganisa inani elikhulu lezinto, njengomthetho, iqukethe inethiwekhi ye-Clos efanayo ngaphakathi kwayo, kodwa kunzima kakhulu ukuyixilonga. Ngokuvamile kuba nzima nakumthengisi ngokwakhe ukuxilonga.
Futhi inenani elikhulu lezimo zokwehluleka lapho idivayisi yehlisa isithunzi, kodwa ayiphumi ngokuphelele ku-topology. Njengoba inethiwekhi yethu inkulu, ukulinganisa phakathi kwezakhi ezifanayo kusetshenziswa ngenkuthalo, inethiwekhi ijwayelekile kakhulu, okungukuthi, indlela eyodwa lapho yonke into ihlelekile ayihlukile kwenye indlela, kunenzuzo enkulu ngathi ukumane silahlekelwe ezinye zazo. amadivaysi avela ku-topology kunokuba agcine esesimweni lapho amanye awo abonakala esebenza, kodwa amanye awo awasebenzi.
Isici esihle esilandelayo samadivayisi e-single-chip ukuthi avela kangcono futhi ngokushesha. Futhi bavame ukuba namandla angcono. Uma sithatha izakhiwo ezinkulu ezihlanganisiwe esinazo kumbuthano, khona-ke umthamo weyunithi ye-rack ngayinye yamachweba wejubane elifanayo cishe uphindwe kabili kunawamadivayisi we-modular. Amadivayisi akhelwe eduze kwe-chip eyodwa ashibhe ngokuphawulekayo kunamamojula futhi asebenzisa amandla amancane.
Kodwa, yiqiniso, konke lokhu kungenxa yesizathu, kukhona futhi nebubi. Okokuqala, i-radix ihlale incane kunaleyo yamadivayisi asetshenziswayo. Uma singathola idivayisi eyakhelwe eduze kwe-chip eyodwa enezimbobo ezingu-128, khona-ke singathola i-modular enamachweba angamakhulu ambalwa manje ngaphandle kwezinkinga.
Lona usayizi omncane ngokuphawulekayo wamathebula okudlulisela phambili futhi, njengomthetho, yonke into ehlobene nokukaleka kwendiza yedatha. Amabhafa angajulile. Futhi, njengomthetho, ukusebenza okulinganiselwe kunalokho. Kodwa kuvela ukuthi uma uyayazi le mikhawulo futhi unakekele ngesikhathi ukuze uyivimbe noma umane uyicabangele, khona-ke lokhu akwethusi kangako. Iqiniso lokuthi i-radix incane akuseyona inkinga kumadivayisi ane-radix ye-128 evele yavela kamuva nje; singakha ngezingqimba ezimbili zomgogodla. Kodwa namanje akunakwenzeka ukwakha noma yini encane kunezimbili ezisithakaselayo. Ngezinga elilodwa, amaqoqo amancane kakhulu atholakala. Ngisho nemiklamo yethu yangaphambili kanye nezidingo zazisedlula kuzo.
Eqinisweni, uma ngokuzumayo isisombululo sisendaweni ethile onqenqemeni, kusekhona indlela yokukala. Njengoba izinga lokugcina (noma elokuqala), eliphansi kakhulu lapho amaseva axhumeke khona amaswishi e-ToR noma amaswishi amaqabunga, akudingekile ukuthi sixhume irack eyodwa kuwo. Ngakho-ke, uma isisombululo siwela cishe ngesigamu, ungacabanga ngokumane usebenzise inkinobho ene-radix enkulu ezingeni eliphansi nokuxhuma, isibonelo, ama-rack amabili noma amathathu ku-switch eyodwa. Lokhu futhi kuyinketho, inezindleko zayo, kodwa isebenza kahle futhi ingaba yisixazululo esihle uma udinga ukufinyelela cishe kabili ubukhulu.
Ukufingqa, sakhela phezu kwe-topology enamazinga amabili omgogodla, anezendlalelo zasembonini eziyisishiyagalombili.
Kuzokwenzekani ngefiziksi? Izibalo ezilula kakhulu. Uma sinamazinga amabili omgogodla, khona-ke sinamazinga amathathu kuphela okushintsha, futhi silindele ukuthi kuzoba nezigaba ezintathu zekhebula kunethiwekhi: kusuka kumaseva kuya ekushintsheni kwamaqabunga, kuya kumgogodla 1, kuya kumgogodla 2. Izinketho esingakwazi ukusetshenziswa - lezi twinax, i-multimode, imodi eyodwa. Futhi lapha sidinga ukucabangela ukuthi imuphi umucu otholakalayo, uzobiza malini, ukuthi ubukhulu bendawo buyini, yiziphi izikhala esingazimboza, nokuthi sizozithuthukisa kanjani.
Ngokwezindleko, yonke into ingahlelwa. Ama-Twinaks ashibhe kakhulu kunama-optics asebenzayo, ashibhile kunama-transceivers e-multimode, uma uwathatha ngendiza ngayinye kusukela ekugcineni, ashibhe kancane kunembobo yokushintsha engu-100-gigabit. Futhi, sicela uqaphele, kubiza ngaphansi kwe-optics yemodi eyodwa, ngoba ezindizeni lapho kudingeka khona imodi eyodwa, ezikhungweni zedatha ngenxa yezizathu eziningi kunengqondo ukusebenzisa i-CWDM, kuyilapho i-parallel single mode (PSM) ingasebenzi kahle. nge, amaphakethe amakhulu kakhulu atholakala imicu, futhi uma sigxila kulobu buchwepheshe, sithola cishe ukulandelana kwentengo okulandelayo.
Enye inothi: ngeshwa, akunakwenzeka ukusebenzisa amachweba we-multimode ayi-100 kuya ku-4x25. Ngenxa yezici zokuklama ze-SFP28 transceivers, ayishibhile kakhulu kune-28 Gbit QSFP100. Futhi le disassembly ye-multimode ayisebenzi kahle kakhulu.
Omunye umkhawulo ukuthi ngenxa yobukhulu bamaqembu ekhompuyutha kanye nenani lamaseva, izikhungo zethu zedatha ziba zikhulu ngokomzimba. Lokhu kusho ukuthi okungenani indiza eyodwa kuzodingeka yenziwe nge-singlemod. Futhi, ngenxa yobukhulu obubonakalayo be-Pods, ngeke kwenzeke ukusebenzisa izikhala ezimbili ze-twinax (izintambo zethusi).
Ngenxa yalokho, uma silungiselela inani futhi sicabangela i-geometry yalo mklamo, sithola i-twinax eyodwa, i-multimode eyodwa kanye ne-singlemode sisebenzisa i-CWDM. Lokhu kucabangela izindlela zokuthuthukisa okungenzeka.
Yilokhu okubukeka muva nje, lapho siphokophele khona nokuthi kungenzekani. Kuyacaca, okungenani, ukuthi ungadlulela kanjani ku-50-Gigabit SerDes yakho kokubili i-multimode ne-singlemode. Ngaphezu kwalokho, uma ubheka okuku-transceivers wemodi eyodwa manje nasesikhathini esizayo ku-400G, ngokuvamile ngisho nalapho i-50G SerDes ifika ohlangothini lukagesi, i-100 Gbps ngomzila ngamunye ingakwazi kakade ukuya kuma-optics. Ngakho-ke, kungenzeka ukuthi esikhundleni sokuthuthela ku-50, kuzoba khona ukushintshela ku-100 Gigabit SerDes kanye ne-100 Gbps ngomzila ngamunye, ngoba ngokwezithembiso zabathengisi abaningi, ukutholakala kwabo kulindeleke maduze nje. Isikhathi lapho i-50G SerDes yayishesha kakhulu, kubonakala sengathi, ngeke ibe yinde kakhulu, ngoba amakhophi okuqala e-100G SerDes aphuma cishe ngonyaka ozayo. Futhi ngemva kwesikhathi esithile ngemva kwalokho cishe bayobe bebiza imali enengqondo.
Enye i-nuance eyengeziwe mayelana nokukhetha kwe-physics. Empeleni, sesingavele sisebenzisa izimbobo ze-Gigabit ezingama-400 noma ezingama-200 sisebenzisa i-50G SerDes. Kodwa kuvela ukuthi lokhu akunangqondo, ngoba, njengoba ngishilo ekuqaleni, sifuna i-radix enkulu kakhulu ekushintsheni, ngaphakathi kwesizathu, kunjalo. Sifuna i-128. Futhi uma sinomkhawulo we-chip umthamo futhi sikhulisa isivinini sokuxhumanisa, khona-ke i-radix iyancipha ngokwemvelo, azikho izimangaliso.
Futhi singakwazi ukwandisa umthamo sisebenzisa izindiza, futhi azikho izindleko ezikhethekile; singakwazi ukwengeza inani lezindiza. Futhi uma silahlekelwa i-radix, kuzodingeka sethule izinga elingeziwe, ngakho-ke esimweni samanje, ngomthamo wamanje otholakalayo ophezulu we-chip ngayinye, kuvela ukuthi kuphumelela kakhulu ukusebenzisa amachweba angu-100-gigabit, ngoba akuvumela ukuthola i-radix enkulu.
Umbuzo olandelayo ukuthi i-physics ihlelwe kanjani, kodwa kusukela ekubukeni kwengqalasizinda yekhebula. Kuvele ukuthi ihlelwe ngendlela ehlekisayo. Ikhebula phakathi kwama-leaf-switches kanye ne-spines yezinga lokuqala - azikho izixhumanisi eziningi lapho, yonke into yakhiwe kalula. Kodwa uma sithatha indiza eyodwa, okwenzekayo ngaphakathi ukuthi sidinga ukuxhuma yonke imigqa yezinga lokuqala nayo yonke imigqa yezinga lesibili.
Futhi, njengomthetho, kunezifiso ezithile zokuthi kufanele ibukeke kanjani ngaphakathi kwesikhungo sedatha. Isibonelo, besifuna ngempela ukuhlanganisa amakhebuli abe yinqwaba futhi siwadonse ukuze iphaneli yepeshi eyodwa enobukhulu obukhulu ingene ngokuphelele kuphaneli yesichibi esisodwa, ukuze kungabikho i-zoo ngokobude. Sikwazile ukuxazulula le nkinga. Uma ekuqaleni ubheka i-topology enengqondo, ungabona ukuthi izindiza zizimele, indiza ngayinye ingakhiwa yodwa. Kodwa uma sengeza inqwaba enjalo futhi sifuna ukuhudula yonke iphaneli yesichibi sibe yiphaneli yesichibi, kufanele sihlanganise izindiza ezihlukene ngaphakathi kwenqwaba eyodwa bese sethula isakhiwo esimaphakathi ngendlela yokuxhuma okuphambanayo ukuze siwapakishe kabusha ngendlela ahlanganiswe ngayo. engxenyeni eyodwa, endleleni azoqoqwa ngayo kwenye ingxenye. Ngenxa yalokhu, sithola isici esihle: konke ukushintsha okuyinkimbinkimbi akudluli kuma-racks. Uma udinga ukuhlanganisa into enamandla kakhulu, "vula izindiza," njengoba ngezinye izikhathi ibizwa ngokuthi amanethiwekhi we-Clos, konke kugxilwe ngaphakathi kwe-rack eyodwa. Asinawo ahlakazeke kakhulu, phansi kwezixhumanisi ngazinye, ukushintsha phakathi kwama-rack.
Lena yindlela ebukeka ngayo kusukela ekubukeni kwenhlangano enengqondo yengqalasizinda yekhebula. Esithombeni esingakwesokunxele, amabhulokhi anemibala eminingi abonisa amabhulokhi okushintshwa komgogodla wezinga lokuqala, izingcezu eziyisishiyagalombili lilinye, nezinqwaba zezintambo ezine ezivela kuwo, ezihamba futhi ziphambana nezinqwaba ezivela emabhuloki amaswishi e-spine-2. .
Izikwele ezincane zibonisa ukuphambana kwemigwaqo. Phezulu kwesokunxele kunoqhekeko lwempambano-mgwaqo ngayinye, lena empeleni imojula yokuxhuma embobeni engu-512 by 512 epakisha kabusha izintambo ukuze zingene ngokuphelele endaweni eyodwa, lapho kukhona indiza eyodwa yomgogodla-2. Futhi kwesokudla, ukuskena kwalesi sithombe kunemininingwane ethe xaxa ngokuhlobene nama-Pods amaningana ezingeni lomgogodla-1, nokuthi ihlanganiswe kanjani ekuxhumaneni okuphambene, ukuthi ifika kanjani ezingeni lomgogodla-2.
Lokhu kubukeka kanjani. Isitendi esingakahlanganiswa ngokugcwele somgogodla-2 (ngakwesokunxele) kanye nesitendi sokuxhumanisa. Ngeshwa, akukuningi ongakubona lapho. Sonke lesi sakhiwo sisatshalaliswa khona manje kwesinye sezikhungo zethu ezinkulu zedatha esinwetshwayo. Lona umsebenzi oqhubekayo, uzobukeka kahle, uzogcwaliswa kangcono.
Umbuzo obalulekile: sikhethe i-topology enengqondo futhi sakha i-physics. Kuzokwenzekani endizeni yokulawula? Kuyaziwa kakhulu ngesipiliyoni sokusebenza, kunemibiko eminingana exhumanisa izivumelwano zombuso zilungile, kuyintokozo ukusebenza nazo, kodwa, ngeshwa, azifiki kahle ku-topology exhumeke kakhulu. Futhi kunesici esisodwa esiyinhloko esivimbela lokhu - lena indlela izikhukhula ezisebenza ngayo kumaphrothokholi wesimo sesixhumanisi. Uma nje uthatha i-algorithm yezikhukhula bese ubheka ukuthi inethiwekhi yethu yakhiwe kanjani, ungabona ukuthi kuzoba ne- fanout enkulu esinyathelweni ngasinye, futhi izovele igcwale indiza elawulayo ngezibuyekezo. Ikakhulukazi, ama-topology anjalo ahlangana kabi kakhulu ne-algorithm evamile yezikhukhula kumaphrothokholi esifunda sesixhumanisi.
Inketho ukusebenzisa i-BGP. Indlela yokuyilungiselela kahle ichazwe ku-RFC 7938 mayelana nokusetshenziswa kwe-BGP ezikhungweni ezinkulu zedatha. Imibono eyisisekelo ilula: inani elincane leziqalo ngosokhaya ngamunye kanye nenani elincane leziqalo kunethiwekhi, sebenzisa ukuhlanganisa uma kungenzeka, futhi cindezela ukuzingela kwendlela. Sifuna ukusabalalisa okucophelela kakhulu, okulawulwa kakhulu kwezibuyekezo, lokho okubizwa ngokuthi i-valley free. Sifuna ukuthi izibuyekezo zisetshenziswe kanye njengoba zidlula kunethiwekhi. Uma zisuka ezansi, ziyakhuphuka, zingazembuki ngaphezu kokukodwa. Akufanele kube nama-zigzag. Ama-Zigzag mabi kakhulu.
Ukwenza lokhu, sisebenzisa umklamo olula ngokwanele ukusebenzisa izindlela ezingaphansi ze-BGP. Okusho ukuthi, sisebenzisa i-eBGP esebenza kusixhumanisi sendawo, futhi amasistimu azimele anikezwa kanje: isistimu ezimele ku-ToR, uhlelo oluzimele kuwo wonke ama-switch switch of spine-1 wePod eyodwa, kanye nohlelo olujwayelekile oluzimele Phezulu lonke. Yendwangu. Akunzima ukubheka futhi ubone ukuthi ngisho nokuziphatha okuvamile kwe-BGP kusinika ukusatshalaliswa kwezibuyekezo esizifunayo.
Ngokwemvelo, ikheli kanye nokuhlanganisa ikheli kufanele kuklanywe ngendlela yokuthi kuhambisane nendlela okwakhiwa ngayo umzila, ukuze kuqinisekiswe ukuzinza kwendiza elawulayo. Ukukhuluma nge-L3 kwezokuthutha kuhambisana ne-topology, ngoba ngaphandle kwalokhu akunakwenzeka ukuzuza ukuhlanganisa; ngaphandle kwalokhu, amakheli angawodwana azongena ohlelweni lomzila. Futhi enye into ukuthi ukuhlanganisa, ngeshwa, akuhlanganisi kahle ne-multi-path, ngoba uma sine-multi-path futhi sine-aggregation, konke kuhamba kahle, lapho inethiwekhi yonke iphilile, akukho ukwehluleka kuyo. Ngeshwa, ngokushesha nje lapho ukwehluleka kubonakala kunethiwekhi futhi ukulinganisa kwe-topology kulahleka, singafika lapho iyunithi yamenyezelwa khona, lapho singakwazi ukuqhubekela phambili lapho sidinga ukuya khona. Ngakho-ke, kungcono ukuhlanganisa lapho ingekho enye indlela enezindlela eziningi, kithina lezi zishintshi ze-ToR.
Eqinisweni, kungenzeka ukuhlanganisa, kodwa ngokucophelela. Uma singenza ukuhlukanisa okulawulwayo lapho kwenzeka ukwehluleka kwenethiwekhi. Kodwa lona kuwumsebenzi onzima impela, saze sazibuza ukuthi kungenzeka yini ukwenza lokhu, ukuthi kungenzeka yini ukwengeza i-automation eyengeziwe, nemishini yesimo esilinganiselwe engakhahlela kahle i-BGP ukuze uthole ukuziphatha okufunayo. Ngeshwa, ukucubungula amacala ekhona akubonakali futhi kuyinkimbinkimbi, futhi lo msebenzi awuxazululwa kahle ngokunamathisela okunamathiselwe kwangaphandle ku-BGP.
Umsebenzi othakazelisa kakhulu mayelana nalokhu wenziwe ngaphakathi kohlaka lwephrothokholi ye-RIFT, okuzoxoxwa ngayo embikweni olandelayo.
Enye into ebalulekile ukuthi izindiza zedatha zikala kanjani kuma-topology aminyene, lapho sinenombolo enkulu yezinye izindlela. Kulokhu, kusetshenziswa izinhlaka ezimbalwa zedatha ezengeziwe: Amaqembu e-ECMP, wona achaza amaqembu e-Next Hop.
Kunethiwekhi evamile yokusebenza, ngaphandle kokwehluleka, lapho sikhuphuka i-topology ye-Clos, kwanele ukusebenzisa iqembu elilodwa kuphela, ngoba yonke into engekho yendawo ichazwa ngokuzenzakalelayo, singakhuphuka. Uma sisuka phezulu siye phansi siya eningizimu, khona-ke zonke izindlela akuzona i-ECMP, ziyizindlela zomzila owodwa. Konke kuhamba kahle. Inkinga iwukuthi, futhi okungavamile kwe-Clos topology yakudala ukuthi uma sibheka Phezulu kwendwangu, kunoma iyiphi into, kunendlela eyodwa kuphela eya kunoma iyiphi into engezansi. Uma kwenzeka ukwehluleka kule ndlela, le nto ethile engaphezulu kwefekthri iba engavumelekile ngokunembile kulezo ziqalo ezingemuva kwendlela ephukile. Kodwa kokunye kuyasebenza, futhi kufanele sihlukanise amaqembu e-ECMP futhi sethule izwe elisha.
Ngabe ukukala kwendiza yedatha kubukeka kanjani kumadivayisi wesimanje? Uma senza i-LPM (ukufana kwesiqalo eside kunazo zonke), yonke into ihamba kahle, ngaphezu kweziqalo ezingu-100k. Uma sikhuluma ngamaqembu e-Next Hop, khona-ke konke kubi kakhulu, izinkulungwane ezingama-2-4. Uma sikhuluma ngetafula eliqukethe incazelo ye-Next Hops (noma i-adjacencies), khona-ke lokhu kuyindawo ethile kusuka ku-16k kuya ku-64k. Futhi lokhu kungaba inkinga. Futhi lapha sifika ekuphambukeni okuthakazelisayo: kwenzekeni ku-MPLS ezikhungweni zedatha? Empeleni, sasifuna ukukwenza.
Zimbili izinto ezenzekayo. Senze i-micro-segmentation kubasingathi; sasingasadingi ukukwenza kunethiwekhi. Bekungekuhle kakhulu ngokusekelwa okuvela kubathengisi abahlukene, futhi nangaphezulu ngokusetshenziswa okuvulekile kumabhokisi amhlophe ane-MPLS. Futhi i-MPLS, okungenani ukuqaliswa kwayo kwendabuko, ngeshwa, ihlangana kabi kakhulu ne-ECMP. Futhi yingakho.
Yilokhu kubukeka kwesakhiwo sokudlulisela i-ECMP se-IP. Inani elikhulu leziqalo lingasebenzisa iqembu elifanayo kanye nebhulokhi ye-Next Hops efanayo (noma ama-adjacencies, lokhu kungase kubizwe ngokuhlukile emibhalweni ehlukene yamadivayisi ahlukene). Iphuzu liwukuthi lokhu kuchazwa njengechweba eliphumayo nokuthi yini okufanele ubhale kuyo kabusha ikheli le-MAC ukuze ufinyelele ku-Next Hop efanele. Ku-IP yonke into ibonakala ilula, ungasebenzisa inani elikhulu kakhulu leziqalo zeqembu elifanayo, ibhulokhi efanayo ye-Next Hops.
Isakhiwo sakudala se-MPLS sisikisela ukuthi, kuye ngesixhumi esibonakalayo esiphumayo, ilebula ingabhalwa kabusha ibe ngamavelu ahlukene. Ngakho-ke, sidinga ukugcina iqembu kanye nebhulokhi ye-Next Hops kulebula ngayinye yokufaka. Futhi lokhu, maye, akukalisi.
Kulula ukubona ukuthi ekwakhiweni kwethu besidinga amaswishi e-ToR angaba ngu-4000, ububanzi obukhulu bebuyizindlela ezingama-64 ze-ECMP, uma sisuka emgogodleni-1 sibheke kumgogodla-2. Asikwazi ukungena etafuleni elilodwa lamaqembu e-ECMP, uma kuphela isiqalo esisodwa esine-ToR sihamba, futhi asingeni nhlobo kuthebula le-Next Hops.
Akukhona konke okungenathemba, ngoba izakhiwo ezifana ne-Segment Routing zibandakanya amalebula omhlaba. Ngokusemthethweni, bekungenzeka ukugoqa wonke lawa mabhulokhi e-Next Hops futhi. Ukuze wenze lokhu, udinga umsebenzi wohlobo lwekhadi lasendle: thatha ilebula bese uyibhala kabusha kulelo elifanayo ngaphandle kwenani elithile. Kodwa ngeshwa, lokhu akukhona kakhulu ekusetshenzisweni okutholakalayo.
Futhi ekugcineni, sidinga ukuletha ithrafikhi yangaphandle esikhungweni sedatha. Kwenziwa kanjani? Ngaphambilini, ithrafikhi yethulwa kunethiwekhi ye-Clos kusuka phezulu. Okusho ukuthi, bekunama-router asemaphethelweni axhumeke kuwo wonke amadivayisi Phezulu kwendwangu. Lesi sixazululo sisebenza kahle kumasayizi amancane naphakathi. Ngeshwa, ukuze sithumele i-traffic symmetrically kunethiwekhi yonke ngale ndlela, sidinga ukufika kanyekanye kuzo zonke izakhi ze-Top of fabric, futhi uma kunezingaphezu kwekhulu zazo, kuvela ukuthi sidinga futhi enkulu. i-radix kuma-odge routers. Ngokuvamile, lokhu kubiza imali, ngoba ama-router onqenqemeni asebenza kakhulu, amachweba kuwo azobiza kakhulu, futhi umklamo awuhle kakhulu.
Enye inketho ukuqala ithrafikhi enjalo kusuka ngezansi. Kulula ukuqinisekisa ukuthi i-Clos topology yakhiwe ngendlela yokuthi ithrafikhi evela ngezansi, okungukuthi, ohlangothini lwe-ToR, isatshalaliswe ngokulinganayo phakathi kwamazinga kuyo yonke i-Top yendwangu ngokuphindaphindiwe okubili, ilayisha yonke inethiwekhi. Ngakho-ke, sethula uhlobo olukhethekile lwe-Pod, i-Edge Pod, ehlinzeka ngokuxhuma kwangaphandle.
Kukhona enye inketho. Yilokhu okwenziwa yi-Facebook, ngokwesibonelo. Bayibiza nge-Fabric Aggregator noma i-HGRID. Ileveli yomgogodla eyengeziwe iyethula ukuze kuxhunywe izikhungo zedatha eziningi. Lo mklamo ungenzeka uma singenayo imisebenzi eyengeziwe noma izinguquko ze-encapsulation ezindaweni zokusebenzelana. Uma kungamaphoyinti okuthinta angeziwe, kunzima. Ngokuvamile, kunemisebenzi eminingi kanye nohlobo lolwelwesi oluhlukanisa izingxenye ezihlukene zesikhungo sedatha. Asikho isidingo sokwenza ulwelwesi olunjalo lube lukhulu, kodwa uma ludingeka ngempela ngesizathu esithile, khona-ke kunengqondo ukucabangela ukuthi kungenzeka yini ukuyisusa, ukuyenza ibe banzi ngangokunokwenzeka futhi uyidlulisele kubabungazi. Lokhu kwenziwa, isibonelo, ngama-opharetha amafu amaningi. Zinembondela, ziqala kubabungazi.
Yimaphi amathuba entuthuko esiwabonayo? Okokuqala, ukuthuthukisa ukwesekwa kwepayipi le-CI/CD. Sifuna ukundiza ngendlela esivivinya ngayo futhi sihlole indlela esindiza ngayo. Lokhu akusebenzi kahle kakhulu, ngoba ingqalasizinda inkulu futhi akunakwenzeka ukuyiphindaphinda ukuze kuhlolwe. Udinga ukuqonda ukuthi ungethula kanjani izinto zokuhlola kungqalasizinda yokukhiqiza ngaphandle kokuyiyeka.
Izinsimbi ezingcono nokuqapha okungcono cishe akulokothi kudlulele. Umbuzo wonke uwukulinganisa komzamo kanye nembuyiselo. Uma ungakwazi ukuyengeza ngomzamo ophusile, kuhle kakhulu.
Vula amasistimu okusebenza wamadivayisi enethiwekhi. Amaphrothokholi angcono nezinhlelo zomzila ezingcono, njenge-RIFT. Ucwaningo luyadingeka futhi mayelana nokusetshenziswa kwezikimu ezingcono zokulawula ukuminyana futhi mhlawumbe nokwethulwa, okungenani ezindaweni ezithile, kokusekelwa kwe-RDMA ngaphakathi kweqoqo.
Uma sibheka phambili esikhathini esizayo, sidinga ama-topology athuthukisiwe kanye namanethiwekhi okungenzeka asebenzisa phezulu kancane. Ezintweni ezintsha, kusanda kushicilelwa mayelana nobuchwepheshe bendwangu ye-HPC Cray Slingshot, esekelwe ku-Ethernet yempahla, kodwa ngenketho yokusebenzisa izihloko ezimfushane kakhulu. Ngenxa yalokho, i-overhead iyancipha.
Konke kufanele kugcinwe kulula ngangokunokwenzeka, kodwa kungabi lula. Ubunkimbinkimbi yisitha sokuqina. Ubulula kanye nezakhiwo ezijwayelekile zingabangane bethu. Uma ungakwazi ukukala endaweni ethile, kwenze. Futhi ngokujwayelekile, kuhle ukuzibandakanya kubuchwepheshe benethiwekhi manje. Ziningi izinto ezithokozisayo ezenzekayo. Ngiyabonga.
Source: www.habr.com