Peb tau tsim cov ntaub ntawv chaw network tsim uas tso cai rau kev xa tawm ntawm kev suav cov pawg loj dua 100 txhiab servers nrog lub ncov bisection bandwidth ntawm ntau dua ib petabyte ib ob.
Los ntawm Dmitry Afanasyev tsab ntawv ceeb toom koj yuav kawm txog cov hauv paus ntsiab lus ntawm kev tsim tshiab, scaling topologies, cov teeb meem uas tshwm sim nrog rau qhov no, cov kev xaiv rau kev daws lawv, cov yam ntxwv ntawm routing thiab scaling lub forwarding dav hlau ua hauj lwm ntawm niaj hnub network pab kiag li lawm nyob rau hauv "densely txuas" topologies nrog ntau txoj hauv kev ECMP. Tsis tas li ntawd, Dima tau hais luv luv txog lub koom haum ntawm kev sib txuas sab nraud, txheej txheej ntawm lub cev, cov kab ke thiab txoj hauv kev los ua kom muaj peev xwm ntxiv.
- Nyob zoo tav su sawv daws! Kuv lub npe yog Dmitry Afanasyev, Kuv yog tus kws tsim qauv network ntawm Yandex thiab feem ntau tsim cov ntaub ntawv chaw network.
Kuv zaj dab neeg yuav yog hais txog lub network tshiab ntawm Yandex cov chaw zov me nyuam. Nws yog ib qho evolution ntau heev ntawm tus tsim peb muaj, tab sis tib lub sij hawm muaj qee yam tshiab. Qhov no yog qhov kev nthuav qhia dav dav vim tias muaj ntau cov ntaub ntawv yuav tsum tau ntim rau hauv me me ntawm lub sijhawm. Peb mam li pib los ntawm kev xaiv lub ntsiab lus topology. Tom qab ntawd yuav muaj kev nthuav dav ntawm lub dav hlau tswj thiab teeb meem nrog cov ntaub ntawv dav hlau scalability, kev xaiv ntawm qhov yuav tshwm sim ntawm lub cev, thiab peb yuav saib qee yam ntawm cov khoom siv. Cia peb kov me ntsis ntawm qhov tshwm sim hauv ib lub chaw ntaub ntawv nrog MPLS, uas peb tau tham txog qee lub sijhawm dhau los.
Yog li, Yandex yog dab tsi ntawm kev thauj khoom thiab kev pabcuam? Yandex yog ib tug raug hyperscaler. Yog tias peb saib cov neeg siv, peb feem ntau ua cov neeg siv thov. Kuj muaj ntau yam kev pabcuam streaming thiab cov ntaub ntawv hloov chaw, vim peb kuj muaj kev pabcuam cia. Yog tias ze rau ntawm qhov backend, ces cov khoom siv thauj khoom thiab cov kev pabcuam tshwm nyob rau ntawd, xws li cov khoom siv faib khoom, cov ntaub ntawv rov ua dua thiab, tau kawg, cov kab txuas ntxiv. Ib yam ntawm cov haujlwm tseem ceeb yog MapReduce thiab cov tshuab zoo sib xws, kev ua haujlwm kwj, kev kawm tshuab, thiab lwm yam.
Yuav ua li cas yog lub infrastructure nyob rau saum toj ntawm tag nrho cov no tshwm sim? Ntxiv dua thiab, peb yog ib tus zoo nkauj hyperscaler, txawm hais tias peb nyob nraum tej zaum me ntsis ze rau qhov tsawg dua hyperscaler sab ntawm lub spectrum. Tab sis peb muaj tag nrho cov cwj pwm. Peb siv cov khoom siv kho vajtse thiab kab rov tav ntsuas qhov twg los tau. Peb muaj cov peev txheej tag nrho: peb tsis ua haujlwm nrog cov tshuab ib leeg, ib tus neeg racks, tab sis muab lawv tso rau hauv lub pas dej loj ntawm kev sib pauv hloov tau nrog qee cov kev pabcuam ntxiv uas cuam tshuam nrog kev npaj thiab faib, thiab ua haujlwm nrog tag nrho lub pas dej.
Yog li peb muaj qib tom ntej - kev khiav hauj lwm ntawm lub khoos phis tawj qib. Nws yog ib qho tseem ceeb heev uas peb tswj tag nrho cov txheej txheem thev naus laus zis uas peb siv. Peb tswj cov ntsiab lus kawg (tus tswv), network thiab software pawg.
Peb muaj ob peb lub chaw loj cov ntaub ntawv nyob rau hauv Russia thiab txawv teb chaws. Lawv koom ua ke los ntawm lub nraub qaum uas siv MPLS thev naus laus zis. Peb cov txheej txheem sab hauv yuav luag tag nrho ua rau IPv6, tab sis txij li peb yuav tsum tau ua haujlwm sab nraud uas tseem tuaj yeem dhau IPv4, peb yuav tsum xa cov lus thov tuaj dhau IPv4 rau cov servers pem hauv ntej, thiab me ntsis ntxiv mus rau sab nraud IPv4- Internet - rau Piv txwv li, rau indexing.
Ob peb zaug dhau los ntawm cov ntaub ntawv chaw network tsim tau siv ntau txheej Clos topologies thiab tsuas yog L3. Peb tawm L2 ib pliag dhau los thiab ua pa nyuaj siab. Thaum kawg, peb cov txheej txheem suav nrog ntau pua txhiab tus lej (server) piv txwv. Qhov siab tshaj plaws pawg loj ib zaug dhau los yog txog 10 txhiab servers. Qhov no feem ntau yog vim li cas cov tib pawg-theem kev ua haujlwm, cov sijhawm teem sijhawm, kev faib cov peev txheej, thiab lwm yam tuaj yeem ua haujlwm. Peb muaj ib txoj hauj lwm - yuav tsum muaj peev xwm tsim network factories uas tso cai rau kev siv tau zoo nyob rau hauv xws li ib pawg.
Peb xav tau dab tsi los ntawm cov chaw zov me nyuam network? Ua ntej tshaj plaws, muaj ntau pheej yig thiab ncaj ncees faib bandwidth. Vim tias lub network yog lub nraub qaum los ntawm qhov uas peb tuaj yeem sau cov peev txheej. Lub hom phiaj tshiab loj yog kwv yees li 100 txhiab servers hauv ib pawg.
Peb kuj, tau kawg, xav tau lub dav hlau tswj kom muaj zog thiab ruaj khov, vim hais tias ntawm cov txheej txheem loj loj no tshwm sim ntau qhov mob taub hau txawm tias los ntawm cov xwm txheej tsis sib xws, thiab peb tsis xav kom lub dav hlau tswj tau ua rau peb mob taub hau thiab. Tib lub sijhawm, peb xav kom txo qis hauv lub xeev hauv nws. Tus mob me dua, qhov zoo dua thiab ruaj khov txhua yam ua haujlwm, thiab nws yooj yim dua los kuaj xyuas.
Ntawm chav kawm, peb xav tau automation, vim hais tias nws yog tsis yooj yim sua kom tswj xws li ib tug infrastructure manually, thiab nws twb tsis yooj yim sua rau tej lub sij hawm. Peb xav tau kev txhawb nqa kev ua haujlwm ntau npaum li qhov ua tau thiab CI / CD kev txhawb nqa rau qhov uas nws tuaj yeem muab tau.
Nrog xws li qhov ntau thiab tsawg ntawm cov chaw zov me nyuam thiab pawg, txoj haujlwm ntawm kev txhawb nqa kev xa tawm thiab nthuav dav yam tsis muaj kev cuam tshuam kev pabcuam tau dhau los ua mob hnyav heev. Yog hais tias nyob rau hauv pawg loj ntawm ib txhiab lub tshuab, tej zaum ze rau kaum txhiab lub tshuab, lawv tseem tuaj yeem muab dov tawm ua ib qho kev ua haujlwm - uas yog, peb tab tom npaj kev nthuav dav ntawm cov txheej txheem, thiab ob peb txhiab lub tshuab tau ntxiv ua ib qho haujlwm, ces ib pawg ntawm ib puas txhiab lub tshuab tsis tshwm sim tam sim li no, nws yog tsim nyob rau hauv ib lub sij hawm ntawm lub sij hawm. Thiab nws yog ntshaw tias tag nrho lub sij hawm no dab tsi twb tau pumped tawm, cov infrastructure uas tau deployed, yuav tsum muaj.
Thiab ib qho kev xav tau uas peb muaj thiab sab laug: kev txhawb nqa rau ntau qhov, uas yog, virtualization lossis network segmentation. Tam sim no peb tsis tas yuav ua qhov no ntawm qib ntaub network, vim hais tias cov sharding tau mus rau cov tswv, thiab qhov no tau ua kom yooj yim heev rau peb. Ua tsaug rau IPv6 thiab qhov chaw nyob loj, peb tsis tas yuav siv qhov chaw nyob sib npaug hauv cov txheej txheem sab hauv; txhua qhov chaw nyob yog qhov tshwj xeeb. Thiab ua tsaug rau qhov tseeb tias peb tau coj kev lim dej thiab kev sib faib network rau cov tswv, peb tsis tas yuav tsim ib qho chaw virtual network hauv cov ntaub ntawv chaw network.
Ib qho tseem ceeb heev yog qhov peb tsis xav tau. Yog tias qee qhov haujlwm tuaj yeem raug tshem tawm ntawm lub network, qhov no ua rau lub neej yooj yim dua, thiab, raws li txoj cai, nthuav dav cov kev xaiv ntawm cov khoom siv thiab software, ua rau kev kuaj mob yooj yim heev.
Yog li, dab tsi yog qhov uas peb tsis xav tau, dab tsi peb tuaj yeem tso tseg, tsis yog ib txwm muaj kev xyiv fab thaum lub sijhawm nws tshwm sim, tab sis nrog kev nplij siab thaum ua tiav?
Ua ntej tshaj plaws, tso tseg L2. Peb tsis xav tau L2, tsis muaj tiag los yog emulated. Tsis siv ntau vim qhov tseeb tias peb tswj cov ntawv thov pawg. Peb daim ntawv thov yog kab rov tav scalable, lawv ua haujlwm nrog L3 qhov chaw, lawv tsis txhawj xeeb tias qee tus neeg piv txwv tau tawm mus, lawv tsuas yob tawm ib qho tshiab, nws tsis tas yuav dov tawm ntawm qhov chaw nyob qub, vim tias muaj ib qho cais theem ntawm kev pabcuam nrhiav pom thiab saib xyuas cov tshuab nyob hauv pawg. Peb tsis muab txoj haujlwm no rau lub network. Lub network txoj haujlwm yog xa cov pob ntawv los ntawm point A mus rau point B.
Peb kuj tsis muaj qhov xwm txheej uas qhov chaw nyob txav mus rau hauv lub network, thiab qhov no yuav tsum tau saib xyuas. Hauv ntau qhov kev tsim qauv no feem ntau xav tau los txhawb VM kev txav mus los. Peb tsis siv kev txav mus los ntawm cov tshuab virtual hauv kev tsim kho vaj tse sab hauv ntawm Yandex loj, thiab, ntxiv rau, peb ntseeg tias txawm tias qhov no ua tiav, nws yuav tsum tsis txhob tshwm sim nrog kev txhawb nqa network. Yog hais tias nws yuav tsum tau ua tiag tiag, nws yuav tsum tau ua nyob rau hauv lub host theem, thiab thawb cov chaw nyob uas yuav tsiv mus nyob rau hauv overlays, thiaj li tsis mus kov los yog ua ntau dynamic hloov mus rau lub routing system ntawm underlay nws tus kheej (kev thauj mus los network) .
Lwm lub tshuab uas peb tsis siv yog multicast. Yog tias koj xav tau, kuv tuaj yeem qhia rau koj paub meej vim li cas. Qhov no ua rau lub neej yooj yim dua, vim hais tias yog ib tug neeg tau daws nws thiab saib raws nraim li cas lub dav hlau tswj multicast zoo li, nyob rau hauv tag nrho tab sis yooj yim installation, qhov no yog mob taub hau loj. Thiab dab tsi ntxiv, nws yog ib qho nyuaj rau nrhiav tau qhov kev ua haujlwm zoo qhib qhov kev siv, piv txwv li.
Thaum kawg, peb tsim peb lub network kom lawv tsis txhob hloov ntau dhau. Peb tuaj yeem suav rau ntawm qhov tseeb tias qhov ntws ntawm cov xwm txheej sab nraud hauv cov txheej txheem routing yog me me.
Cov teeb meem dab tsi tshwm sim thiab cov kev txwv dab tsi yuav tsum tau coj mus rau hauv tus account thaum peb tsim ib lub chaw zov me nyuam network? Nqi, tau kawg. Scalability, theem uas peb xav kom loj hlob. Qhov yuav tsum tau nthuav dav yam tsis muaj kev txwv tsis pub siv. Bandwidth, muaj. Kev pom ntawm qhov tshwm sim ntawm lub network rau kev soj ntsuam xyuas, rau pawg ua haujlwm. Kev them nyiaj yug Automation - dua, ntau npaum li qhov ua tau, txij li cov haujlwm sib txawv tuaj yeem daws tau ntawm ntau theem, suav nrog kev qhia txog cov txheej txheem ntxiv. Zoo, tsis yog [tejzaum] nyob ntawm cov neeg muag khoom. Txawm hais tias nyob rau lub sijhawm keeb kwm sib txawv, nyob ntawm seb koj saib ntawm ntu twg, qhov kev ywj pheej no yooj yim dua los yog nyuaj rau ua tiav. Yog hais tias peb coj ib tug ntoo khaub lig-section ntawm network ntaus ntawv chips, ces txog rau thaum tsis ntev los no nws yog heev conditional mus tham txog kev ywj pheej ntawm cov neeg muag khoom, yog hais tias peb kuj xav chips nrog high throughput.
Dab tsi logical topology peb yuav siv los tsim peb lub network? Qhov no yuav yog ntau theem Clos. Qhov tseeb, tsis muaj lwm txoj hauv kev tam sim no. Thiab Clos topology yog qhov zoo heev, txawm tias piv rau ntau yam kev tshaj lij topologies uas muaj ntau dua hauv thaj tsam ntawm kev kawm tam sim no, yog tias peb muaj cov hloov pauv loj radix.
Qhov ntau-theem Clos network sib xyaw ua ke li cas thiab cov ntsiab lus sib txawv hu ua dab tsi hauv nws? Ua ntej ntawm tag nrho cov, cua sawv, taw qhia koj tus kheej qhov twg yog sab qaum teb, qhov twg yog sab qab teb, qhov twg yog sab hnub tuaj, qhov twg yog sab hnub poob. Cov tes hauj lwm ntawm hom no feem ntau yog tsim los ntawm cov neeg uas muaj tsheb loj heev sab hnub poob-sab hnub tuaj. Raws li rau cov ntsiab lus tseem ceeb, nyob rau sab saum toj yog qhov hloov pauv virtual sib sau los ntawm cov keyboards me. Qhov no yog lub tswv yim tseem ceeb ntawm kev tsim kho ntawm Clos networks. Peb coj cov ntsiab lus nrog qee yam radix thiab txuas rau lawv kom qhov peb tau txais tuaj yeem suav tias yog kev hloov pauv nrog radix loj dua. Yog tias koj xav tau ntau dua, cov txheej txheem tuaj yeem rov ua dua.
Hauv qhov xwm txheej, piv txwv li, nrog ob-theem Clos, thaum nws muaj peev xwm txheeb xyuas cov ntsiab lus uas muaj ntsug hauv kuv daim duab, lawv feem ntau hu ua dav hlau. Yog tias peb yuav tsim Clos nrog peb theem ntawm cov nqaj qaum (tag nrho cov uas tsis yog ciam teb lossis ToR keyboards thiab uas tsuas yog siv rau kev thauj mus los), ces cov dav hlau yuav zoo li nyuaj dua; ob theem zoo li qhov no. Peb hu ib lub thaiv ntawm ToR los yog nplooj ntawv hloov thiab thawj theem qaum keyboards txuam nrog lawv Pod. Cov pob txha hloov ntawm tus txha nraub qaum-1 qib nyob rau sab saum toj ntawm Pod yog sab saum toj ntawm Pod, sab saum toj ntawm Pod. Cov keyboards uas nyob rau sab saum toj ntawm tag nrho lub Hoobkas yog cov txheej txheem sab saum toj ntawm lub Hoobkas, Sab saum toj ntawm cov ntaub.
Tau kawg, cov lus nug tshwm sim: Kaw tes hauj lwm tau tsim rau qee lub sij hawm; lub tswv yim nws tus kheej feem ntau yog los ntawm lub sij hawm ntawm classical telephony, TDM tes hauj lwm. Tej zaum ib yam dab tsi zoo dua tau tshwm sim, tej zaum ib yam dab tsi tuaj yeem ua tau zoo dua? Yog thiab tsis yog. Theoretically yog, nyob rau hauv kev xyaum nyob rau hauv lub ze yav tom ntej yeej tsis. Vim tias muaj ntau qhov kev nthuav dav topologies, qee qhov ntawm lawv txawm siv hauv kev tsim khoom, piv txwv li, Dragonfly yog siv hauv HPC daim ntawv thov; Tseem muaj cov lus nthuav dav xws li Xpander, FatClique, Jellyfish. Yog tias koj saib cov lus ceeb toom ntawm cov rooj sib tham xws li SIGCOMM lossis NSDI tsis ntev los no, koj tuaj yeem pom ntau qhov haujlwm ntawm lwm cov topologies uas muaj cov khoom zoo (ib lossis lwm qhov) dua li Clos.
Tab sis tag nrho cov topologies no muaj ib qho cuab yeej nthuav. Nws tiv thaiv lawv cov kev siv nyob rau hauv cov ntaub ntawv chaw tes hauj lwm, uas peb tab tom sim tsim cov khoom siv kho vajtse thiab cov nqi tsim nyog heev. Nyob rau hauv tag nrho cov lwm txoj kev topologies, feem ntau ntawm cov bandwidth yog hmoov tsis nkag tau los ntawm shortest paths. Yog li ntawd, peb tam sim ntawd poob lub sijhawm los siv lub dav hlau tswj ib txwm muaj.
Theoretically, kev daws teeb meem yog paub. Cov no yog, piv txwv li, kev hloov kho ntawm lub xeev txuas siv k-shortest txoj kev, tab sis, dua, tsis muaj cov txheej txheem zoo li no uas yuav raug siv hauv kev tsim khoom thiab dav siv hauv cov khoom siv.
Ntxiv mus, vim tias feem ntau ntawm lub peev xwm tsis tuaj yeem nkag tau los ntawm txoj kev luv tshaj plaws, peb yuav tsum hloov kho ntau dua li lub dav hlau tswj kom xaiv tag nrho cov kev mus (thiab los ntawm txoj kev, qhov no tseem ceeb dua lub xeev hauv lub dav hlau tswj). Peb tseem yuav tsum tau hloov kho lub dav hlau xa mus, thiab, raws li txoj cai, yuav tsum muaj tsawg kawg yog ob yam ntxiv. Qhov no yog lub peev xwm los txiav txim siab txog pob ntawv xa mus ib zaug, piv txwv li, ntawm tus tswv tsev. Qhov tseeb, qhov no yog qhov chaw routing, qee zaum hauv cov ntaub ntawv ntawm kev sib txuas lus sib txuas lus no yog hu ua kev txiav txim siab xa mus rau txhua qhov. Thiab adaptive routing yog ib tug muaj nuj nqi uas peb xav tau ntawm lub network ntsiab, uas boils down, piv txwv li, rau qhov tseeb hais tias peb xaiv lub tom ntej hop raws li cov ntaub ntawv hais txog qhov tsawg kawg nkaus load ntawm lub queue. Piv txwv li, muaj lwm txoj kev xaiv.
Yog li, kev taw qhia yog qhov nthuav, tab sis, alas, peb tsis tuaj yeem siv nws tam sim no.
Okay, peb tau txiav txim siab ntawm Clos logical topology. Peb yuav ntsuas nws li cas? Cia peb saib seb nws ua haujlwm li cas thiab yuav ua li cas.
Hauv Clos network muaj ob qhov tseem ceeb uas peb tuaj yeem sib txawv thiab tau txais qee yam txiaj ntsig: lub radix ntawm cov ntsiab lus thiab tus lej ntawm qib hauv lub network. Kuv muaj daim duab schematic ntawm ob qho tib si cuam tshuam li cas. Qhov zoo tshaj plaws, peb muab ob qho tib si.
Nws tuaj yeem pom tau tias qhov dav kawg ntawm Clos network yog cov khoom lag luam ntawm txhua theem ntawm cov nqaj qaum hloov mus rau yav qab teb radix, pes tsawg qhov txuas peb tau nqis, nws cov ceg li cas. Nov yog qhov peb ntsuas qhov loj ntawm lub network.
Hais txog lub peev xwm, tshwj xeeb tshaj yog nyob rau ntawm ToR keyboards, muaj ob txoj kev ntsuas ntsuas. Txawm li cas los xij peb tuaj yeem, thaum tswj hwm qhov dav dav, siv cov kev sib txuas sai dua, lossis peb tuaj yeem ntxiv cov dav hlau ntxiv.
Yog tias koj saib ntawm qhov nthuav dav ntawm Clos network (hauv kaum sab xis sab xis) thiab rov qab mus rau daim duab no nrog Clos network hauv qab no ...
... ces qhov no yog raws nraim tib yam topology, tab sis nyob rau hauv no swb nws yog vau ntau compactly thiab lub dav hlau ntawm lub Hoobkas yog superimposed rau ib leeg. Nws yog tib yam.
Qhov kev ntsuas qhov Clos network zoo li cas hauv cov lej? Ntawm no kuv muab cov ntaub ntawv hais txog qhov dav tshaj plaws ntawm lub network tuaj yeem tau txais, ntau npaum li cas ntawm cov racks, ToR keyboards lossis nplooj hloov pauv, yog tias lawv tsis nyob hauv racks, peb tuaj yeem tau txais nyob ntawm seb lub radix ntawm cov keyboards peb siv rau qaum-theem, thiab peb siv pes tsawg qib.
Ntawm no yog pes tsawg racks peb tuaj yeem muaj, pes tsawg servers thiab kwv yees li cas tag nrho cov no tuaj yeem noj raws li 20 kW ib rack. Ib me ntsis ua ntej kuv tau hais tias peb tab tom npaj rau pawg loj ntawm 100 txhiab servers.
Nws tuaj yeem pom tias nyob rau hauv tag nrho cov qauv no, ob thiab ib nrab kev xaiv yog txaus siab. Muaj ib qho kev xaiv nrog ob txheej ntawm spines thiab 64-chaw nres nkoj keyboards, uas ntog me ntsis. Tom qab ntawd muaj cov kev xaiv zoo kawg nkaus rau 128-chaw nres nkoj (nrog radix 128) qaum hloov nrog ob theem, lossis hloov nrog radix 32 nrog peb theem. Thiab nyob rau hauv txhua rooj plaub, qhov twg muaj ntau radixes thiab ntau cov khaubncaws sab nraud povtseg, koj muaj peev xwm ua ib tug loj heev network, tab sis yog hais tias koj saib nyob rau hauv kev cia siab noj, feem ntau muaj gigawatts. Nws muaj peev xwm nteg ib lub cable, tab sis peb tsis zoo li yuav tau txais hluav taws xob ntau ntawm ib qhov chaw. Yog tias koj saib cov ntaub ntawv txheeb cais thiab cov ntaub ntawv pej xeem ntawm cov chaw khaws ntaub ntawv, koj tuaj yeem pom cov ntaub ntawv tsawg heev uas muaj peev xwm kwv yees ntau dua 150 MW. Cov loj dua feem ntau yog cov chaw zov me nyuam cov ntaub ntawv, ntau lub chaw cov ntaub ntawv loj nyob ze rau ib leeg.
Muaj lwm qhov tseem ceeb parameter. Yog tias koj saib ntawm sab laug, siv tau bandwidth tau teev nyob rau ntawd. Nws yog ib qho yooj yim kom pom tias hauv Clos network ib feem tseem ceeb ntawm cov chaw nres nkoj tau siv los txuas cov keyboards rau ib leeg. Usable bandwidth, ib qho khoom siv tau zoo, yog ib yam uas tuaj yeem muab rau sab nraud, ntawm cov servers. Lawm, kuv tab tom tham txog cov chaw nres nkoj muaj cai thiab tshwj xeeb txog cov qhab. Raws li txoj cai, kev sib txuas hauv lub network nrawm dua li kev sib txuas ntawm cov servers, tab sis ib chav tsev ntawm bandwidth, ntau npaum li peb tuaj yeem xa tawm mus rau peb cov khoom siv server, tseem muaj qee qhov bandwidth hauv lub network nws tus kheej. Thiab ntau theem peb ua, ntau dua tus nqi tshwj xeeb ntawm kev muab cov kab txaij no rau sab nraud.
Ntxiv mus, txawm tias qhov no ntxiv band tsis zoo ib yam. Thaum lub sijhawm luv luv, peb tuaj yeem siv qee yam xws li DAC (txuas txuas tooj liab, uas yog, twinax cables), lossis multimode optics, uas raug nqi ntau dua lossis tsawg dua cov nyiaj tsim nyog. Sai li sai tau thaum peb txav mus ntev dua - raws li txoj cai, cov no yog ib hom optics, thiab tus nqi ntawm qhov bandwidth ntxiv no nce ntxiv.
Thiab ib zaug ntxiv, rov qab mus rau qhov swb dhau los, yog tias peb tsim Clos network yam tsis muaj kev tshaj tawm, ces nws yooj yim los saib ntawm daim duab, saib seb lub network tau tsim li cas - ntxiv txhua qib ntawm tus txha nraub qaum keyboards, peb rov ua dua tag nrho cov kab xev uas nyob ntawm qhov hauv qab. Ntxiv rau qib - ntxiv rau tib pawg, tib tus naj npawb ntawm cov chaw nres nkoj ntawm cov keyboards raws li muaj nyob rau qib dhau los, thiab tib tus naj npawb ntawm transceivers. Yog li ntawd, nws yog ib qho tsim nyog kom txo qis tus naj npawb ntawm cov leeg txha nraub qaum.
Raws li daim duab no, nws yog qhov tseeb tias peb xav tsim ib yam dab tsi zoo li cov keyboards nrog lub radix ntawm 128.
Ntawm no, hauv paus ntsiab lus, txhua yam zoo ib yam li qhov kuv nyuam qhuav hais; qhov no yog ib qho swb rau kev txiav txim siab tom qab.
Muaj kev xaiv dab tsi uas peb tuaj yeem xaiv xws li cov keyboards? Nws yog cov xov xwm zoo heev rau peb tias tam sim no cov tes hauj lwm zoo li no thaum kawg tuaj yeem ua rau ntawm ib qho-chip keyboards. Thiab qhov no yog qhov txias heev, lawv muaj ntau yam zoo. Piv txwv li, lawv yuav luag tsis muaj cov qauv sab hauv. Qhov no txhais tau tias lawv tawg yooj yim dua. Lawv rhuav txhua txoj hauv kev, tab sis hmoov zoo lawv tawg tag nrho. Hauv cov khoom siv modular muaj ntau qhov kev ua txhaum (tsis txaus siab heev), thaum los ntawm qhov pom ntawm cov neeg nyob ze thiab lub dav hlau tswj nws zoo li ua haujlwm, tab sis, piv txwv li, ib feem ntawm cov ntaub tau ploj lawm thiab tsis ua haujlwm. ntawm lub peev xwm. Thiab cov tsheb khiav mus rau nws yog sib npaug raws li qhov tseeb tias nws ua haujlwm tau zoo, thiab peb tuaj yeem tau txais ntau dhau.
Los yog, piv txwv li, teeb meem tshwm sim nrog lub backplane, vim hais tias nyob rau hauv lub modular ntaus ntawv kuj muaj high-ceev SerDes - nws yog heev complex sab hauv. Txawm hais tias cov cim qhia ntawm kev xa mus yog synchronized lossis tsis synchronized. Feem ntau, txhua yam khoom tsim tau tsim muaj ntau lub ntsiab lus, raws li txoj cai, muaj tib lub Clos network hauv nws tus kheej, tab sis nws nyuaj heev rau kev kuaj mob. Feem ntau nws nyuaj rau tus neeg muag khoom nws tus kheej los kuaj xyuas.
Thiab nws muaj ntau qhov xwm txheej tsis ua haujlwm nyob rau hauv uas lub cuab yeej degrades, tab sis tsis poob ntawm topology kiag li. Txij li thaum peb lub network loj, qhov sib npaug ntawm cov ntsiab lus zoo ib yam yog siv, lub network yog qhov tsis tu ncua, uas yog, ib txoj hauv kev uas txhua yam nyob rau hauv kev txiav txim tsis txawv ntawm lwm txoj kev, nws muaj txiaj ntsig ntau dua rau peb kom tsuas yog poob qee qhov. cov khoom siv los ntawm topology dua li xaus rau hauv qhov xwm txheej uas qee qhov zoo li ua haujlwm, tab sis qee qhov tsis ua.
Cov yam ntxwv zoo tom ntej ntawm ib leeg-chip li yog tias lawv hloov kho kom zoo dua thiab sai dua. Lawv kuj zoo li muaj peev xwm zoo dua. Yog tias peb coj cov qauv sib dhos loj uas peb muaj nyob rau hauv lub voj voog, ces lub peev xwm ntawm ib chav tsev khib rau cov chaw nres nkoj ntawm tib qhov ceev yog yuav luag ob zaug zoo li cov khoom siv modular. Cov cuab yeej tsim nyob ib puag ncig ib nti yog qhov pheej yig dua li cov khoom siv modular thiab siv zog tsawg.
Tab sis, ntawm chav kawm, qhov no yog tag nrho vim li cas, kuj muaj qhov tsis zoo. Ua ntej, lub radix yuav luag ib txwm me dua li ntawm cov khoom siv modular. Yog tias peb tuaj yeem tau txais ib lub cuab yeej ua ib ncig ntawm ib lub nti nrog 128 chaw nres nkoj, ces peb tuaj yeem tau txais ib qho modular nrog ntau pua chaw nres nkoj tam sim no yam tsis muaj teeb meem.
Qhov no yog qhov me me me me ntawm cov lus xa mus thiab, raws li txoj cai, txhua yam ntsig txog cov ntaub ntawv dav hlau scalability. Nthuav buffers. Thiab, raws li txoj cai, theej txwv functionality. Tab sis nws hloov tawm tias yog tias koj paub cov kev txwv no thiab saib xyuas lub sijhawm los hla lawv lossis tsuas yog coj lawv mus rau hauv tus account, ces qhov no tsis txaus ntshai. Qhov tseeb tias radix me me tsis yog ib qho teeb meem ntawm cov khoom siv nrog lub radix ntawm 128 uas tau tshwm sim thaum kawg tsis ntev los no; peb tuaj yeem tsim hauv ob txheej ntawm cov txha nraub qaum. Tab sis nws tseem tsis tuaj yeem tsim ib yam dab tsi me dua li ob qho uas txaus siab rau peb. Nrog ib qib, pawg me me tau txais. Txawm tias peb cov qauv tsim thiab cov kev xav tau dhau los tseem tshaj lawv.
Qhov tseeb, yog tias tam sim ntawd qhov kev daws teeb meem yog qhov chaw ntawm lub brink, tseem muaj txoj hauv kev los ntsuas. Txij li thaum kawg (lossis thawj zaug), qib qis tshaj qhov twg cov servers txuas nrog yog ToR keyboards lossis nplooj hloov pauv, peb tsis tas yuav txuas ib lub khib rau lawv. Yog li ntawd, yog tias cov tshuaj poob qis los ntawm ib nrab, koj tuaj yeem xav txog tsuas yog siv qhov hloov pauv nrog lub radix loj ntawm qib qis thiab txuas, piv txwv li, ob lossis peb racks rau hauv ib qho kev hloov. Qhov no kuj yog ib qho kev xaiv, nws muaj nws cov nqi, tab sis nws ua haujlwm zoo heev thiab tuaj yeem yog qhov kev daws teeb meem zoo thaum koj xav tau txog li ob npaug ntawm qhov loj.
Txhawm rau kom ua tiav, peb tab tom tsim lub topology nrog ob theem ntawm spines, nrog yim lub Hoobkas txheej.
Yuav ua li cas rau physics? Kev suav yooj yim heev. Yog hais tias peb muaj ob theem ntawm spines, ces peb tsuas muaj peb theem ntawm switches, thiab peb cia siab tias yuav muaj peb cables nyob rau hauv lub network: los ntawm servers mus rau nplooj hloov, mus rau qaum 1, mus rau qaum 2. Cov kev xaiv uas peb yuav tau. siv yog - cov no yog twinax, multimode, ib hom. Thiab ntawm no peb yuav tsum xav txog dab tsi sawb muaj, nws yuav raug nqi npaum li cas, qhov ntev ntawm lub cev yog dab tsi, qhov ntev peb tuaj yeem npog, thiab peb yuav hloov kho li cas.
Hais txog tus nqi, txhua yam tuaj yeem ua ke. Twinaxes yog pheej yig dua li active optics, pheej yig dua li multimode transceivers, yog tias koj coj nws ib lub davhlau los ntawm qhov kawg, me ntsis pheej yig dua li 100-gigabit hloov chaw nres nkoj. Thiab, thov nco ntsoov, nws raug nqi tsawg dua ib hom optics, vim hais tias ntawm kev ya davhlau uas yuav tsum muaj ib hom, hauv cov chaw zov me nyuam rau ntau qhov laj thawj nws ua rau kev txiav txim siab siv CWDM, thaum sib npaug tib hom (PSM) tsis yooj yim rau kev ua haujlwm. nrog, cov pob khoom loj heev tau txais fibers, thiab yog tias peb tsom rau cov thev naus laus zis no, peb tau txais kwv yees li cov nqi hauv qab no.
Ib qho ntxiv: Hmoov tsis zoo, nws tsis tuaj yeem siv disassembled 100 rau 4x25 multimode ports. Vim yog cov qauv tsim ntawm SFP28 transceivers, nws tsis yog pheej yig dua li 28 Gbit QSFP100. Thiab qhov no disassembly rau multimode tsis ua haujlwm zoo heev.
Lwm qhov kev txwv yog vim qhov loj ntawm cov pawg hauv computer thiab cov naj npawb ntawm cov servers, peb cov chaw khaws ntaub ntawv tau dhau los ua lub cev loj. Qhov no txhais tau hais tias tsawg kawg ib lub davhlau yuav tsum tau ua nrog singlemod. Ntxiv dua thiab, vim lub cev loj ntawm lub Pods, nws yuav tsis muaj peev xwm khiav ob ntu ntawm twinax (tooj liab cables).
Yog li ntawd, yog tias peb ua kom zoo rau tus nqi thiab coj mus rau hauv tus account lub geometry ntawm qhov tsim no, peb tau txais ib ntu ntawm twinax, ib ntu ntawm multimode thiab ib ntu ntawm singlemode siv CWDM. Qhov no yuav siv sij hawm mus rau hauv tus account ua tau upgrade paths.
Nov yog qhov nws zoo li tsis ntev los no, qhov twg peb tab tom mus thiab qhov ua tau. Nws yog qhov tseeb, tsawg kawg, yuav ua li cas txav mus rau 50-Gigabit SerDes rau ob qho tib si multimode thiab singlemode. Ntxiv mus, yog tias koj saib dab tsi nyob rau hauv ib hom transceivers tam sim no thiab yav tom ntej rau 400G, feem ntau txawm tias thaum 50G SerDes tuaj txog ntawm hluav taws xob sab, 100 Gbps ib txoj kab tuaj yeem mus rau optics. Yog li ntawd, nws yog qhov ua tau heev uas tsis hloov mus rau 50, yuav muaj kev hloov pauv mus rau 100 Gigabit SerDes thiab 100 Gbps rau ib txoj kab, vim hais tias raws li cov lus cog tseg ntawm ntau tus neeg muag khoom, lawv qhov muaj yuav tsum tau sai sai. Lub sijhawm thaum 50G SerDes yog qhov ceev tshaj plaws, zoo li, yuav tsis ntev heev, vim tias thawj cov ntawv luam ntawm 100G SerDes tau dov tawm yuav luag xyoo tom ntej. Thiab tom qab qee lub sij hawm tom qab ntawd lawv tej zaum yuav tsim nyog cov nyiaj tsim nyog.
Ib qho ntxiv nuance txog kev xaiv ntawm physics. Hauv txoj cai, peb tuaj yeem siv 400 lossis 200 Gigabit chaw nres nkoj siv 50G SerDes. Tab sis nws hloov tawm tias qhov no tsis ua rau muaj kev nkag siab ntau, vim hais tias, raws li kuv tau hais ua ntej, peb xav tau ib lub radix loj loj ntawm cov keyboards, hauv qhov laj thawj, tau kawg. Peb xav tau 128. Thiab yog tias peb muaj peev xwm chips tsawg thiab peb nce qhov txuas ceev, ces cov radix ib txwm txo qis, tsis muaj txuj ci tseem ceeb.
Thiab peb tuaj yeem ua kom tag nrho lub peev xwm siv dav hlau, thiab tsis muaj nqi tshwj xeeb; peb tuaj yeem ntxiv cov dav hlau. Thiab yog tias peb poob lub radix, peb yuav tau qhia ib qib ntxiv, yog li nyob rau hauv qhov xwm txheej tam sim no, nrog rau qhov muaj peev xwm tshaj plaws tam sim no rau ib nti, nws hloov tawm tias nws siv tau ntau dua los siv 100-gigabit ports, vim tias lawv tso cai rau koj. kom tau ib tug loj radix.
Cov lus nug tom ntej yog yuav ua li cas physics raug teeb tsa, tab sis los ntawm qhov pom ntawm cable infrastructure. Nws hloov tawm hais tias nws yog teem nyob rau hauv ib tug theej funny txoj kev. Cabling ntawm nplooj-hloov thiab thawj-qib spines - tsis muaj ntau qhov txuas rau ntawd, txhua yam yog tsim tau yooj yim. Tab sis yog tias peb nqa ib lub dav hlau, dab tsi tshwm sim sab hauv yog tias peb yuav tsum tau txuas tag nrho cov txha nraub qaum ntawm thawj theem nrog tag nrho cov txha nraub qaum ntawm qib thib ob.
Ntxiv rau, raws li txoj cai, muaj qee qhov kev xav tau li cas nws yuav tsum saib hauv cov ntaub ntawv chaw. Piv txwv li, peb yeej xav muab cov cables rau hauv ib lub nras thiab rub lawv kom ib lub vaj huam sib luag high-density mus rau hauv ib lub vaj huam sib luag, kom tsis muaj vaj tse nyob rau qhov ntev. Peb tswj daws qhov teeb meem no. Yog tias koj pib saib lub logical topology, koj tuaj yeem pom tias cov dav hlau muaj kev ywj pheej, txhua lub dav hlau tuaj yeem tsim los ntawm nws tus kheej. Tab sis thaum peb ntxiv xws li ib pob khoom thiab xav rub tag nrho cov vaj huam sib luag rau hauv lub vaj huam sib luag, peb yuav tsum sib xyaw cov dav hlau sib txawv hauv ib pob thiab qhia cov qauv nruab nrab hauv daim ntawv ntawm kev sib txuas kho qhov muag kom rov ntim lawv li cas lawv tau sib sau ua ke. ntawm ib ntu, nyob rau hauv li cas lawv yuav raug sau rau lwm ntu. Ua tsaug rau qhov no, peb tau txais ib qho zoo feature: tag nrho cov complex switching tsis mus dhau lub racks. Thaum koj xav tau kev sib cuam tshuam ib yam dab tsi uas muaj zog heev, "tawm lub dav hlau," raws li qee zaum hu ua Clos tes hauj lwm, nws yog tag nrho cov concentrated hauv ib lub khib. Peb tsis muaj disassembled heev, nqes mus rau ib tus neeg txuas, hloov ntawm racks.
Qhov no yog yuav ua li cas nws zoo li los ntawm qhov kev pom ntawm lub koom haum logic ntawm cable infrastructure. Nyob rau hauv daim duab ntawm sab laug, cov blocks ntau xim qhia txog cov blocks ntawm thawj-qib txha nraub qaum hloov, yim daim txhua, thiab plaub pob khoom ntawm cov cables los ntawm lawv, uas mus thiab sib tshuam nrog cov pob khoom los ntawm cov pob txha caj qaum-2 keyboards. .
Cov squares me me qhia txog kev sib tshuam. Nyob rau sab laug sab saum toj yog qhov kev puas tsuaj ntawm txhua qhov kev sib tshuam, qhov no yog qhov tseeb 512 los ntawm 512 qhov chaw nres nkoj hla-txuas module uas rov ntim cov cables kom lawv tuaj yeem nkag mus rau hauv ib lub khib nyiab, qhov twg tsuas muaj ib lub dav hlau qaum-2 xwb. Thiab ntawm sab xis, kev luam theej duab ntawm daim duab no yog me ntsis ntxaws ntxiv txog kev cuam tshuam rau ntau lub Pods ntawm tus txha nraub qaum-1 qib, thiab nws tau ntim li cas hauv kev sib txuas, yuav ua li cas nws los txog rau tus txha nraub qaum-2 qib.
Qhov no yog qhov nws zoo li. Qhov tseem tsis tau tag nrho cov txha nraub qaum-2 sawv ntsug (sab laug) thiab tus ntoo khaub lig-txuas sawv ntsug. Hmoov tsis, tsis muaj ntau pom nyob ntawd. Tag nrho cov qauv no tau siv tam sim no nyob rau hauv ib qho ntawm peb cov ntaub ntawv loj loj uas tau nthuav dav. Qhov no yog kev ua haujlwm hauv kev kawm, nws yuav zoo dua, nws yuav ua tiav zoo dua.
Ib lo lus nug tseem ceeb: peb xaiv lub ntsiab lus topology thiab tsim lub physics. Yuav ua li cas rau lub dav hlau tswj? Nws yog qhov paub zoo los ntawm kev ua haujlwm dhau los, muaj ntau cov lus ceeb toom uas txuas lub xeev cov txheej txheem yog qhov zoo, nws yog qhov zoo siab ua haujlwm nrog lawv, tab sis, hmoov tsis, lawv tsis ntsuas qhov zoo ntawm qhov sib txuas nrog qhov sib txuas. Thiab muaj ib qho tseem ceeb uas tiv thaiv qhov no - qhov no yog li cas dej nyab ua haujlwm hauv kev sib txuas hauv xeev cov txheej txheem. Yog tias koj tsuas yog coj cov dej nyab algorithm thiab saib seb peb lub network tau teeb tsa li cas, koj tuaj yeem pom tias yuav muaj cov kiv cua loj heev ntawm txhua kauj ruam, thiab nws tsuas yog dej nyab lub dav hlau tswj nrog kev hloov kho tshiab. Tshwj xeeb, xws li topologies sib xyaw ua ke tsis zoo nrog cov tsoos dej nyab algorithm hauv kev sib txuas hauv xeev cov txheej txheem.
Qhov kev xaiv yog siv BGP. Yuav ua li cas npaj nws kom raug yog piav qhia hauv RFC 7938 txog kev siv BGP hauv cov chaw loj cov ntaub ntawv. Cov tswv yim yooj yim yog qhov yooj yim: tsawg kawg ntawm cov lus ua ntej rau ib tus tswv tsev thiab feem ntau yam tsawg kawg ntawm cov lus ua ntej ntawm lub network, siv kev sib sau ua ke yog tias ua tau, thiab txwv txoj kev yos hav zoov. Peb xav kom ceev faj heev, tswj kev faib tawm tshiab, yam hu ua hav dawb. Peb xav kom hloov tshiab kom xa mus raws nraim ib zaug thaum lawv dhau los ntawm lub network. Yog tias lawv tshwm sim hauv qab, lawv nce mus, nthuav tawm tsis pub ntau tshaj ib zaug. Yuav tsum tsis muaj zigzags. Zigzags phem heev.
Ua li no, peb siv tus qauv tsim uas yooj yim txaus los siv cov txheej txheem BGP hauv qab. Ntawd yog, peb siv eBGP khiav ntawm qhov txuas hauv zos, thiab cov kab ke tswj hwm tus kheej tau muab raws li hauv qab no: ib qho kev tswj hwm tus kheej ntawm ToR, ib qho kev tswj hwm ntawm tag nrho cov pob txha caj qaum-1 hloov ntawm ib lub Pod, thiab ib qho kev tswj hwm tus kheej ntawm tag nrho sab saum toj. ntawm Fabric. Nws tsis yog qhov nyuaj los saib thiab pom tias txawm tias tus cwj pwm zoo li qub ntawm BGP muab peb cov kev faib tawm tshiab uas peb xav tau.
Lawm, qhov chaw nyob thiab chaw nyob sib sau ua ke yuav tsum tau tsim kom nws sib haum nrog txoj kev routing tsim, kom nws ua kom muaj kev ruaj ntseg ntawm lub dav hlau tswj. L3 hais txog kev thauj mus los yog khi rau topology, vim tias tsis muaj qhov no nws tsis tuaj yeem ua tiav kev sib sau ua ke; yog tsis muaj qhov no, tus kheej chaw nyob yuav nkag mus rau hauv txoj kab ke. Thiab ib qho ntxiv yog tias kev sib sau ua ke, hmoov tsis zoo, tsis sib xyaw nrog ntau txoj hauv kev, vim tias thaum peb muaj ntau txoj hauv kev thiab peb muaj kev sib sau, txhua yam zoo, thaum tag nrho lub network noj qab haus huv, tsis muaj kev ua tsis tiav hauv nws. Hmoov tsis zoo, sai li sai tau thaum qhov kev ua tsis tiav tshwm sim hauv lub network thiab qhov sib luag ntawm lub topology ploj lawm, peb tuaj yeem mus rau qhov chaw uas lub tsev tau tshaj tawm, los ntawm qhov uas peb tsis tuaj yeem mus ntxiv mus rau qhov uas peb yuav tsum mus. Yog li ntawd, nws yog qhov zoo tshaj plaws los sib sau ua ke qhov twg tsis muaj ntau txoj hauv kev, hauv peb cov ntaub ntawv no yog ToR keyboards.
Qhov tseeb, nws yog ib qho ua tau kom sib sau ua ke, tab sis ua tib zoo. Yog tias peb tuaj yeem ua tau tswj kev sib cais thaum lub network tsis ua haujlwm. Tab sis qhov no yog ib txoj haujlwm nyuaj heev, peb txawm xav tias nws puas tuaj yeem ua tau qhov no, txawm tias nws muaj peev xwm ntxiv automation ntxiv, thiab lub xeev cov tshuab txiav txim siab uas yuav raug ncaws BGP kom tau txais tus cwj pwm xav tau. Hmoov tsis zoo, kev ua cov rooj plaub yog qhov tsis pom tseeb thiab nyuaj, thiab txoj haujlwm no tsis tau daws tau zoo los ntawm kev txuas cov ntawv txuas sab nraud rau BGP.
Kev nthuav dav heev hauv qhov no tau ua tiav nyob rau hauv lub moj khaum ntawm RIFT raws tu qauv, uas yuav tau tham hauv tsab ntawv tshaj tawm tom ntej.
Lwm qhov tseem ceeb yog li cas cov ntaub ntawv dav hlau nplai hauv cov toctologies ntom, uas peb muaj ntau txoj hauv kev. Hauv qhov no, ob peb cov ntaub ntawv ntxiv yog siv: ECMP pawg, uas nyob rau hauv lem piav txog Next Hop pawg.
Nyob rau hauv ib txwm ua haujlwm network, tsis muaj kev ua tsis tiav, thaum peb nce mus rau Clos topology, nws txaus los siv ib pab pawg, vim txhua yam uas tsis yog hauv zos tau piav qhia los ntawm lub neej ntawd, peb tuaj yeem nce mus. Thaum peb mus rau sab saum toj mus rau hauv qab mus rau sab qab teb, ces txhua txoj kev tsis yog ECMP, lawv yog ib txoj kev. Txhua yam zoo. Qhov teeb meem yog, thiab qhov peculiarity ntawm classic Clos topology yog tias yog peb saib saum cov ntaub, ntawm txhua lub caij, tsuas muaj ib txoj hauv kev rau ib qho hauv qab no. Yog hais tias tsis ua hauj lwm tshwm sim nyob rau hauv txoj kev no, ces qhov tshwj xeeb lub ntsiab nyob rau sab saum toj ntawm lub Hoobkas ua invalid precisely rau cov prefixes nyob tom qab txoj kev tawg. Tab sis rau qhov seem nws siv tau, thiab peb yuav tsum txheeb xyuas pawg ECMP thiab qhia lub xeev tshiab.
Cov ntaub ntawv dav hlau scalability zoo li cas ntawm cov khoom siv niaj hnub no? Yog tias peb ua LPM (ntev qhov sib tw ua ntej tshaj plaws), txhua yam zoo heev, dhau 100k prefixes. Yog hais tias peb tab tom tham txog Next Hop pab pawg, ces txhua yam yog phem, 2-4 txhiab. Yog tias peb tab tom tham txog lub rooj uas muaj cov lus piav qhia ntawm Next Hops (lossis qhov chaw nyob), ces qhov no yog qhov chaw ntawm 16k txog 64k. Thiab qhov no tuaj yeem dhau los ua teeb meem. Thiab ntawm no peb tuaj rau qhov kev xav tsis zoo: ua li cas rau MPLS hauv cov chaw zov me nyuam? Hauv txoj cai, peb xav ua nws.
Ob yam tshwm sim. Peb tau ua micro-segmentation ntawm cov tswv; peb tsis tas yuav ua nws ntawm lub network. Nws tsis zoo heev nrog kev txhawb nqa los ntawm cov neeg muag khoom sib txawv, thiab ntau ntxiv nrog rau kev siv qhib ntawm cov thawv dawb nrog MPLS. Thiab MPLS, tsawg kawg yog nws cov kev coj ua ib txwm muaj, hmoov tsis zoo, ua ke tsis zoo nrog ECMP. Thiab yog vim li cas.
Qhov no yog qhov ECMP xa mus rau IP zoo li. Ntau tus thawj coj tuaj yeem siv tib pab pawg thiab tib lub Next Hops block (lossis qhov chaw nyob, qhov no yuav raug hu ua txawv ntawm cov ntaub ntawv sib txawv rau cov khoom sib txawv). Lub ntsiab lus yog qhov no tau piav qhia tias yog qhov chaw nres nkoj tawm thiab dab tsi yuav rov sau dua MAC chaw nyob kom thiaj li tau mus rau qhov tseeb Next Hop. Rau IP txhua yam zoo li yooj yim, koj tuaj yeem siv ntau cov npe ua ntej rau tib pab pawg, tib lub Next Hops block.
Classic MPLS architecture txhais tau hais tias, nyob ntawm qhov kev sib tshuam tawm, daim ntawv lo tuaj yeem rov sau dua rau qhov sib txawv. Yog li ntawd, peb yuav tsum khaws ib pab pawg thiab Next Hops thaiv rau txhua daim ntawv sau npe. Thiab qhov no, alas, tsis scale.
Nws yooj yim pom tias hauv peb tus qauv tsim peb xav tau txog 4000 ToR keyboards, qhov siab tshaj plaws yog 64 ECMP txoj hauv kev, yog tias peb txav deb ntawm qaum-1 mus rau qaum-2. Peb nyuam qhuav nkag mus rau hauv ib lub rooj ntawm ECMP pawg, yog tias tsuas yog ib qho ua ntej nrog ToR ploj mus, thiab peb tsis nkag mus rau hauv lub rooj tom ntej Hops txhua.
Nws tsis yog txhua yam kev cia siab, vim hais tias architectures zoo li Segment Routing koom nrog cov ntawv sau thoob ntiaj teb. Raws li txoj cai, nws yuav tuaj yeem tawg tag nrho cov Next Hops blocks dua. Txhawm rau ua qhov no, koj xav tau daim npav qus hom kev ua haujlwm: nqa daim ntawv lo thiab rov sau dua rau tib yam tsis muaj nqi tshwj xeeb. Tab sis hmoov tsis, qhov no tsis yog tam sim no nyob rau hauv cov kev siv uas muaj.
Thiab thaum kawg, peb yuav tsum coj cov tsheb khiav sab nraud mus rau hauv cov ntaub ntawv chaw. Yuav ua li cas? Yav dhau los, kev khiav tsheb tau nkag mus rau hauv Clos network los ntawm saum toj no. Ntawd yog, muaj ntug routers uas txuas nrog txhua yam khoom siv rau saum cov ntaub. Qhov kev daws teeb meem no ua haujlwm zoo heev ntawm qhov me me thiab nruab nrab. Hmoov tsis zoo, txhawm rau xa cov tsheb khiav sib luag mus rau tag nrho cov network hauv txoj kev no, peb yuav tsum tuaj txog ib txhij ntawm txhua lub ntsiab lus ntawm Sab saum toj ntawm cov ntaub, thiab thaum muaj ntau tshaj li ib puas ntawm lawv, nws hloov tawm tias peb kuj xav tau ib qho loj. radix ntawm ntug routers. Feem ntau, qhov no raug nqi nyiaj, vim tias ntug routers ua haujlwm ntau dua, cov chaw nres nkoj ntawm lawv yuav kim dua, thiab tus qauv tsim tsis zoo nkauj heev.
Lwm qhov kev xaiv yog pib cov tsheb khiav zoo li hauv qab no. Nws yog ib qho yooj yim los xyuas kom meej tias Clos topology yog tsim nyob rau hauv xws li ib txoj kev uas cov tsheb khiav los ntawm cov hauv qab no, uas yog, los ntawm ToR sab, yog sib npaug zos faib ntawm cov theem thoob plaws tag nrho cov saum toj kawg nkaus ntawm ntaub nyob rau hauv ob iterations, loading tag nrho cov network. Yog li ntawd, peb qhia txog hom Pod, Edge Pod, uas muab kev sib txuas sab nraud.
Muaj ib qho kev xaiv ntxiv. Qhov no yog qhov Facebook ua, piv txwv. Lawv hu nws Fabric Aggregator lossis HGRID. Ib theem qaum ntxiv tau raug qhia los txuas ntau lub chaw cov ntaub ntawv. Qhov kev tsim no tuaj yeem ua tau yog tias peb tsis muaj lub luag haujlwm ntxiv lossis kev hloov pauv ntawm cov interfaces. Yog tias lawv yog cov ntsiab lus kov ntxiv, nws nyuaj. Feem ntau, muaj ntau lub zog thiab ib hom membrane cais qhov sib txawv ntawm cov ntaub ntawv chaw. Tsis muaj lub ntsiab lus hauv kev ua kom cov membrane loj, tab sis yog tias nws xav tau tiag tiag rau qee qhov laj thawj, nws ua rau kev txiav txim siab txog qhov muaj peev xwm ntawm kev tshem tawm, ua kom dav li sai tau thiab hloov mus rau cov tswv. Qhov no yog ua tiav, piv txwv li, los ntawm ntau tus neeg ua haujlwm huab. Lawv muaj overlays, lawv pib los ntawm cov tswv.
Peb pom txoj kev txhim kho dab tsi? Ua ntej tshaj plaws, kev txhim kho kev txhawb nqa rau CI / CD pipeline. Peb xav ya txoj kev peb sim thiab sim txoj kev peb ya. Qhov no tsis ua hauj lwm zoo heev, vim hais tias cov infrastructure yog loj thiab nws yog tsis yooj yim sua kom duplicate nws rau kev xeem. Koj yuav tsum nkag siab yuav ua li cas qhia cov ntsiab lus ntsuas rau hauv cov txheej txheem tsim khoom yam tsis tau tso tseg.
Kev ntsuas zoo dua thiab kev saib xyuas zoo dua yuav luag tsis muaj txiaj ntsig. Cov lus nug tag nrho yog qhov sib npaug ntawm kev siv zog thiab rov qab los. Yog tias koj tuaj yeem ntxiv nws nrog kev siv zog tsim nyog, zoo heev.
Qhib kev khiav hauj lwm systems rau network pab kiag li lawm. Cov txheej txheem zoo dua thiab cov txheej txheem kev sib txuas zoo dua, xws li RIFT. Kev tshawb fawb tseem xav tau rau hauv kev siv cov txheej txheem tswj kev tswj kom zoo dua qub thiab tej zaum qhov kev taw qhia, tsawg kawg ntawm qee cov ntsiab lus, ntawm RDMA kev txhawb nqa hauv pawg.
Saib ntxiv mus rau yav tom ntej, peb xav tau kev tshaj lij topologies thiab tej zaum cov tes hauj lwm uas siv nyiaj tsawg dua. Ntawm cov khoom tshiab, tsis ntev los no tau tshaj tawm txog cov ntaub ntawv thev naus laus zis rau HPC Cray Slingshot, uas yog raws li cov khoom lag luam Ethernet, tab sis nrog kev xaiv ntawm kev siv luv luv headers. Yog li ntawd, nyiaj siv ua haujlwm raug txo.
Txhua yam yuav tsum tau khaws cia kom yooj yim li sai tau, tab sis tsis yooj yim dua. Complexity yog yeeb ncuab ntawm scalability. Kev yooj yim thiab cov qauv tsis tu ncua yog peb cov phooj ywg. Yog tias koj tuaj yeem ua kom tiav qhov chaw, ua nws. Thiab feem ntau, nws yog qhov zoo rau kev koom tes hauv network technologies tam sim no. Muaj ntau yam nthuav tawm mus. Ua tsaug.
Tau qhov twg los: www.hab.com