Ua hoʻomohala mākou i kahi hoʻolālā pūnaewele kikowaena data e hiki ai i ka hoʻolaha ʻana o nā puʻupuʻu computing ʻoi aku ka nui ma mua o 100 tausani mau kikowaena me ka bandwidth bisection peak ma luna o hoʻokahi petabyte i kekona.
Mai ka hōʻike a Dmitry Afanasyev e aʻo ʻoe e pili ana i nā kumu kumu o ka hoʻolālā hou, nā topologies scaling, nā pilikia e kū mai ana me kēia, nā koho no ka hoʻoponopono ʻana iā lākou, nā hiʻohiʻona o ke alahele a me ka hoʻonui ʻana i nā hana o ka mokulele i mua o nā ʻenehana pūnaewele hou i ka "pili paʻa". topologies me ka nui o nā ala ECMP. Eia kekahi, ua ʻōlelo pōkole ʻo Dima e pili ana i ka hoʻonohonoho ʻana o ka pilina o waho, ka papa kino, ka ʻōnaehana cabling a me nā ala e hoʻonui ai i ka hiki.
- Aloha ahiahi kākou a pau! ʻO Dmitry Afanasyev koʻu inoa, he mea hoʻolālā pūnaewele wau ma Yandex a hoʻolālā mua i nā pūnaewele kikowaena data.
ʻO kaʻu moʻolelo e pili ana i ka pūnaewele hou o nā kikowaena data Yandex. He hoʻololi loa ia o ka hoʻolālā i loaʻa iā mākou, akā i ka manawa like aia kekahi mau mea hou. He hōʻike manaʻo nui kēia no ka mea he nui nā ʻike e hoʻopili ʻia i loko o kahi manawa liʻiliʻi. E hoʻomaka mākou ma ke koho ʻana i kahi topology logical. A laila e loaʻa kahi ʻike o ka mokulele hoʻokele a me nā pilikia me ka scalability plane data, kahi koho o nā mea e hiki mai ana ma ka pae kino, a e nānā mākou i kekahi mau hiʻohiʻona o nā mea hana. E hoʻopā iki i ka mea e hana nei i kahi kikowaena data me MPLS, a mākou i kamaʻilio ai i kekahi manawa i hala.
No laila, he aha ka Yandex e pili ana i nā ukana a me nā lawelawe? ʻO Yandex kahi hyperscaler maʻamau. Inā mākou e nānā i nā mea hoʻohana, hana mua mākou i nā noi mea hoʻohana. ʻO nā lawelawe streaming like ʻole a me ka hoʻoili ʻikepili, no ka mea, loaʻa iā mākou nā lawelawe mālama. Inā kokoke i ka hope, a laila e ʻike ʻia nā ukana a me nā lawelawe i laila, e like me ka waiho ʻana i nā mea i hoʻokaʻawale ʻia, ka hoʻopili ʻana i ka ʻikepili a, ʻoiaʻiʻo, nā queues mau. ʻO kekahi o nā ʻano hana nui ʻo MapReduce a me nā ʻōnaehana like, ka hoʻoili kahawai, ke aʻo ʻana i ka mīkini, etc.
Pehea ka ʻōnaehana ma luna o kēia mau mea a pau? Eia hou, he hyperscaler maʻamau mākou, ʻoiai ua kokoke mākou i ka ʻaoʻao hyperscaler liʻiliʻi o ka spectrum. Akā iā mākou nā ʻano a pau. Hoʻohana mākou i nā lako waiwai a me ka hoʻonui ʻana i ka pae i ka hiki. Loaʻa iā mākou ka hoʻohui waiwai piha: ʻaʻole mākou e hana me nā mīkini pākahi, nā pahu pākahi, akā hoʻohui iā lākou i loko o kahi wai nui o nā kumuwaiwai hoʻololi me kekahi mau lawelawe ʻē aʻe e pili ana i ka hoʻolālā ʻana a me ka hoʻokaʻawale ʻana, a hana pū me kēia pūnāwai holoʻokoʻa.
No laila, loaʻa iā mākou ka pae aʻe - ʻo ka ʻōnaehana hana ma ka pae puʻupuʻu computing. He mea koʻikoʻi loa mākou e hoʻomalu piha i ka ʻenehana ʻenehana a mākou e hoʻohana ai. Mālama mākou i nā helu hope (host), pūnaewele a me ka waihona polokalamu.
Loaʻa iā mākou nā kikowaena data nui ma Rusia a ma waho. Hoʻohui ʻia lākou e kahi iwi kuamoʻo e hoʻohana ana i ka ʻenehana MPLS. ʻO kā mākou ʻoihana kūloko i kūkulu ʻia ma IPv6, akā no ka mea pono mākou e lawelawe i nā kaʻa waho e hele mau ana ma luna o IPv4, pono mākou e hāʻawi i nā noi e hele mai ana ma IPv4 i nā kikowaena frontend, a hele iki i waho IPv4- Pūnaewele - no ka laʻana, no ka indexing.
Ua hoʻohana nā mea hou hope loa o nā hoʻolālā kikowaena kikowaena data i nā topologies Clos multi-layer a he L3 wale nō. Ua haʻalele mākou iā L2 i kekahi manawa i hala aku nei a hanu i ka ʻoluʻolu. ʻO ka mea hope loa, loaʻa i kā mākou ʻoihana nā haneli haneli o ka helu helu (server). ʻO ka nui o ka cluster nui i kekahi manawa i hala ma kahi o 10 tausani mau kikowaena. ʻO kēia ma muli o ke ʻano o ka hana ʻana o kēlā mau ʻōnaehana hoʻonohonoho like, nā mea hoʻonohonoho, ka hoʻokaʻawale ʻana i nā kumuwaiwai, a me nā mea ʻē aʻe. Loaʻa iā mākou kahi hana - hiki iā mākou ke kūkulu i nā hale hana pūnaewele e ʻae i ka hoʻohui ʻana i nā kumuwaiwai kūpono i loko o ia pūʻulu.
He aha kā mākou makemake mai kahi kikowaena kikowaena data? ʻO ka mea mua, aia ka nui o ka bandwidth i hoʻokaʻawale ʻia a kūlike. No ka mea, ʻo ka pūnaewele ka iwi kuamoʻo e hiki ai iā mākou ke hoʻohui i nā kumuwaiwai. ʻO ka nui o ka pahuhopu hou ma kahi o 100 tausani mau kikowaena i hoʻokahi pūpū.
ʻO mākou nō hoʻi, makemake mākou i kahi mokulele hoʻokele scalable a paʻa, no ka mea ma luna o kahi ʻoihana nui e ala mai ka nui o nā poʻo mai nā hanana maʻamau, a ʻaʻole mākou makemake i ka mokulele hoʻokele e lawe mai iā mākou i nā poʻo. I ka manawa like, makemake mākou e hōʻemi i ka mokuʻāina i loko. ʻO ka liʻiliʻi o ke kūlana, ʻoi aku ka maikaʻi a me ka paʻa o nā mea a pau, a ʻoi aku ka maʻalahi o ka ʻike.
ʻOiaʻiʻo, pono mākou i ka automation, no ka mea hiki ʻole ke hoʻokele i kēlā ʻano hana me ka lima, a ua hiki ʻole ia no kekahi manawa. Pono mākou i ke kākoʻo hana e like me ka hiki a me ke kākoʻo CI/CD i ka mea hiki ke hāʻawi ʻia.
Me ka nui o nā kikowaena ʻikepili a me nā puʻupuʻu, ua lilo ka hana o ke kākoʻo ʻana i ka hoʻonui ʻana a me ka hoʻonui ʻana me ka ʻole o ka pau ʻana o ka lawelawe. Inā ma nā pūʻulu o ka nui o hoʻokahi kaukani mīkini, kokoke paha i ka ʻumi kaukani mīkini, hiki ke ʻōwili ʻia lākou ma ke ʻano he hana hoʻokahi - ʻo ia hoʻi, ke hoʻolālā nei mākou i ka hoʻonui ʻana i ka ʻōnaehana, a ua hoʻohui ʻia he mau tausani mīkini i hoʻokahi hana. a laila, ʻaʻole e ala koke mai kahi hui o ka nui o hoʻokahi haneli kaukani e like me kēia, ua kūkulu ʻia i kahi manawa. A makemake ʻia e loaʻa nā mea i hoʻopuka ʻia i kēia manawa a pau, nā ʻōnaehana i kau ʻia.
A hoʻokahi koi i loaʻa iā mākou a haʻalele: kākoʻo no ka multitenancy, ʻo ia hoʻi, virtualization a i ʻole ka ʻāpana pūnaewele. I kēia manawa ʻaʻole pono mākou e hana i kēia ma ka pae o ka lole, no ka mea, ua hele ka sharding i nā pūʻali, a ua maʻalahi kēia i ka scaling no mākou. Mahalo iā IPv6 a me kahi wahi kikoʻī nui, ʻaʻole pono mākou e hoʻohana i nā helu ʻelua i loko o ka ʻōnaehana kūloko; ua kū hoʻokahi nā ʻōlelo āpau. A mahalo i ka ʻoiaʻiʻo ua lawe mākou i ka kānana a me ka hoʻokaʻawale ʻana i ka pūnaewele i nā mea hoʻokipa, ʻaʻole pono mākou e hana i nā ʻoihana pūnaewele virtual i nā kikowaena kikowaena data.
ʻO kahi mea koʻikoʻi ka mea a mākou e pono ʻole ai. Inā hiki ke hoʻoneʻe ʻia kekahi mau hana mai ka pūnaewele, ʻoi aku ka maʻalahi o ke ola, a, ma ke ʻano he kānāwai, hoʻonui i ke koho o nā lako a me nā lako polokalamu, e hana maʻalahi loa i nā diagnostics.
No laila, he aha ka mea a mākou e pono ʻole ai, he aha ka mea i hiki iā mākou ke haʻalele, ʻaʻole mau me ka hauʻoli i ka manawa i hana ʻia ai, akā me ka hōʻoluʻolu nui i ka pau ʻana o ke kaʻina hana?
ʻO ka mea mua, haʻalele iā L2. ʻAʻole pono mākou i ka L2, ʻaʻole maoli a i hoʻohālike ʻia. ʻAʻole i hoʻohana nui ʻia ma muli o ka ʻoiaʻiʻo e hoʻokele mākou i ka waihona noi. Hiki ke hoʻonui ʻia kā mākou mau noi, hana lākou me ka L3 addressing, ʻaʻole lākou hopohopo nui i ka hele ʻana o kekahi mea hoʻokahi, ʻōwili wale lākou i kahi mea hou, ʻaʻole pono e ʻōwili ʻia ma ka helu kahiko, no ka mea aia kahi pae ʻokoʻa o ka ʻike lawelawe a me ka nānā ʻana i nā mīkini i loaʻa i ka pūʻulu. ʻAʻole mākou e hāʻawi i kēia hana i ka pūnaewele. ʻO ka hana a ka pūnaewele ʻo ka hoʻopuka ʻana i nā ʻeke mai kahi A a i kahi B.
ʻAʻohe o mākou kūlana e neʻe ai nā ʻōlelo i loko o ka pūnaewele, a pono e nānā ʻia kēia. I nā hoʻolālā he nui kēia e pono ai e kākoʻo i ka neʻe ʻana o VM. ʻAʻole mākou e hoʻohana i ka neʻe ʻana o nā mīkini virtual i loko o ka ʻōnaehana kūloko o ka Yandex nui, a, ʻoi aku, ke manaʻoʻiʻo nei mākou ʻoiai inā hana ʻia kēia, ʻaʻole pono ia e hana me ke kākoʻo pūnaewele. Inā pono e hana ʻia, pono e hana ʻia ma ka pae hoʻokipa, a paʻi i nā ʻōlelo i hiki ke neʻe i loko o nā overlays, i ʻole e hoʻopā a i ʻole e hana i nā hoʻololi hoʻoikaika nui i ka ʻōnaehana routing o ka underlay ponoʻī (transport network) .
ʻO kekahi ʻenehana a mākou e hoʻohana ʻole ai he multicast. Inā makemake ʻoe, hiki iaʻu ke haʻi kikoʻī iā ʻoe i ke kumu. ʻOi aku ka maʻalahi o ke ola, no ka mea inā ua hana kekahi a nānā pono i ke ʻano o ka mokulele multicast control, i nā mea āpau akā maʻalahi loa, he poʻo nui kēia. A ʻo ka mea hou aku, paʻakikī ka loaʻa ʻana o kahi hoʻokō open source maikaʻi, no ka laʻana.
ʻO ka hope, hoʻolālā mākou i kā mākou pūnaewele i ʻole lākou e loli nui. Hiki iā mākou ke hilinaʻi i ka liʻiliʻi o ke kahe o nā hanana waho i ka ʻōnaehana ala.
He aha nā pilikia e kū mai ana a he aha nā palena e pono e noʻonoʻo ʻia ke kūkulu mākou i kahi kikowaena kikowaena data? ʻO ke kumukūʻai, ʻoiaʻiʻo. Scalability, ka pae a mākou e makemake ai e ulu. ʻO ka pono e hoʻonui me ka hoʻōki ʻole i ka lawelawe. Bandwidth, loaʻa. Ka ʻike ʻana i nā mea e hana nei ma ka pūnaewele no ka nānā ʻana i nā ʻōnaehana, no nā hui hana. Kākoʻo Automation - hou, e like me ka mea hiki, no ka mea hiki ke hoʻoholo i nā hana like ʻole ma nā pae like ʻole, me ka hoʻokomo ʻana i nā papa hou. ʻAe, ʻaʻole [hiki] hilinaʻi i nā mea kūʻai aku. ʻOiai i nā wā mōʻaukala like ʻole, ma muli o ka pauku āu e nānā ai, ua maʻalahi a paʻakikī paha kēia kūʻokoʻa. Inā mākou e lawe i kahi ʻāpana o nā ʻāpana ʻāpana pūnaewele, a laila a hiki i kēia manawa he mea kūpono loa ke kamaʻilio e pili ana i ke kūʻokoʻa mai nā mea kūʻai aku, inā makemake mākou i nā chips me ka throughput kiʻekiʻe.
He aha ka topology logical e hoʻohana ai mākou e kūkulu i kā mākou pūnaewele? ʻO kēia kahi Clos pae-nui. ʻO kaʻoiaʻiʻo,ʻaʻohe mea'ē aʻe maoli i kēia manawa. A ʻoi aku ka maikaʻi o ka topology Clos, ʻoiai ke hoʻohālikelike ʻia i nā topologies kiʻekiʻe i ʻoi aku ma kahi o ka hoihoi hoʻonaʻauao i kēia manawa, inā loaʻa iā mākou nā hoʻololi radix nui.
Pehea i hoʻonohonoho ʻia ai kahi ʻoihana Clos multi-level a he aha nā mea like ʻole i kapa ʻia i loko? ʻO ka mua, ua piʻi ka makani, e kuhikuhi iā ʻoe iho ma hea ka ʻākau, ma hea ka hema, ma hea ka hikina, ma hea ke komohana. Kūkulu mau ʻia nā ʻupena o kēia ʻano e ka poʻe i loaʻa nā kaʻa nui komohana-hikina. No nā mea i koe, aia ma luna kahi hoʻololi virtual i hui ʻia mai nā hoʻololi liʻiliʻi. ʻO kēia ka manaʻo nui o ke kūkulu hou ʻana o nā pūnaewele Clos. Lawe mākou i nā mea me kekahi ʻano radix a hoʻohui iā lākou i hiki ke noʻonoʻo ʻia nā mea i loaʻa iā mākou he hoʻololi me kahi radix nui aʻe. Inā makemake ʻoe i nā mea hou aʻe, hiki ke hana hou ʻia ke kaʻina hana.
I nā hihia, no ka laʻana, me nā Clos pae ʻelua, inā hiki ke ʻike maopopo i nā ʻāpana e kū pololei i kaʻu kiʻi, ua kapa ʻia lākou he mokulele. Inā mākou e kūkulu i kahi Clos me nā pae ʻekolu o nā hoʻololi spine (ʻaʻole ia he palena a i ʻole nā hoʻololi ToR a hoʻohana wale ʻia no ka transit), a laila e ʻoi aku ka paʻakikī o nā mokulele; ʻelua mau pae e like me kēia. Kāhea mākou i kahi poloka o ToR a i ʻole nā lau lau a me nā hoʻololi o ka spine pae mua e pili ana iā lākou he Pod. ʻO nā hoʻololi kuamoʻo o ka pae kuamoʻo-1 ma ka piko o ka Pod ka piko o Pod, ka piko o ka Pod. ʻO nā hoʻololi i loaʻa ma ka piko o ka hale hana holoʻokoʻa ka papa luna o ka hale hana, Top of fabric.
ʻOiaʻiʻo, ua kū mai ka nīnau: Ua kūkulu ʻia nā ʻupena Clos no kekahi manawa; ʻo ka manaʻo ponoʻī mai ka manawa o ke kelepona kahiko, nā pūnaewele TDM. Malia paha ua puka mai kekahi mea ʻoi aku ka maikaʻi, hiki paha ke hana maikaʻi ʻia kekahi mea? ʻAe a ʻaʻole. Theoretically ʻae, ma ka hoʻomaʻamaʻa i ka wā e hiki mai ana maopopo ʻole. No ka mea he nui nā topologies hoihoi, hoʻohana ʻia kekahi o lākou i ka hana ʻana, no ka laʻana, hoʻohana ʻia ʻo Dragonfly i nā noi HPC; Aia kekahi mau topologies hoihoi e like me Xpander, FatClique, Jellyfish. Inā ʻoe e nānā i nā hōʻike ma nā ʻaha kūkā e like me SIGCOMM a i ʻole NSDI i kēia manawa, hiki iā ʻoe ke ʻike i ka nui o nā hana ma nā topologies ʻē aʻe i ʻoi aku ka maikaʻi o nā waiwai (kekahi a i ʻole) ma mua o Clos.
Akā hoʻokahi waiwai hoihoi kēia mau topologies. Kāohi ia i kā lākou hoʻokō ʻana i nā kikowaena kikowaena data, a mākou e hoʻāʻo nei e kūkulu ma luna o nā lako waiwai a me ke kālā kūpono loa. Ma kēia mau topologies ʻē aʻe, ʻaʻole hiki ke loaʻa ka hapa nui o ka bandwidth ma nā ala pōkole loa. No laila, nalowale koke mākou i ka manawa e hoʻohana ai i ka mokulele hoʻokele kuʻuna.
ʻO ka manaʻo, ʻike ʻia ka hopena o ka pilikia. ʻO kēia, no ka laʻana, ka hoʻololi ʻana i ka mokuʻāina pili me ka hoʻohana ʻana i ke ala pōkole loa, akā, ʻaʻohe mea e like me nā protocols e hoʻokō ʻia i ka hana ʻana a loaʻa nui i nā lako.
Eia kekahi, no ka mea ʻaʻole hiki ke loaʻa ka hapa nui o ka hiki ma nā ala pōkole loa, pono mākou e hoʻololi i nā mea ʻoi aku ma mua o ka mokulele hoʻokele e koho i kēlā mau ala āpau (a ma ke ala, ʻoi aku ka nui o kēia mokuʻāina i ka mokulele hoʻokele). Pono mākou e hoʻololi i ka mokulele hoʻokele, a, ma ke ʻano he kānāwai, pono ʻelua mau hiʻohiʻona hou. ʻO kēia ka hiki ke hoʻoholo i nā hoʻoholo āpau e pili ana i ka hoʻouna ʻana i ka packet i hoʻokahi manawa, no ka laʻana, ma ka host. ʻO ka ʻoiaʻiʻo, ʻo ia ka hoʻokele kumu, i kekahi manawa ma ka palapala e pili ana i nā ʻoihana pili i kapa ʻia kēia i nā hoʻoholo hoʻoholo holo mua. A ʻo ka hoʻokele adaptive kahi hana e pono ai mākou ma nā ʻenehana pūnaewele, e hoʻoheheʻe ʻia, no ka laʻana, i ka ʻoiaʻiʻo e koho mākou i ka hop aʻe e pili ana i ka ʻike e pili ana i ka haʻahaʻa liʻiliʻi ma ka pila. Ma keʻano he laʻana, hiki i nā koho'ē aʻe.
No laila, hoihoi ke kuhikuhi, akā, auwe, ʻaʻole hiki iā mākou ke hoʻohana i kēia manawa.
ʻAe, ua hoʻoholo mākou i ka Clos logical topology. Pehea mākou e hoʻonui ai? E ʻike kākou pehea e hana ai a pehea e hana ai.
I loko o kahi pūnaewele Clos aia ʻelua mau palena nui e hiki ai iā mākou ke hoʻololi a loaʻa i kekahi mau hopena: ka radix o nā mea a me ka helu o nā pae i ka pūnaewele. Loaʻa iaʻu kahi kiʻi schematic o ka hopena o nā mea ʻelua i ka nui. ʻO ke kūpono, hoʻohui mākou i nā mea ʻelua.
Hiki ke ʻike ʻia ʻo ka laula hope loa o ka pūnaewele Clos ka huahana o nā pae āpau o nā hoʻololi spine o ka radix hema, ehia mau loulou i lalo, pehea ia e lālā ai. Penei mākou e hoʻonui ai i ka nui o ka pūnaewele.
E pili ana i ka hiki, ʻoi loa ma nā hoʻololi ToR, ʻelua mau koho scaling. Hiki iā mākou, ʻoiai e mālama ana i ka topology maʻamau, hoʻohana i nā loulou wikiwiki, a i ʻole hiki iā mākou ke hoʻohui i nā mokulele.
Inā ʻoe e nānā i ka mana hoʻonui o ka pūnaewele Clos (ma ke kihi ʻākau haʻahaʻa) a hoʻi i kēia kiʻi me ka pūnaewele Clos ma lalo nei...
... a laila ʻo kēia ka topology like, akā ma kēia paheʻe ua hiolo ʻoi aku ka paʻa a hoʻopaʻa ʻia nā mokulele o ka hale hana ma luna o kekahi. Ua like ia.
He aha ke ʻano o ka hoʻonui ʻana i kahi pūnaewele Clos i nā helu? Maʻaneʻi au e hāʻawi i kaʻikepili e pili ana i ka laulā kiʻekiʻe loa e hiki ke loaʻa i kahi pūnaewele, he aha ka helu kiʻekiʻe o nā racks, nā hoʻololi ToR a iʻole nā lau lau, ināʻaʻole lākou i loko o nā pahu, hiki iā mākou ke loaʻa ma muli o ka radix o nā hoʻololi a mākou e hoʻohana ai no ka spine -levels, a ehia mau pae a mākou e hoʻohana ai.
Eia ka nui o na racks hiki ke loaa ia makou, ehia mau kikowaena a me ka nui o keia mau mea e pau ai ma ka 20 kW no ka rack. Ma kahi liʻiliʻi ma mua aʻu i ʻōlelo ai ke manaʻo nei mākou i ka nui o ka cluster ma kahi o 100 tausani mau kikowaena.
Hiki keʻikeʻia i loko o kēia hoʻolālā holoʻokoʻa,ʻelua a me ka hapa koho o ka hoihoi. Aia kahi koho me nā ʻāpana ʻelua o nā spines a me nā hoʻololi 64-port, e hāʻule liʻiliʻi. A laila aia nā koho kūpono kūpono no ka 128-port (me radix 128) spine switch me ʻelua pae, a i ʻole nā hoʻololi me radix 32 me ʻekolu pae. A i nā hihia a pau, kahi i ʻoi aku ka nui o nā radix a me nā papa hou aʻe, hiki iā ʻoe ke hana i kahi pūnaewele nui loa, akā inā ʻoe e nānā i ka hoʻohana i manaʻo ʻia, maʻamau he mau gigawatts. Hiki ke kau i ke kaula, akā ʻaʻole paha e loaʻa iā mākou ka nui o ka uila ma kahi kahua. Inā ʻoe e nānā i nā helu helu a me nā ʻikepili lehulehu ma nā kikowaena ʻikepili, hiki iā ʻoe ke loaʻa i nā kikowaena ʻikepili liʻiliʻi loa me ka mana i manaʻo ʻia ʻoi aku ma mua o 150 MW. ʻO nā mea nui ka mea maʻamau i nā kahua kikowaena data, he nui nā kikowaena data nui e pili kokoke ana kekahi i kekahi.
Aia kekahi ʻāpana koʻikoʻi. Inā ʻoe e nānā i ka kolamu hema, ua helu ʻia ka bandwidth hoʻohana ma laila. He mea maʻalahi ke ʻike i loko o kahi kikowaena Clos kahi hapa nui o nā awa i hoʻohana ʻia e hoʻohui i nā hoʻololi i kekahi i kekahi. ʻO ka bandwidth hiki ke hoʻohana ʻia, kahi ʻāpana pono, kahi mea hiki ke hāʻawi ʻia ma waho, i nā kikowaena. ʻO ka mea maʻamau, ke kamaʻilio nei wau e pili ana i nā awa kūlana a pili pono i ka hui. Ma ke ʻano maʻamau, ʻoi aku ka wikiwiki o nā loulou i loko o ka pūnaewele ma mua o nā loulou i nā kikowaena, akā no kēlā me kēia ʻāpana o ka bandwidth, e like me ka hiki iā mākou ke hoʻouna aku i kā mākou lako kikowaena, aia nō kekahi bandwidth i loko o ka pūnaewele ponoʻī. A ʻoi aku ka nui o nā pae a mākou e hana ai, ʻoi aku ka nui o ke kumukūʻai kikoʻī o ka hāʻawi ʻana i kēia kaha i waho.
Eia kekahi, ʻaʻole like kēia hui hou. ʻOiai he pōkole nā kiko, hiki iā mākou ke hoʻohana i kekahi mea e like me DAC (hoʻopili pololei i ke keleawe, ʻo ia hoʻi, nā kaula twinax), a i ʻole multimode optics, ʻoi aku ka nui o ke kālā kūpono a i ʻole. Ke neʻe nei mākou i kahi lōʻihi lōʻihi - ma ke ʻano he kānāwai, ʻo ia nā optics mode hoʻokahi, a ʻo ke kumukūʻai o kēia bandwidth hou e piʻi nui ana.
A eia hou, e hoʻi i ka paheʻe mua, inā mākou e hana i kahi pūnaewele Clos me ka ʻole o ka oversubscription, a laila maʻalahi ke nānā i ke kiʻikuhi, e ʻike i ke ʻano o ke kūkulu ʻia ʻana o ka pūnaewele - e hoʻohui i kēlā me kēia pae o nā hoʻololi spine, e hana hou mākou i ke kaula holoʻokoʻa ma ka lalo. ʻO ka pae hoʻohui - me ka hui like, ka helu like o nā awa ma nā hoʻololi e like me ka pae mua, a me ka helu like o nā transceivers. No laila, makemake nui ʻia e hōʻemi i ka nui o nā pae o nā hoʻololi spine.
Ma muli o kēia kiʻi, maopopo ka makemake maoli mākou e kūkulu i kahi mea e like me nā hoʻololi me kahi radix o 128.
Eia, ma ke kumu, ua like nā mea a pau me kaʻu i ʻōlelo ai; he paheʻe kēia no ka noʻonoʻo ʻana ma hope.
He aha nā koho i hiki iā mākou ke koho e like me ia mau hoʻololi? He lono leʻaleʻa loa ia no mākou i kēia manawa hiki ke kūkulu ʻia ia mau pūnaewele ma nā hoʻololi pākahi hoʻokahi. A ʻoluʻolu loa kēia, he nui nā hiʻohiʻona maikaʻi. No ka laʻana, ʻaneʻane ʻaʻohe hale o loko. ʻO ia hoʻi, ʻoi aku ka maʻalahi o ka haki ʻana. Haʻihaʻi lākou ma nā ʻano ʻano like ʻole, akā naʻe, ua haki loa lākou. I loko o nā mea modular he nui nā hewa (ʻoluʻolu ʻole), i ka wā mai ka manaʻo o nā hoalauna a me ka mokulele hoʻokele me he mea lā e hana ana, akā, no ka laʻana, ua nalowale kekahi hapa o ka lole a ʻaʻole hana. ma ka piha piha. A ʻo ke kaʻa ʻana iā ia ua kaulike ʻia ma muli o ka hana piha ʻana, a hiki iā mākou ke hoʻonui.
A i ʻole, no ka laʻana, kū mai nā pilikia me ka mokulele hope, no ka mea i loko o ka hāmeʻa modular aia pū kekahi SerDes kiʻekiʻe - paʻakikī maoli i loko. ʻO nā hōʻailona ma waena o nā mea hoʻouna i hoʻonohonoho ʻia a ʻaʻole i hoʻonohonoho ʻia. Ma keʻano laulā, ʻo kēlā me kēia mea hana modular e loaʻa ana i kahi helu nui o nā mea, ma ke ʻano he lula, aia ka pūnaewele Clos like i loko ona iho, akā paʻakikī loa ke ʻike. ʻO ka manawa pinepine he paʻakikī no ka mea kūʻai aku iā ia iho e ʻike.
A he nui nā hiʻohiʻona hemahema kahi e hoʻohaʻahaʻa ai ka mea hana, akā ʻaʻole hāʻule i waho o ka topology. No ka nui o kā mākou pūnaewele, hoʻohana ikaika ʻia ke kaupaona ʻana ma waena o nā mea like, maʻa mau ka ʻupena, ʻo ia hoʻi, hoʻokahi ala i hoʻonohonoho pono ʻia nā mea a pau ʻaʻole ʻokoʻa mai ke ala ʻē aʻe, ʻoi aku ka maikaʻi o ka nalo wale ʻana i kekahi. nā mea mai ka topology ma mua o ka hoʻopau ʻana i kahi kūlana i manaʻo ʻia e hana kekahi o lākou, akā ʻaʻole hana kekahi.
ʻO ka hiʻohiʻona maikaʻi aʻe o nā hāmeʻa hoʻokahi-chip ʻo ia ka ulu ʻana o ka maikaʻi a me ka wikiwiki. ʻOi aku ka maikaʻi o ko lākou mana. Inā mākou e lawe i nā hale nui i hui pū ʻia a mākou ma ka pōʻai, a laila ʻoi aku ka maikaʻi o ka mana o kēlā me kēia ʻāpana rack no nā awa o ka wikiwiki hoʻokahi i ʻelua ʻoi aku ka maikaʻi o nā mea modular. ʻO nā mea hana i kūkulu ʻia a puni ka puʻupuʻu hoʻokahi i ʻike ʻia ma mua o nā mea modular a ʻoi aku ka liʻiliʻi o ka ikehu.
Akā, ʻoiaʻiʻo, no ke kumu kēia, aia kekahi mau hemahema. ʻO ka mea mua, ʻoi aku ka liʻiliʻi o ka radix ma mua o nā mea modular. Inā hiki iā mākou ke kiʻi i kahi mea hana i kūkulu ʻia a puni hoʻokahi chip me nā awa 128, a laila hiki iā mākou ke kiʻi i kahi modular me nā haneli he nui i kēia manawa me ka pilikia ʻole.
He liʻiliʻi liʻiliʻi kēia o nā papa hoʻouna mua a, ma ke ʻano he kānāwai, nā mea āpau e pili ana i ka scalability plane data. Nā pale pāpaʻu. A, ma ke ʻano he kānāwai, ʻoi aku ka palena o ka hana. Akā ʻike ʻia inā ʻike ʻoe i kēia mau palena a mālama i ka manawa e kāpae iā lākou a i ʻole e noʻonoʻo pono iā lākou, a laila ʻaʻole makaʻu kēia. ʻO ka ʻoiaʻiʻo o ka liʻiliʻi o ka radix ʻaʻole ia he pilikia ma nā polokalamu me kahi radix o 128 i ʻike hope ʻia; hiki iā mākou ke kūkulu i ʻelua mau ʻāpana spines. Akā ʻaʻole hiki ke kūkulu i kahi mea liʻiliʻi ma mua o ʻelua mea hoihoi iā mākou. Me ka pae hoʻokahi, loaʻa nā pūpū liʻiliʻi loa. ʻO kā mākou mau hoʻolālā a me nā koi mua i ʻoi aku ma mua o lākou.
ʻO ka ʻoiaʻiʻo, inā hiki koke ka hopena ma kahi o ka ʻaoʻao, aia ke ala e hoʻonui ai. ʻOiai ʻo ka pae hope loa (a i ʻole ka mua), ʻo ka pae haʻahaʻa kahi i hoʻopili ʻia ai nā kikowaena ʻo nā hoʻololi ToR a i ʻole nā lau lau, ʻaʻole pono mākou e hoʻopili i hoʻokahi rack iā lākou. No laila, inā hāʻule ka hopena ma kahi o ka hapalua, hiki iā ʻoe ke noʻonoʻo e pili ana i ka hoʻohana ʻana i kahi hoʻololi me kahi radix nui ma ka pae haʻahaʻa a hoʻohui ʻia, no ka laʻana, ʻelua a ʻekolu mau pahu i hoʻokahi kī. He koho nō hoʻi kēia, aia kāna mau kumukūʻai, akā hana maikaʻi a hiki ke lilo i mea hoʻonā maikaʻi inā pono ʻoe e hōʻea i kahi ʻelua o ka nui.
No ka hōʻuluʻulu ʻana, ke kūkulu nei mākou i kahi topology me nā pae ʻelua o nā spines, me nā papa hana ʻewalu.
He aha ka hopena i ka physics? Heluhelu maalahi loa. Inā loaʻa iā mākou nā pae ʻelua o nā spines, a laila ʻekolu wale nō pae o kā mākou hoʻololi, a ke manaʻo nei mākou aia ʻekolu mau ʻāpana kelepona i ka pūnaewele: mai nā kikowaena i nā hoʻololi lau, i ka spine 1, i ka spine 2. ʻO nā koho e hiki iā mākou ke hiki ʻO ka hoʻohana ʻana - he twinax kēia, multimode, hoʻokahi mode. A ma ʻaneʻi pono mākou e noʻonoʻo i ka mea i loaʻa, ehia ke kumu kūʻai, he aha nā ana kino, he aha nā ʻāpana hiki iā mākou ke uhi, a pehea mākou e hoʻomaikaʻi ai.
Ma keʻano o ke kumukūʻai, hiki ke laina i nā mea a pau. ʻOi aku ka maikaʻi o Twinaxes ma mua o nā optics ikaika, ʻoi aku ka liʻiliʻi ma mua o nā transceivers multimode, inā ʻoe e lawe i kēlā me kēia lele mai ka hopena, ʻoi aku ka liʻiliʻi ma mua o kahi awa hoʻololi 100-gigabit. A, e ʻoluʻolu e hoʻomaopopo, ʻoi aku ka liʻiliʻi ma mua o nā optics mode hoʻokahi, no ka mea ma nā mokulele kahi e koi ʻia ai ke ʻano hoʻokahi, ma nā kikowaena ʻikepili no nā kumu he nui ke kūpono ke hoʻohana ʻana i ka CWDM, ʻoiai ʻaʻole maʻalahi ka hana ʻana o ka mode parallel single (PSM). me, loaʻa nā pūʻolo nui loa i nā fibers, a inā mākou e nānā i kēia mau ʻenehana, loaʻa mākou ma kahi o ka hierarchy kumu kūʻai.
Hoʻokahi memo: akā naʻe, ʻaʻole hiki ke hoʻohana i nā awa multimode 100 a 4x25 i wehe ʻia. Ma muli o nā hiʻohiʻona hoʻolālā o nā transceivers SFP28, ʻaʻole i ʻoi aku ka liʻiliʻi ma mua o 28 Gbit QSFP100. A ʻaʻole maikaʻi loa kēia wehe ʻana no ka multimode.
ʻO kekahi palena ʻē aʻe ma muli o ka nui o nā puʻupuʻu computing a me ka helu o nā kikowaena, huli ko mākou mau kikowaena ʻikepili i ke kino nui. 'O ia ho'i, e hana 'ia ma ka li'ili'i loa o ho'okahi lele me ka singlemod. Eia hou, ma muli o ka nui kino o nā Pods, ʻaʻole hiki ke holo i ʻelua spans o twinax (nā kaula keleawe).
ʻO ka hopena, inā mākou e koho i ke kumukūʻai a noʻonoʻo i ka geometry o kēia hoʻolālā, loaʻa iā mākou hoʻokahi span o twinax, hoʻokahi span o multimode a me hoʻokahi span o singlemode me CWDM. E noʻonoʻo ana kēia i nā ala hoʻonui.
ʻO kēia ke ʻano o kēia manawa, kahi a mākou e hele nei a me ka mea hiki. Ua maopopo, ma ka liʻiliʻi, pehea e neʻe ai i ka 50-Gigabit SerDes no ka multimode a me ka singlemode. Eia kekahi, inā ʻoe e nānā i ka mea i loko o nā transceivers hoʻokahi-mode i kēia manawa a i ka wā e hiki mai ana no 400G, pinepine a hiki i ka 50G SerDes mai ka ʻaoʻao uila, hiki i ka 100 Gbps i kēlā me kēia ala ke hele i ka optics. No laila, hiki i ka neʻe ʻana i 50, aia kahi hoʻololi i 100 Gigabit SerDes a me 100 Gbps i kēlā me kēia ala, no ka mea, e like me nā ʻōlelo hoʻohiki a nā mea kūʻai aku, ua manaʻo koke ʻia ko lākou loaʻa. ʻO ka manawa i wikiwiki loa ai ka 50G SerDes, me he mea lā, ʻaʻole lōʻihi loa, no ka mea, ke holo nei nā kope mua o 100G SerDes kokoke i ka makahiki e hiki mai ana. A ma hope o kekahi manawa ma hope o ia mea e lilo paha lākou i kālā kūpono.
Hoʻokahi nuance e pili ana i ke koho ʻana i ka physics. Ma ke kumu, hiki iā mākou ke hoʻohana i nā awa 400 a i ʻole 200 Gigabit me 50G SerDes. Akā, ʻaʻole ia he manaʻo nui, no ka mea, e like me kaʻu i ʻōlelo ai ma mua, makemake mākou i kahi radix nui loa ma nā hoʻololi, i loko o ke kumu, ʻoiaʻiʻo. Makemake mākou i ka 128. A inā he liʻiliʻi ka mana chip a hoʻonui mākou i ka wikiwiki o ka loulou, a laila emi maoli ka radix, ʻaʻohe hana mana.
A hiki iā mākou ke hoʻonui i ka hiki ke hoʻohana i nā mokulele, ʻaʻohe kumu kūʻai kūikawā; hiki iā mākou ke hoʻohui i ka helu o nā mokulele. A inā e nalowale mākou i ka radix, pono mākou e hoʻokomo i kahi pae hou, no laila, i ke kūlana o kēia manawa, me ka nui o ka hiki ke loaʻa i kēlā me kēia chip, ua ʻoi aku ka maikaʻi o ka hoʻohana ʻana i nā awa 100-gigabit, no ka mea, ʻae lākou iā ʻoe. e loaa ka radix nui.
ʻO ka nīnau aʻe pehea e hoʻonohonoho ʻia ai ka physics, akā mai ka manaʻo o ka ʻōnaehana kelepona. ʻIke ʻia ua hoʻonohonoho ʻia ma kahi ʻano ʻakaʻaka. ʻO ka hoʻopaʻa ʻana ma waena o nā lau-hoʻololi a me nā spines pae mua - ʻaʻole nui nā loulou ma laila, kūkulu ʻia nā mea āpau ma ke ʻano maʻalahi. Akā inā lawe mākou i hoʻokahi mokulele, ʻo ka mea e hana i loko, pono mākou e hoʻohui i nā spines āpau o ka pae mua me nā spines āpau o ka pae ʻelua.
Eia kekahi, ma ke ʻano he kānāwai, aia kekahi mau makemake no ka nānā ʻana i loko o ka kikowaena data. No ka laʻana, makemake maoli mākou e hoʻohui i nā kaula i loko o kahi puʻupuʻu a huki iā lākou i hiki i kahi papa hoʻopaʻa kiʻekiʻe kiʻekiʻe ke hele holoʻokoʻa i hoʻokahi panel patch, no laila ʻaʻohe zoo e pili ana i ka lōʻihi. Ua hoʻoholo mākou i kēia pilikia. Inā ʻoe e nānā mua i ka topology logical, hiki iā ʻoe ke ʻike he kūʻokoʻa nā mokulele, hiki ke kūkulu ʻia kēlā me kēia mokulele ma kāna iho. Akā ke hoʻohui mākou i kahi pūʻulu a makemake mākou e kauo i ka panel patch holoʻokoʻa i loko o kahi panel patch, pono mākou e hoʻohui i nā mokulele like ʻole i loko o hoʻokahi puʻupuʻu a hoʻokomo i kahi hoʻolālā waena ma ke ʻano o nā hoʻohui ʻokoʻa optical e hoʻopaʻa hou iā lākou mai ke ʻano o ka hui ʻana. ma kekahi ʻāpana, pehea lākou e hōʻiliʻili ai ma kekahi ʻāpana. Mahalo i kēia, loaʻa iā mākou kahi hiʻohiʻona maikaʻi: ʻaʻole hele nā hoʻololi paʻakikī āpau ma mua o nā pā. Ke pono ʻoe e hoʻohui i kahi mea ikaika loa, "e wehe i nā mokulele," e like me ka mea i kapa ʻia i kekahi manawa ma nā pūnaewele ʻo Clos, ua hoʻopaʻa ʻia i loko o hoʻokahi pahu. ʻAʻole mākou i hoʻokaʻawale loa, i lalo i nā loulou pākahi, ke hoʻololi ʻana ma waena o nā pā.
ʻO kēia ke ʻano o ka nānā ʻana mai ka manaʻo o ka hoʻonohonoho pono o ka ʻōnaehana kelepona. Ma ke kiʻi ma ka hema, ua hōʻike ʻia nā ʻāpana ʻulaʻula i nā poloka o nā hoʻololi kuamoʻo pae mua, ʻewalu mau ʻāpana i kēlā me kēia, a me ʻehā pūʻolo o nā kaula e hele mai ana mai o lākou mai, e hele ana a hui pū me nā pūʻolo e hele mai ana mai nā poloka o nā kuamoʻo-2. .
Hōʻike nā ʻāpana liʻiliʻi i nā ʻāpana. Aia ma ka ʻaoʻao hema ka haʻihaʻi ʻana o kēlā me kēia ʻāpana, ʻo ia ka 512 by 512 port cross-connect module e hoʻopaʻa hou ana i nā kaula i hiki ke komo i loko o ka pahu hoʻokahi, kahi hoʻokahi mokulele spine-2. A ma ka ʻaoʻao ʻākau, ʻoi aku ka kikoʻī o ka scan o kēia kiʻi e pili ana i kekahi mau Pods ma ka pae spine-1, a pehea e hoʻopili ʻia ai i kahi hoʻohui keʻa, pehea e hiki mai ai i ka pae spine-2.
Penei kona ano. ʻAʻole i hui piha ʻia ka spine-2 (ma ka ʻaoʻao hema) a me ke kū kea. ʻO ka mea pōʻino, ʻaʻole nui ka ʻike ma laila. Ke kau ʻia nei kēia hale holoʻokoʻa i kēia manawa ma kekahi o kā mākou mau kikowaena data nui e hoʻonui ʻia nei. He hana kēia e holo nei, ʻoi aku ka maikaʻi o ka nānā ʻana, e hoʻopiha maikaʻi ʻia.
He nīnau koʻikoʻi: ua koho mākou i ka topology logical a kūkulu i ka physics. He aha ka hopena i ka mokulele hoʻomalu? Ua ʻike maikaʻi ʻia mai ka ʻike hana, aia kekahi mau hōʻike e pili ana i nā protocol state states maikaʻi, he mea hauʻoli ia e hana pū me lākou, akā naʻe, ʻaʻole lākou e hoʻonui maikaʻi i kahi topology pili. A aia kekahi kumu nui e pale ai i kēia - ʻo ia ke ʻano o ka hana ʻana o ka waikahe i nā protocol state state link. Inā ʻoe e lawe wale i ka algorithm wai kahe a nānā i ke ʻano o kā mākou pūnaewele, ʻike ʻoe e loaʻa ana kahi fanout nui loa i kēlā me kēia ʻanuʻu, a e kahe wale i ka mokulele hoʻokele me nā mea hou. Ma ke ʻano kikoʻī, ua hui maikaʻi ʻole ia mau topologies me ka algorithm wai kahe maʻamau i nā protocol state link.
ʻO ka koho e hoʻohana i ka BGP. Pehea e hoʻomākaukau pono ai i wehewehe ʻia ma RFC 7938 e pili ana i ka hoʻohana ʻana o BGP i nā kikowaena data nui. He mea maʻalahi nā manaʻo kumu: ka helu liʻiliʻi o nā prefixes no kēlā me kēia host a me ka helu liʻiliʻi o nā prefixes ma ka pūnaewele, e hoʻohana i ka hōʻuluʻulu inā hiki, a kāpae i ka hopu ala. Makemake mākou i ka hoʻolaha ʻana i nā mea hou, ka mea i kapa ʻia ʻo Valley Free. Makemake mākou e kau ʻia nā mea hou i hoʻokahi manawa i ko lākou hele ʻana ma ka pūnaewele. Inā mai lalo lākou, piʻi lākou i luna, ʻaʻole i ʻoi aku ma mua o hoʻokahi manawa. ʻAʻole pono he zigzags. He ino loa ka zigzags.
No ka hana ʻana i kēia, hoʻohana mākou i kahi hoʻolālā maʻalahi e hoʻohana i nā mīkini BGP lalo. ʻO ia hoʻi, hoʻohana mākou i ka eBGP e holo ana ma ka loulou kūloko, a ua hāʻawi ʻia nā ʻōnaehana autonomous e like me kēia: kahi ʻōnaehana autonomous ma ToR, kahi ʻōnaehana autonomous ma nā poloka holoʻokoʻa o nā spine-1 hoʻololi o hoʻokahi Pod, a me kahi ʻōnaehana autonomous ma ka Top holoʻokoʻa. o ka lole. ʻAʻole paʻakikī ke nānā a ʻike ʻo ke ʻano maʻamau o BGP hāʻawi iā mākou i ka hāʻawi ʻana i nā mea hou a mākou e makemake ai.
ʻO ka mea maʻamau, pono e hoʻolālā ʻia ka ʻōlelo a me ka hōʻuluʻulu ʻana i kūpono i ke ala i kūkulu ʻia ai ke alahele, i mea e hōʻoia ai i ka paʻa o ka mokulele hoʻokele. Hoʻopili ʻia ka ʻōlelo L3 i ka lawe ʻana i ka topology, no ka mea, ʻaʻole hiki ke hoʻokō i ka hōʻuluʻulu ʻana; me ka ʻole o kēia, e kolo nā ʻōlelo pilikino i loko o ka ʻōnaehana ala. A ʻo kekahi mea ʻē aʻe, ʻo ka hui ʻana, ʻaʻole naʻe, ʻaʻole i hui maikaʻi loa me ka multi-alanui, no ka mea, ke loaʻa iā mākou ka multi-alanui a loaʻa iā mākou ka hōʻuluʻulu ʻana, maikaʻi nā mea a pau, ke olakino ka pūnaewele holoʻokoʻa, ʻaʻohe hemahema i loko. ʻO ka mea pōʻino, ke ʻike koke ʻia nā hemahema i ka pūnaewele a nalowale ka symmetry o ka topology, hiki iā mākou ke hele i kahi i hoʻolaha ʻia ai ka ʻāpana, ʻaʻole hiki iā mākou ke hele i kahi e pono ai mākou e hele. No laila, ʻoi aku ka maikaʻi o ka hōʻuluʻulu ʻana i kahi ʻaʻohe ala hou aʻe, i kā mākou hihia ʻo ia nā hoʻololi ToR.
ʻOiaʻiʻo, hiki ke hōʻuluʻulu, akā mālama pono. Inā hiki iā mākou ke hana i ka hoʻokaʻawale ʻana i ka wā e hiki mai ana nā hemahema o ka pūnaewele. Akā, he hana paʻakikī kēia, ua noʻonoʻo mākou inā hiki ke hana i kēia, inā hiki ke hoʻohui i ka automation hou, a me nā mīkini mokuʻāina palena ʻole e kiki pololei i ka BGP e kiʻi i ka ʻano makemake. ʻO ka mea pōʻino, ʻaʻole maopopo a paʻakikī ka hoʻoponopono ʻana i nā hihia kihi, a ʻaʻole i hoʻoholo maikaʻi ʻia kēia hana ma ka hoʻopili ʻana i nā mea hoʻopili waho i BGP.
Ua hana ʻia nā hana hoihoi loa ma kēia ʻano i loko o ke ʻano o ka protocol RIFT, e kūkākūkā ʻia ma ka hōʻike aʻe.
ʻO kekahi mea koʻikoʻi ʻo ke ʻano o ka hoʻonui ʻana o nā mokulele data i nā topologies paʻa, kahi i loaʻa iā mākou ka nui o nā ala ʻē aʻe. I kēia hihia, hoʻohana ʻia kekahi mau hale ʻikepili hou aʻe: nā pūʻulu ECMP, ʻo ia hoʻi ka wehewehe ʻana i nā hui Next Hop.
Ma kahi pūnaewele hana maʻamau, me ka ʻole o ka hāʻule ʻole, ke piʻi mākou i ka topology Clos, ua lawa ka hoʻohana ʻana i hoʻokahi hui wale nō, no ka mea, ʻo nā mea āpau ʻaʻole kūloko i wehewehe ʻia e ka paʻamau, hiki iā mākou ke piʻi i luna. Ke hele mākou mai luna a lalo i ka hema, a laila ʻaʻole ECMP nā ala āpau, he ala ala hoʻokahi lākou. Ua maikaʻi nā mea a pau. ʻO ka pilikia, a ʻo ka ʻokoʻa o ka topology Clos maʻamau inā mākou e nānā i ka Top o ka lole, i kēlā me kēia mea, hoʻokahi wale nō ala i kekahi mea ma lalo. Inā loaʻa nā hemahema ma kēia ala, a laila lilo kēia mea kikoʻī ma luna o ka hale hana i mea kūpono ʻole no kēlā mau prefix e waiho nei ma hope o ke ala haʻihaʻi. Akā no ke koena he kūpono ia, a pono mākou e hoʻokaʻawale i nā hui ECMP a hoʻokomo i kahi mokuʻāina hou.
He aha ke ʻano o ka scalability plane data ma nā polokalamu hou? Inā mākou e hana i ka LPM (lōʻihi prefix match), maikaʻi loa nā mea a pau, ma luna o 100k prefixes. Inā mākou e kamaʻilio e pili ana i nā hui Next Hop, a laila ʻoi aku ka maikaʻi o nā mea āpau, 2-4 tausani. Inā mākou e kamaʻilio e pili ana i kahi papaʻaina i loaʻa kahi wehewehe o Next Hops (a i ʻole nā mea pili), a laila aia kēia ma kahi mai 16k a 64k. A hiki i kēia ke lilo i pilikia. A eia mākou i hele mai i kahi digression hoihoi: he aha ka mea i hiki i MPLS i nā kikowaena data? Ma ke kumu, ua makemake mākou e hana.
ʻElua mea i hana ʻia. Ua hana mākou i ka micro-segmentation ma nā pūʻali; ʻaʻole pono mākou e hana ia ma ka pūnaewele. ʻAʻole maikaʻi loa me ke kākoʻo mai nā mea kūʻai like ʻole, a ʻoi aku hoʻi me nā hoʻokō hāmama ma nā pahu keʻokeʻo me MPLS. A ʻo MPLS, ma ka liʻiliʻi o kāna hoʻokō kuʻuna, ʻaʻole naʻe, hui maikaʻi ʻole me ECMP. A ʻo ia ke kumu.
ʻO kēia ke ʻano o ka hoʻolālā hoʻouna ECMP no IP. Hiki i ka nui o nā prefix ke hoʻohana i ka hui like a me ka poloka Next Hops like (a i ʻole nā mea pili, hiki ke kapa ʻia kēia ma nā palapala like ʻole no nā mea like ʻole). ʻO ke kumu, ua wehewehe ʻia kēia ma ke awa puka a me ka mea e kākau hou ai i ka helu MAC i mea e hiki ai i ka Next Hop pololei. No ka IP he mea maʻalahi nā mea a pau, hiki iā ʻoe ke hoʻohana i ka nui o nā prefixes no ka hui like, ʻo ia ka pahu Next Hops.
Hōʻike ka papa hana MPLS maʻamau, ma muli o ke kikowaena puka, hiki ke kākau hou ʻia ka lepili i nā waiwai like ʻole. No laila, pono mākou e mālama i kahi hui a me kahi poloka Next Hops no kēlā me kēia lepili hoʻokomo. A ʻo kēia, auwe, ʻaʻole nui.
He mea maʻalahi ke ʻike i kā mākou hoʻolālā e pono ai mākou ma kahi o 4000 ToR hoʻololi, ʻo ka laulā kiʻekiʻe he 64 ala ECMP, inā mākou e neʻe aku mai ka spine-1 a i ka spine-2. ʻAʻole hiki iā mākou ke komo i hoʻokahi papa o nā pūʻulu ECMP, inā hoʻokahi wale nō prefix me ToR e hele, a ʻaʻole mākou e komo i ka papaʻaina Next Hops.
ʻAʻole pau ka manaʻolana, no ka mea, ʻo nā hale hana e like me Segment Routing e pili ana i nā lepili honua. Ma ke ʻano maʻamau, hiki ke hāʻule hou i kēia mau poloka Next Hops. No ka hana ʻana i kēia, pono ʻoe i kahi hana ʻano kāleka hihiu: e lawe i kahi lepili a kākau hou i ka mea like me ka ʻole o ka waiwai kikoʻī. Akā naʻe, ʻaʻole i loaʻa kēia i nā hoʻokō i loaʻa.
A ʻo ka hope, pono mākou e lawe mai i nā kaʻa waho i loko o ke kikowaena data. Pehea e hana ai? Ma mua, ua hoʻokomo ʻia ke kaʻa i ka pūnaewele Clos mai luna mai. ʻO ia hoʻi, aia nā ala ala e pili ana i nā mea hana a pau ma luna o ka lole. Hana maikaʻi kēia hoʻonā i nā liʻiliʻi a i ʻole ka nui waena. ʻO ka mea pōʻino, i mea e hoʻouna ai i nā kaʻa symmetrically i ka pūnaewele holoʻokoʻa ma kēia ala, pono mākou e hōʻea i ka manawa like i nā mea āpau o ka Top o ka lole, a i ka wā i ʻoi aku ma mua o hoʻokahi haneli o lākou, ʻike ʻia he pono mākou i kahi nui. radix ma nā ala ala. Ma keʻano holoʻokoʻa, ʻoi aku ka nui o ke kālā, no ka mea, ʻoi aku ka maikaʻi o nā ala ala, ʻoi aku ka pipiʻi o nā awa ma luna o lākou, a ʻaʻole nani loa ka hoʻolālā.
ʻO kahi koho ʻē aʻe ʻo ka hoʻomaka ʻana i kēlā mau kaʻa mai lalo. He mea maʻalahi ka hōʻoia ʻana ua kūkulu ʻia ka topology Clos i ke ʻano o ka hele ʻana mai lalo mai, ʻo ia hoʻi, mai ka ʻaoʻao ToR, ua puʻunaue like ʻia ma waena o nā pae ma ka ʻaoʻao o luna o ka lole i ʻelua iterations, e hoʻouka ana i ka pūnaewele holoʻokoʻa. No laila, hoʻolauna mākou i kahi ʻano Pod kūikawā, Edge Pod, e hāʻawi ana i ka pilina o waho.
Aia kekahi koho hou aʻe. ʻO kēia ka hana a Facebook, no ka laʻana. Kapa lākou iā Fabric Aggregator a i ʻole HGRID. Hoʻokomo ʻia kahi pae spine hou e hoʻohui i nā kikowaena ʻikepili lehulehu. Hiki i kēia hoʻolālā ke loaʻa ʻole nā hana hou a i ʻole nā hoʻololi encapsulation ma nā interface. Inā he mau wahi hoʻopā hou lākou, paʻakikī. ʻO ka maʻamau, ʻoi aku ka nui o nā hana a me kahi ʻano membrane e hoʻokaʻawale ana i nā ʻāpana like ʻole o ke kikowaena data. ʻAʻohe kumu o ka hana ʻana i kahi membrane nui, akā inā pono maoli ia no kekahi kumu, a laila kūpono ke noʻonoʻo i ka hiki ke lawe ʻia, e hoʻonui i ka laulā a me ka hoʻoili ʻana i nā pūʻali. Hana ʻia kēia, no ka laʻana, e nā mea hoʻokele kapua he nui. Loaʻa iā lākou nā uhi, hoʻomaka lākou mai nā pūʻali.
He aha nā manawa hoʻomohala a mākou e ʻike ai? ʻO ka mea mua, hoʻomaikaʻi i ke kākoʻo no ka pipeline CI / CD. Makemake mākou e lele i ke ala a mākou e hoʻāʻo ai a hoʻāʻo i ke ala a mākou e lele ai. ʻAʻole maikaʻi loa kēia, no ka mea he nui ka ʻōnaehana a ʻaʻole hiki ke hoʻopaʻa ʻia no nā hoʻokolohua. Pono ʻoe e hoʻomaopopo pehea e hoʻokomo ai i nā mea hoʻāʻo i loko o ka ʻōnaehana hana me ka ʻole e hāʻule.
ʻO ka ʻoi aku ka maikaʻi o ka mea kani a me ka nānā ʻana i ʻoi aku ka maikaʻi ʻole. ʻO ka nīnau holoʻokoʻa he kaulike o ka hoʻoikaika a me ka hoʻi. Inā hiki iā ʻoe ke hoʻohui me ka hoʻoikaika kūpono, maikaʻi loa.
Wehe i nā ʻōnaehana hana no nā ʻenehana pūnaewele. ʻOi aku ka maikaʻi o nā protocols a me nā ʻōnaehana ala ʻoi aku ka maikaʻi, e like me RIFT. Pono nō hoʻi ka noiʻi ʻana i ka hoʻohana ʻana i nā papa hana hoʻomalu ʻoi aku ka maikaʻi a me ka hoʻolauna ʻana paha, ma kahi liʻiliʻi loa, o ke kākoʻo RDMA i loko o ka pūʻulu.
Ke nānā hou aku nei i ka wā e hiki mai ana, pono mākou i nā topologies kiʻekiʻe a me nā ʻoihana pūnaewele e hoʻohana liʻiliʻi i luna. ʻO nā mea hou, ua hoʻopuka hou ʻia e pili ana i ka ʻenehana lole no HPC Cray Slingshot, i hoʻokumu ʻia ma ka Ethernet commodity, akā me ke koho o ka hoʻohana ʻana i nā poʻomanaʻo pōkole loa. ʻO ka hopena, ua hoʻemi ʻia ka overhead.
Pono e mālamaʻia nā mea a pau e like me ka maʻalahi, akā,ʻaʻole i maʻalahi. ʻO ka paʻakikī ka ʻenemi o ka scalability. ʻO ka maʻalahi a me nā hale maʻamau ko mākou mau hoaaloha. Inā hiki iā ʻoe ke hoʻonui i kekahi wahi, e hana. A ma keʻano laulā, maikaʻi ke komo ʻana i nā ʻenehana pūnaewele i kēia manawa. Nui nā mea hoihoi e hana nei. Mahalo iā ʻoe.
Source: www.habr.com