M'nkhani ziwiri zoyamba, ndinadzutsa nkhani ya automation ndi sketched out chimango chake, chachiwiri ndinapanga kubwerera mu maukonde virtualization, monga njira yoyamba automating kasinthidwe misonkhano.
Tsopano ndi nthawi yojambula chithunzi cha intaneti.

Ngati simukudziwa kukhazikitsa ma network a data center, ndiye ndikupangira kuti muyambe nkhani za iwo.

Nkhani zonse:

• 0. ADSM. Gawo Zero. Kukonzekera
• 1. ADSM. Gawo loyamba (lomwe liri pambuyo pa ziro). Network virtualization
• 2. ADSM. Gawo lachiwiri. Kapangidwe ka maukonde

Zochita zomwe zafotokozedwa mndandandawu ziyenera kugwiritsidwa ntchito pamtundu uliwonse wa intaneti, kukula kulikonse, ndi ogulitsa osiyanasiyana (osati). Komabe, ndizosatheka kufotokoza chitsanzo chapadziko lonse lapansi chakugwiritsa ntchito njirazi. Chifukwa chake, ndiyang'ana kwambiri kamangidwe kamakono ka netiweki ya DC: Kloz Factory.
Tidzachita DCI pa MPLS L3VPN.

Netiweki ya Overlay imayenda pamwamba pa netiweki yakuthupi kuchokera kwa wolandila (iyi ikhoza kukhala VXLAN ya OpenStack kapena Tungsten Fabric kapena china chilichonse chomwe chimangofunika kulumikizana kwa IP kuchokera pa netiweki).

Pankhaniyi, timapeza zochitika zosavuta zopangira makina, chifukwa tili ndi zida zambiri zomwe zimapangidwira mofanana.

Tisankha DC yozungulira mu vacuum:

• Mtundu umodzi wamapangidwe paliponse.
• Ogulitsa awiri omwe amapanga ndege ziwiri zochezera.
• DC mmodzi ali ngati wina ngati nandolo ziwiri mumtsuko.

Zamkatimu

• Physical topology
• Njira
• IP plan
• Laba
• Pomaliza
• maulalo othandiza

Lolani Wopereka Ntchito LAN_DC, mwachitsanzo, apangire makanema ophunzitsira okhudzana ndi kupulumuka m'zikepe zomata.

Mu megacities izi ndizodziwika kwambiri, kotero mumafunikira makina ambiri akuthupi.

Choyamba, ndifotokoza za netiweki pafupifupi momwe ndikanafunira. Ndiyeno ndifewetsera labu.

Physical topology

Malo

LAN_DC idzakhala ndi ma DC 6:

• Russia (RU):
  • Moscow (msk)
  • Kazan (kzn)

• Spain (SP):
  • Barcelona (bcn)
  • Malaga (mlg)

• China (CN):
  • Shanghai (sha)
  • Xi'an (sia)

Mkati mwa DC (Intra-DC)

Ma DC onse ali ndi maukonde ofanana olumikizirana mkati kutengera Clos topology.
Ndi maukonde amtundu wanji wa Clos ndipo chifukwa chiyani ali osiyana nkhani.

DC iliyonse ili ndi ma rack 10 okhala ndi makina, adzawerengedwa ngati A, B, C Ndipo kotero.

Choyika chilichonse chili ndi makina 30. Sadzatisangalatsa.

Komanso mu chipika chilichonse pali chosinthira chomwe makina onse amalumikizidwa - izi ndi Pamwamba pa Rack switch - ToR kapena ayi, malinga ndi fakitale ya Clos, tidzayitcha Leaf.

General chithunzi cha fakitale.

Tidzawatcha iwo xxx-tsambaYkumene xxx - zilembo zitatu chidule DC, ndi Y - nambala ya siriyo. Mwachitsanzo, kzn-tsamba11.

M'nkhani zanga ndidzilola kugwiritsa ntchito mawu akuti Leaf ndi ToR mosasamala ngati mawu ofanana. Komabe, tiyenera kukumbukira kuti izi siziri choncho.
ToR ndi chosinthira chomwe chimayikidwa mu rack yomwe makina amalumikizidwa.
Leaf ndi gawo la chipangizo pa netiweki yakuthupi kapena kusintha koyambira koyamba malinga ndi Cloes topology.
Ndiko kuti, Leaf != ToR.
Chifukwa chake Leaf itha kukhala EndofRaw switch, mwachitsanzo.
Komabe, mkati mwa dongosolo la nkhaniyi tidzawatengabe ngati mawu ofanana.

Kusintha kulikonse kwa ToR kumalumikizidwa ndi masiwichi anayi apamwamba kwambiri - Mphepete. Rack imodzi mu DC imaperekedwa kwa Spines. Tizitchula mofananamo: xxx- msanaY.

Choyikamo chomwechi chidzakhala ndi zida zama network zolumikizirana pakati pa ma routers a DC - 2 okhala ndi MPLS m'bwalo. Koma mokulira, awa ndi ma ToR omwewo. Ndiye kuti, kuchokera pakuwona kusintha kwa Spine, ToR wamba yokhala ndi makina olumikizidwa kapena rauta ya DCI zilibe kanthu - kungotumiza.

Ma ToR apadera oterewa amatchedwa Mphepete-tsamba. Tidzawatcha iwo xxx-mbaliY.

Zidzawoneka chonchi.

Pachithunzi pamwambapa, ndinayika m'mphepete ndi tsamba pamlingo womwewo. Ma network osanjika atatu Adatiphunzitsa kuti tiziganiza zokweza (chifukwa chake mawuwo) ngati ma uplink. Ndipo apa zikuwoneka kuti "uplink" ya DCI imabwerera pansi, yomwe kwa ena imaswa pang'ono malingaliro anthawi zonse. Pankhani ya ma netiweki akulu, pomwe ma data agawidwa m'magawo ang'onoang'ono - POD's (Point Of Delivery), onetsani mosiyana Mphepete-PODza DCI ndi mwayi wopita kumanetiweki akunja.

Kuti mukhale omasuka m'tsogolomu, ndidzajambulabe Edge pamwamba pa Spine, pamene tidzakumbukira kuti palibe nzeru pa Spine ndipo palibe kusiyana pamene mukugwira ntchito ndi Leaf wamba ndi Edge-leaf (ngakhale pangakhale ma nuances apa. , koma zambiri Izi ndi zoona).

Dongosolo la fakitale yokhala ndi masamba am'mphepete.

Utatu wa Leaf, Spine ndi Edge amapanga maukonde a Underlay kapena fakitale.

Ntchito ya fakitale yama network (werengani Underlay), monga tafotokozera kale nkhani yapitayi, kwambiri, zosavuta kwambiri - kupereka IP kulumikizana pakati pa makina onse mkati mwa DC yemweyo ndi pakati pawo.
Ichi ndichifukwa chake maukonde amatchedwa fakitale, monga, mwachitsanzo, fakitale yosinthira mkati mwa mabokosi amtaneti, omwe mutha kuwerenga zambiri mu Chithunzi cha SDSM14.

Kawirikawiri, topology yotereyi imatchedwa fakitale, chifukwa nsalu yomasulira imatanthawuza nsalu. Ndipo n'zovuta kuvomereza:

Fakitale ndi L3 kwathunthu. Palibe VLAN, palibe Broadcast - tili ndi opanga mapulogalamu abwino kwambiri ku LAN_DC, amadziwa kulemba mapulogalamu omwe amakhala mu L3 paradigm, ndipo makina enieni safuna Kusamuka Kwamoyo ndikusunga adilesi ya IP.

Ndipo kachiwiri: yankho la funso chifukwa fakitale ndi chifukwa L3 ali osiyana nkhani.

DCI - Data Center Interconnect (Inter-DC)

DCI idzakonzedwa pogwiritsa ntchito Edge-Leaf, ndiye kuti, ndi malo athu otulukira mumsewu waukulu.
Kuti zikhale zosavuta, timaganiza kuti ma DC amalumikizana wina ndi mzake ndi maulalo achindunji.
Tiyeni tichotse kulumikizana kwakunja kuti tisaganizire.

Ndikudziwa kuti nthawi iliyonse ndikachotsa chinthu, ndimachepetsa maukonde. Ndipo tikamasinthiratu maukonde athu osamveka, zonse zikhala bwino, koma zenizeni padzakhala ndodo.
Izi ndi Zow. Komabe, mfundo ya mndandandawu ndi kulingalira ndi kugwiritsira ntchito njira, osati kuthetsa mavuto ongoganizira.

Pa Edge-Leafs, choyikapo pansi chimayikidwa mu VPN ndikufalikira kudzera pamsana wa MPLS (ulalo womwewo wachindunji).

Ichi ndiye chithunzi chapamwamba chomwe timapeza.

Njira

Pakuyenda mkati mwa DC tidzagwiritsa ntchito BGP.
Pa thunthu la MPLS OSPF + LDP.
Kwa DCI, ndiye kuti, kukonza kulumikizana mobisa - BGP L3VPN pa MPLS.

General routing scheme

Palibe OSPF kapena ISIS (protocol yololeza yoletsedwa ku Russian Federation) pafakitale.

Izi zikutanthauza kuti sipadzakhala Zodziwikiratu kapena kuwerengera njira zazifupi kwambiri - zolemba zokha (zokha - tikukamba za makina apa) kukhazikitsa ndondomeko, oyandikana nawo ndi ndondomeko.

BGP routing scheme mkati mwa DC

Chifukwa chiyani BGP?

Pa mutu uwu pali RFC yonse dzina la Facebook ndi Arista, lomwe limafotokoza momwe amanga chachikulu kwambiri ma data center network pogwiritsa ntchito BGP. Zimawerengedwa ngati zopeka, ndikupangira madzulo osowa.

Ndipo palinso gawo lonse m'nkhani yanga yoperekedwa kwa izi. Ndikukutengerani kuti ndi Ndikutumiza.

Komabe, mwachidule, palibe IGP yomwe ili yoyenera kwa maukonde a malo akuluakulu a deta, kumene chiwerengero cha zipangizo zamakono zimapita ku zikwizikwi.

Kuphatikiza apo, kugwiritsa ntchito BGP kulikonse kumakupatsani mwayi kuti musataye nthawi pothandizira ma protocol angapo osiyanasiyana ndi kulumikizana pakati pawo.

Dzanja pa mtima, mu fakitale yathu, amene ndi mkulu mlingo wa Mwina sadzakula mofulumira, OSPF adzakhala okwanira maso. Awa ndiye mavuto a megascalers ndi titans amtambo. Koma tiyeni tingoyerekeza zotulutsa zochepa zomwe tikuzifuna, ndipo tidzagwiritsa ntchito BGP, monga momwe Pyotr Lapukhov adapereka.

Ndondomeko Zamayendedwe

Pa masiwichi a Leaf, timalowetsa ma prefixes kuchokera ku maukonde a Underlay network kupita ku BGP.
Tidzakhala ndi gawo la BGP pakati aliyense a Leaf-Spine pair, momwe ma prefixes a Underlay awa adzalengezedwa pamaneti kumbuyo ndi mtsogolo.

Mkati mwa malo amodzi a data tidzagawira zomwe tidatumiza ku ToRe. Pa Edge-Leafs tidzawaphatikiza ndikuwalengeza ku ma DC akutali ndikuwatumiza ku TORs. Ndiye kuti, ToR iliyonse idzadziwa momwe mungafikire ku ToR ina mu DC yomweyo komanso komwe polowera ndikufika ku ToR mu DC ina.

Mu DCI, misewu idzaperekedwa ngati VPNv4. Kuti tichite izi, pa Edge-Leaf, mawonekedwe opita ku fakitale adzayikidwa mu VRF, tiyeni titchule UNDERLAY, ndipo oyandikana nawo Spine pa Edge-Leaf adzakwera mkati mwa VRF, ndi pakati pa Edge-Leafs mu VPNv4- banja.

Tiletsanso kulengezanso njira zolandilidwa kuchokera ku spines kubwerera kwa iwo.

Pa Leaf ndi Spine sitidzalowetsa ma Loopbacks. Timangofunikira kuti tidziwe ID ya router.

Koma pa Edge-Leafs timazilowetsa ku Global BGP. Pakati pa ma adilesi a Loopback, Edge-Leafs adzakhazikitsa gawo la BGP mu IPv4 VPN-banja wina ndi mnzake.

Tidzakhala ndi OSPF + LDP msana pakati pa zida za EDGE. Chilichonse chili muzoni imodzi. Kusintha kophweka kwambiri.

Ichi ndi chithunzi chokhala ndi mayendedwe.

BGP ASN

Edge-Leaf ASN

Edge-Leafs adzakhala ndi ASN imodzi mu ma DC onse. Ndikofunika kuti pakhale iBGP pakati pa Edge-Leafs, ndipo tisatengeke ndi zovuta za eBGP. Lolani kuti ikhale 65535. Zoonadi, izi zikhoza kukhala chiwerengero cha AS public.

Mtengo wa ASN

Pa Spine tidzakhala ndi ASN imodzi pa DC. Tiyeni tiyambire apa ndi nambala yoyamba kuchokera pagulu la AS - 64512, 64513 Ndi zina zotero.

Chifukwa chiyani ASN pa DC?

Tiyeni tigawe funso ili pawiri:

• Chifukwa chiyani ma ASN ali ofanana pamizere yonse ya DC imodzi?
• Chifukwa chiyani amasiyana mu ma DC osiyanasiyana?

Chifukwa chiyani ma ASN omwewo ali pamizere yonse ya DC imodzi?

Izi ndi zomwe AS-Path ya Underlay njira pa Edge-Leaf idzawoneka ngati:
[leafX_ASN, spine_ASN, edge_ASN]
Mukayesa kulengezanso ku Spine, imataya chifukwa AS (Spine_AS) ili kale pamndandanda.

Komabe, mkati mwa DC timakhutira kwathunthu kuti njira za Underlay zomwe zimakwera ku Mphepete mwa nyanja sizidzatha kutsika. Kuyankhulana konse pakati pa makamu mkati mwa DC kuyenera kuchitika mkati mwa msana.

Panthawi imodzimodziyo, njira zophatikizana za ma DC ena zidzafika mosavuta ku ToRs - AS-Path yawo idzakhala ndi ASN 65535 yokha - chiwerengero cha AS Edge-Leafs, chifukwa ndi kumene analengedwa.

Chifukwa chiyani amasiyana mu ma DC osiyanasiyana?

Mwamwayi, tingafunike kukoka Loopback ndi makina ena apakatikati a DCs.

Mwachitsanzo, pa wolandirayo tidzayendetsa Route Reflector kapena VNGW yomweyo (Virtual Network Gateway), yomwe idzatsekera ndi TopR kudzera pa BGP ndikulengeza za loopback yake, yomwe iyenera kupezeka kuchokera ku ma DC onse.

Chifukwa chake izi ndi momwe AS-Path yake idzawoneka:
[VNF_ASN, leafX_DC1_ASN, spine_DC1_ASN, edge_ASN, spine_DC2_ASN, leafY_DC2_ASN]

Ndipo pasakhale ma ASN obwereza kulikonse.

Izi ndizo, Spine_DC1 ndi Spine_DC2 ziyenera kukhala zosiyana, monga leafX_DC1 ndi leafY_DC2, zomwe ndizomwe tikuyandikira.

Monga mukudziwira, pali ma hacks omwe amakulolani kuvomereza mayendedwe omwe ali ndi ma ASN obwereza ngakhale ali ndi njira yopewera kulupu (malo olowera pa Cisco). Ndipo imakhala ndi ntchito zovomerezeka. Koma ichi ndi kusiyana komwe kungatheke pakukhazikika kwa netiweki. Ndipo ine ndekha ndinagwa mu izo kangapo.

Ndipo ngati tili ndi mwayi wosagwiritsa ntchito zinthu zoopsa, tidzaugwiritsa ntchito.

Leaf ASN

Tidzakhala ndi ASN payekha pa Leaf switch iliyonse pamanetiweki.
Timachita izi pazifukwa zomwe zaperekedwa pamwambapa: AS-Path yopanda malupu, kasinthidwe ka BGP popanda ma bookmark.

Kuti njira zapakati pa Masamba zidutse bwino, AS-Path iyenera kuwoneka motere:
[leafX_ASN, spine_ASN, leafY_ASN]
pomwe leafX_ASN ndi leafY_ASN zingakhale zabwino kukhala zosiyana.

Izi ndizofunikiranso pazomwe zikuchitika ndikulengeza kwa VNF loopback pakati pa DCs:
[VNF_ASN, leafX_DC1_ASN, spine_DC1_ASN, edge_ASN, spine_DC2_ASN, leafY_DC2_ASN]

Tidzagwiritsa ntchito 4-byte ASN ndikuipanga kutengera ASN ya Spine ndi nambala yosinthira Leaf, monga chonchi: Spine_ASN.0000X.

Ichi ndi chithunzi ndi ASN.

IP plan

Chofunika kwambiri, tiyenera kugawa maadiresi kuti tigwirizane ndi zotsatirazi:

1. Kuyika ma adilesi a netiweki pakati pa ToR ndi makina. Ayenera kukhala apadera mkati mwa netiweki yonse kuti makina aliwonse azilumikizana ndi ena. Kukwanira kwakukulu 10/8. Pachiyika chilichonse pali / 26 yokhala ndi posungira. Tigawa / 19 pa DC ndi / 17 pachigawo chilichonse.
2. Lumikizani ma adilesi pakati pa Leaf/Tor ndi Spine.

   Ndikufuna kuwapatsa algorithmically, ndiko kuti, kuwerengera kuchokera ku mayina a zipangizo zomwe ziyenera kulumikizidwa.

   Zikhale ... 169.254.0.0/16.
   Ndiko kuti 169.254.00X.Y/31kumene X - Nambala ya msana, Y - P2P network / 31.
   Izi zikuthandizani kuti mutsegule ma rack 128, mpaka 10 Spines mu DC. Maulalo adilesi amatha (ndipo) abwerezedwa kuchokera ku DC kupita ku DC.

3. Timakonza mphambano ya Spine-Edge-Leaf pama subnets 169.254.10X.Y/31, pomwe chimodzimodzi X - Nambala ya msana, Y - P2P network / 31.
4. Lumikizani ma adilesi kuchokera ku Edge-Leaf kupita ku MPLS backbone. Apa zinthu ndizosiyana - malo omwe zidutswa zonse zimalumikizidwa kukhala chitumbuwa chimodzi, kotero kugwiritsanso ntchito ma adilesi omwewo sikungagwire ntchito - muyenera kusankha subnet yaulere yotsatira. Chifukwa chake, tiyeni titenge ngati maziko 192.168.0.0/16 ndipo tidzawachotsa m'menemo (omasulidwawo).
5. Ma adilesi a Loopback. Tidzapereka gawo lonse kwa iwo 172.16.0.0/12.
   • Leaf - / 25 pa DC - ma rack 128 ofanana. Tigawa / 23 pachigawo chilichonse.
   • Msana - / 28 pa DC - mpaka 16 Spine. Tiyeni tigawane /26 pachigawo chilichonse.
   • Edge-Leaf - / 29 pa DC - mpaka mabokosi 8. Tiyeni tigawane /27 pachigawo chilichonse.

Ngati tilibe magawo okwanira mu DC (ndipo sipadzakhala - timati ndi ma hyperscalers), timangosankha chipika chotsatira.

Ichi ndi chithunzi chokhala ndi ma adilesi a IP.

Loopbacks:

Mawu Oyamba
Chipangizo udindo
Chigawo
ДЦ

172.16.0.0/23
m'mphepete
 
 

172.16.0.0/27
ru
 

172.16.0.0/29
msk

172.16.0.8/29
kzn

172.16.0.32/27
sp
 

172.16.0.32/29
bcn

172.16.0.40/29
mlg

172.16.0.64/27
cn
 

172.16.0.64/29
sha

172.16.0.72/29
sia

172.16.2.0/23
msana
 
 

172.16.2.0/26
ru
 

172.16.2.0/28
msk

172.16.2.16/28
kzn

172.16.2.64/26
sp
 

172.16.2.64/28
bcn

172.16.2.80/28
mlg

172.16.2.128/26
cn
 

172.16.2.128/28
sha

172.16.2.144/28
sia

172.16.8.0/21
tsamba
 
 

172.16.8.0/23
ru
 

172.16.8.0/25
msk

172.16.8.128/25
kzn

172.16.10.0/23
sp
 

172.16.10.0/25
bcn

172.16.10.128/25
mlg

172.16.12.0/23
cn
 

172.16.12.0/25
sha

172.16.12.128/25
sia

Pansi:

Mawu Oyamba
Chigawo
ДЦ

10.0.0.0/17
ru
 

10.0.0.0/19
msk

10.0.32.0/19
kzn

10.0.128.0/17
sp
 

10.0.128.0/19
bcn

10.0.160.0/19
mlg

10.1.0.0/17
cn
 

10.1.0.0/19
sha

10.1.32.0/19
sia

Laba

Mavenda awiri. Network imodzi. Chithunzi cha ADSM.

Juniper + Arista. Ubuntu. Eva wakale wabwino.

Kuchuluka kwazinthu pa seva yathu yeniyeni ku Mirana kukadali kochepa, kotero kuti tigwiritse ntchito tidzagwiritsa ntchito maukonde omwe amaphweka mpaka malire.

Malo awiri a data: Kazan ndi Barcelona.

• Misana iwiri iliyonse: Juniper ndi Arista.
• Torasi imodzi (Leaf) mu iliyonse - Juniper ndi Arista, ndi wolandira m'modzi wolumikizidwa (tiyeni titenge Cisco IOL yopepuka pa izi).
• Node imodzi ya Edge-Leaf iliyonse (pakali pano Juniper yokha).
• Kusinthana kumodzi kwa Cisco kuti alamulire onse.
• Kuphatikiza pa mabokosi a netiweki, makina owongolera akuyenda. Kuyendetsa Ubuntu.
  Imakhala ndi zida zonse, idzayendetsa machitidwe a IPAM/DCIM, zolemba zambiri za Python, Ansible ndi china chilichonse chomwe tingafunike.

Kukonzekera kwathunthu pazida zonse za netiweki, zomwe tidzayesa kupanganso pogwiritsa ntchito makina opangira okha.

Pomaliza

Kodi izonso zimavomerezedwa? Kodi ndilembe mawu omaliza aafupi m'nkhani iliyonse?

Choncho tinasankha magawo atatu Kloz network mkati mwa DC, popeza tikuyembekezera anthu ambiri Kum'mawa ndi Kumadzulo ndipo tikufuna ECMP.

Netiweki idagawika kukhala yakuthupi (pansi) ndi pafupifupi (yophimba). Panthawi imodzimodziyo, kuphimba kumayambira kuchokera kwa wolandira - potero kumachepetsa zofunikira za underlay.

Tidasankha BGP ngati njira yoyendetsera maukonde pamanetiweki chifukwa cha scalability komanso kusinthasintha kwa mfundo.

Tidzakhala ndi ma node osiyana okonzekera DCI - Edge-leaf.
Msana udzakhala ndi OSPF + LDP.
DCI idzakhazikitsidwa kutengera MPLS L3VPN.
Pa maulalo a P2P, tiwerengera ma adilesi a IP motengera mayina a chipangizocho.
Tidzagawa ma loopbacks molingana ndi ntchito ya zida ndi malo awo motsatizana.
Zoyambira pansi - pamasinthidwe a Leaf okha motsatizana ndi malo awo.

Tiyerekeze kuti pakadali pano tilibe zida zoyikapo.
Chifukwa chake, masitepe athu otsatirawa ndikuwawonjezera ku machitidwe (IPAM, inventory), kukonza zofikira, kupanga masinthidwe ndikutumiza.

M'nkhani yotsatira tidzakambirana ndi Netbox - njira yowerengera ndi kasamalidwe ka IP space mu DC.

Zikomo

• Andrey Glazkov aka @glazgoo kuti awonenso ndikuwongolera
• Alexander Klimenko aka @v00lk kuti awonenso ndikusintha
• Artyom Chernobay kwa KDPV

Source: www.habr.com