A cikin kasidu biyu na farko, na tayar da batun sarrafa kansa kuma na zayyana tsarinsa, a cikin na biyu na yi ja da baya a cikin hanyar sadarwa ta hanyar sadarwa, a matsayin hanyar farko ta sarrafa sarrafa tsarin ayyuka.
Yanzu lokaci ya yi da za a zana zane na cibiyar sadarwar jiki.

Idan baku saba da kafa cibiyoyin sadarwar bayanai ba, to ina bada shawarar farawa da karfi labarai game da su.

Duk batutuwa:

• 0. ADSM. Sashe na Zero. Tsare-tsare
• 1. ADSM. Kashi na daya (wanda ke bayan sifili). Ƙwararren hanyar sadarwa
• 2. ADSM. Kashi na biyu. Tsarin hanyar sadarwa

Ayyukan da aka bayyana a cikin wannan jerin ya kamata su dace da kowane nau'i na hanyar sadarwa, kowane girman, tare da kowane nau'i na masu sayarwa (ba). Koyaya, ba shi yiwuwa a kwatanta misali na duniya na aikace-aikacen waɗannan hanyoyin. Don haka, zan mai da hankali kan gine-ginen zamani na cibiyar sadarwar DC: Kamfanin Kloz.
Za mu yi DCI akan MPLS L3VPN.

Cibiyar sadarwa mai rufi tana gudana a saman cibiyar sadarwar jiki daga mai watsa shiri (wannan zai iya zama OpenStack's VXLAN ko Tungsten Fabric ko wani abu da ke buƙatar haɗin haɗin IP kawai daga cibiyar sadarwa).

A wannan yanayin, muna samun yanayi mai sauƙi mai sauƙi don sarrafa kansa, saboda muna da kayan aiki da yawa waɗanda aka daidaita su a cikin hanya ɗaya.

Za mu zaɓi DC mai siffar zobe a cikin sarari:

• Daya zane version a ko'ina.
• Dillalai biyu suna samar da jiragen sadarwa guda biyu.
• Daya DC kamar wani kamar wake biyu ne a cikin kwasfa.

Abubuwa

• Topology na jiki
• Hanyar hanya
• IP shirin
• Laba
• ƙarshe
• hanyoyi masu amfani

Bari Mai Ba da Sabis ɗin mu LAN_DC, alal misali, ya dauki nauyin bidiyon horarwa game da tsira a cikin lif masu makale.

A cikin megacities wannan ya shahara sosai, don haka kuna buƙatar injina da yawa.

Da farko, zan kwatanta hanyar sadarwar kamar yadda nake so ta kasance. Sannan zan sauƙaƙa shi don lab.

Topology na jiki

Wurare

LAN_DC zai sami 6 DCs:

• Rasha (RU):
  • Moscow (msk)
  • Kazan (kzn)

• Spain (SP):
  • Barcelona (bcn)
  • Malaga (mlg)

• China (CN):
  • Shanghai (sha)
  • Xi'an (Sia)

Ciki DC (Intra-DC)

Duk DCs suna da cibiyoyin sadarwa iri ɗaya na ciki dangane da Clos topology.
Wadanne nau'ikan cibiyoyin sadarwar Clos ne kuma me yasa suke cikin daban labarin.

Kowane DC yana da racks 10 tare da injuna, za a ƙidaya su azaman A, B, C Da sauransu.

Kowane taragon yana da injuna 30. Ba za su sha'awar mu ba.

Har ila yau, a cikin kowane rak ɗin akwai maɓalli wanda aka haɗa dukkan injuna - wannan shine Saman Rack switch - ToR ko in ba haka ba, dangane da masana'antar Clos, za mu kira shi leaf.

Babban zane na masana'anta.

Za mu kira su XXX- ganyeYinda XXX - gajartawar harafi uku DC, da Y - lambar serial. Misali, kzn-leaf11.

A cikin kasidu na zan ƙyale kaina in yi amfani da kalmomin Leaf da ToR maimakon ma'ana kamar ma'ana. Duk da haka, dole ne mu tuna cewa ba haka lamarin yake ba.
ToR shine maɓalli da aka shigar a cikin rakiyar da ake haɗa inji.
Leaf shine aikin na'ura a cikin hanyar sadarwa ta jiki ko matakin farko na canji dangane da Cloes topology.
Wato Leaf != ToR.
Don haka Leaf na iya zama canjin EndofRaw, alal misali.
Koyaya, a cikin tsarin wannan labarin har yanzu za mu ɗauke su azaman ma'ana.

Kowane jujjuyawar ToR bi da bi yana haɗe zuwa manyan juzu'ai huɗu na tara - kashin baya. An ware tara guda ɗaya a cikin DC don Spines. Za mu sanya masa suna kamar haka: XXX- kashin bayaY.

Rack iri ɗaya zai ƙunshi kayan aikin cibiyar sadarwa don haɗin kai tsakanin DC - 2 Router tare da MPLS a kan jirgin. Amma gabaɗaya, waɗannan ToR iri ɗaya ne. Wato, daga ra'ayi na Spine switches, ToR na yau da kullun tare da injunan da aka haɗa ko na'ura mai ba da hanya tsakanin hanyoyin sadarwa don DCI ba kome ba - kawai turawa.

Ana kiran irin waɗannan ToR na musamman Ganye-leaf. Za mu kira su XXX-gejiY.

Zai yi kama da wannan.

A cikin zanen da ke sama, a zahiri na sanya baki da ganye akan matakin guda. Classic hanyoyin sadarwa mai Layer uku Sun koya mana yin la'akari da haɓakawa (saboda haka kalmar) azaman haɓakawa. Kuma a nan ya bayyana cewa DCI "uplink" ya koma ƙasa, wanda wasu kaɗan ya karya tunanin da aka saba. A cikin manyan cibiyoyin sadarwa, idan aka raba cibiyoyin bayanai zuwa ƙananan raka'a - POD's (Point Of Delivery), haskaka daban Edge-POD's don DCI da samun dama ga cibiyoyin sadarwa na waje.

Don sauƙin fahimta a nan gaba, har yanzu zan zana Edge akan Spine, yayin da za mu tuna cewa babu hankali akan Spine kuma babu bambance-bambance yayin aiki tare da Leaf na yau da kullun da Edge-leaf (ko da yake akwai iya zama nuances a nan. , amma gabaɗaya Wannan gaskiya ne).

Tsarin masana'anta mai ganye-gefen.

Triniti na Leaf, Spine da Edge suna samar da hanyar sadarwa ta Underlay ko masana'anta.

Ayyukan masana'antar cibiyar sadarwa (karanta Underlay), kamar yadda muka riga muka bayyana a ciki fitowar karshe, sosai, mai sauqi qwarai - don samar da haɗin IP tsakanin injina duka a cikin DC ɗaya da tsakanin su.
Shi ya sa ake kiran cibiyar sadarwa da masana’anta, kamar misali, masana’antar sauya sheka a cikin akwatunan sadarwa na zamani, wanda za ka iya karantawa a ciki. SDSM14.

Gabaɗaya, ana kiran irin wannan topology masana'anta, saboda masana'anta a cikin fassarar yana nufin masana'anta. Kuma yana da wuya a ƙi yarda:

Ma'aikata gaba daya L3. Babu VLAN, babu Watsa shirye-shirye - muna da irin waɗannan masu shirye-shirye masu ban mamaki a LAN_DC, sun san yadda ake rubuta aikace-aikacen da ke rayuwa a cikin yanayin L3, kuma injunan kama-da-wane ba sa buƙatar ƙaura ta Live tare da adana adireshin IP.

Kuma sake: amsar tambayar dalilin da yasa masana'anta da kuma dalilin da yasa L3 ke cikin daban labarin.

DCI - Cibiyar Sadarwar Bayanai (Inter-DC)

Za a shirya DCI ta amfani da Edge-Leaf, wato, su ne hanyar fita zuwa babbar hanya.
Don sauƙi, muna ɗauka cewa an haɗa DCs da juna ta hanyar haɗin kai tsaye.
Bari mu ware haɗin waje daga la'akari.

Ina sane da cewa duk lokacin da na cire wani sashi, nakan sauƙaƙa cibiyar sadarwa sosai. Kuma idan muka sarrafa hanyar sadarwar mu ta atomatik, komai zai yi kyau, amma a kan ainihin za a sami crutches.
Wannan gaskiya ne. Duk da haka, manufar wannan silsilar ita ce yin tunani da yin aiki kan hanyoyin da za a bi, ba wai a jarumtaka wajen warware matsalolin da aka zayyana ba.

A kan Edge-Leafs, ana sanya ƙasa a cikin VPN kuma ana watsa shi ta kashin bayan MPLS (haɗin kai tsaye ɗaya).

Wannan shine babban zane-zane da muke samu.

Hanyar hanya

Don kewaya cikin DC za mu yi amfani da BGP.
Akan gangar jikin MPLS OSPF+LDP.
Don DCI, wato, tsara haɗin kai a cikin ƙasa - BGP L3VPN akan MPLS.

Gabaɗaya tsarin tuƙi

Babu OSPF ko ISIS (ka'idar da aka haramta a cikin Tarayyar Rasha) a masana'anta.

Wannan yana nufin cewa ba za a sami ganowa ta atomatik ko lissafin mafi guntu hanyoyin ba - kawai jagora (a zahiri atomatik - muna magana akan sarrafa kansa anan) kafa ƙa'ida, unguwanni da manufofi.

Tsarin birki na BGP a cikin DC

Me yasa BGP?

A kan wannan batu akwai duk RFC mai suna Facebook da Arista, wanda ke ba da labarin yadda ake ginawa babba sosai cibiyoyin sadarwar bayanai ta amfani da BGP. Yana karanta kusan kamar almara, Ina ba da shawarar shi sosai don maraice maraice.

Kuma akwai ma wani sashe gabaɗaya a cikin labarin da aka sadaukar don wannan. Ina zan kai ku kuma ina aikawa.

Amma har yanzu, a takaice, babu IGP da ya dace da cibiyoyin sadarwa na manyan cibiyoyin bayanai, inda adadin na'urorin sadarwar ke gudana zuwa dubbai.

Bugu da ƙari, yin amfani da BGP a ko'ina zai ba ku damar ɓata lokaci kan tallafawa ka'idoji daban-daban da aiki tare a tsakanin su.

Hannun zuciya, a cikin ma'aikata, wanda tare da babban matakin yiwuwar ba zai yi girma da sauri ba, OSPF zai isa ga idanu. Waɗannan su ne ainihin matsalolin megascalers da titan girgije. Amma bari mu yi tunanin kawai don 'yan sakewa da muke bukata, kuma za mu yi amfani da BGP, kamar yadda Pyotr Lapukhov ya yi wasiyya.

Manufofin Gudanarwa

Akan Canjin Leaf, muna shigo da prefixes daga hanyoyin sadarwa na Underlay zuwa BGP.
Za mu sami zaman BGP tsakanin kowane Leaf-Spine biyu, wanda a ciki za a sanar da waɗannan prefixes na Underlay akan hanyar sadarwar gaba da gaba.

A cikin cibiyar bayanai ɗaya, za mu rarraba ƙayyadaddun bayanan da muka shigo da su cikin ToRe. A kan Edge-Leafs za mu tara su kuma mu sanar da su zuwa DCs masu nisa kuma mu aika su zuwa TORs. Wato, kowane ToR zai san ainihin yadda ake zuwa wani ToR a cikin DC guda kuma inda wurin shiga shine zuwa ToR a wani DC.

A cikin DCI, za a watsa hanyoyin kamar VPNv4. Don yin wannan, a kan Edge-Leaf, za a sanya ma'amala zuwa masana'anta a cikin VRF, bari mu kira shi UNDERLAY, kuma unguwar da Spine akan Edge-Leaf za ta tashi a cikin VRF, kuma tsakanin Edge-Leafs a cikin VPNv4- iyali.

Za mu kuma haramta sake sanar da hanyoyin da aka karɓa daga kashin baya zuwa gare su.

A kan Leaf da Spine ba za mu shigo da Loopbacks ba. Muna buƙatar su kawai don ƙayyade ID na Router.

Amma akan Edge-Leafs muna shigo da shi cikin BGP na Duniya. Tsakanin adiresoshin Loopback, Edge-Leafs za su kafa zaman BGP a cikin IPv4 VPN-iyali tare da juna.

Za mu sami kashin baya na OSPF+LDP tsakanin na'urorin EDGE. Komai yana cikin shiyya daya. Tsari mai sauqi qwarai.

Wannan shine hoton tare da hanya.

BGP ASN

Edge-Leaf ASN

Edge-Leafs za su sami ASN guda ɗaya a duk DCs. Yana da mahimmanci cewa akwai iBGP tsakanin Edge-Leafs, kuma ba mu sami kama cikin nuances na eBGP ba. Bari ya zama 65535. A gaskiya, wannan zai iya zama adadin jama'a AS.

Kashin baya ASN

A kan Spine za mu sami ASN guda ɗaya kowace DC. Bari mu fara anan da lamba ta farko daga kewayon AS - 64512, 64513 Da sauransu.

Me yasa ASN akan DC?

Mu raba wannan tambayar gida biyu:

• Me yasa ASNs iri ɗaya suke akan duk kashin baya na DC ɗaya?
• Me yasa suka bambanta a cikin DCs daban-daban?

Me yasa ASNs iri ɗaya suke akan duk spines na DC ɗaya?

Wannan shine abin da hanyar AS-Path na hanyar Underlay akan Edge-Leaf zai yi kama da:
[leafX_ASN, spine_ASN, edge_ASN]
Lokacin da kuka yi ƙoƙarin sake tallata shi zuwa Spine, zai watsar da shi saboda AS (Spine_AS) ya riga ya shiga cikin jerin.

Koyaya, a cikin DC mun gamsu da cewa hanyoyin ƙarƙashin ƙasa waɗanda ke hawa zuwa Edge ba za su iya sauka ba. Duk sadarwa tsakanin runduna a cikin DC dole ne ya faru a cikin matakin kashin baya.

A lokaci guda kuma, haɗaɗɗun hanyoyin wasu DCs a kowane hali za su iya isa ga ToRs - hanyar AS-Path ɗin su kawai za su sami ASN 65535 - adadin AS Edge-Leafs, domin a nan ne aka ƙirƙira su.

Me yasa suka bambanta a cikin DCs daban-daban?

A ka'ida, ƙila mu buƙaci ja Loopback da wasu injunan kama-da-wane na sabis tsakanin DCs.

Misali, a kan mai watsa shiri za mu gudu Route Reflector ko sama VNGW (Virtual Network Gateway), wanda zai kulle tare da TopR ta hanyar BGP kuma ya sanar da madaidaicin sa, wanda ya kamata a samu daga duk DCs.

Don haka wannan shine yadda hanyar AS-Path zata yi kama:
[VNF_ASN, leafX_DC1_ASN, spine_DC1_ASN, edge_ASN, spine_DC2_ASN, leafY_DC2_ASN]

Kuma kada a sami kwafin ASN a ko'ina.

Wato Spine_DC1 da Spine_DC2 dole ne su bambanta, kamar leafX_DC1 da leafY_DC2, wanda shine ainihin abin da muke gabatowa.

Kamar yadda ƙila kuka sani, akwai hacks waɗanda ke ba ku damar karɓar hanyoyi tare da kwafin ASNs duk da tsarin rigakafin madauki (allowas-in akan Cisco). Kuma har ma yana da halalcin amfani. Amma wannan shi ne yuwuwar gibi a cikin kwanciyar hankali na hanyar sadarwa. Kuma ni da kaina na fada cikinsa sau biyu.

Kuma idan muka sami damar kada mu yi amfani da abubuwa masu haɗari, za mu yi amfani da su.

Farashin ASN

Za mu sami ASN guda ɗaya akan kowane Canjin Leaf a duk hanyar sadarwar.
Muna yin haka saboda dalilan da aka bayar a sama: Hanyar AS-Path ba tare da madaukai ba, tsarin BGP ba tare da alamun shafi ba.

Domin hanyoyin tsakanin Leafs su wuce sumul, hanyar AS-Path yakamata yayi kama da haka:
[leafX_ASN, spine_ASN, leafY_ASN]
inda leafX_ASN da leafY_ASN zai yi kyau su bambanta.

Ana kuma buƙatar wannan don yanayin tare da sanarwar madauki na VNF tsakanin DCs:
[VNF_ASN, leafX_DC1_ASN, spine_DC1_ASN, edge_ASN, spine_DC2_ASN, leafY_DC2_ASN]

Za mu yi amfani da 4-byte ASN kuma mu samar da shi bisa la'akari da ASN na Spine da Leaf canza lambar, wato, kamar haka: Spine_ASN.0000X.

Wannan shine hoton tare da ASN.

IP shirin

Ainihin, muna buƙatar ware adireshi don mahaɗa masu zuwa:

1. Ƙarƙashin adiresoshin cibiyar sadarwa tsakanin ToR da na'ura. Dole ne su kasance na musamman a cikin dukkan hanyar sadarwa ta yadda kowace na'ura za ta iya sadarwa da kowace irin. Kyakkyawan dacewa 10/8. Ga kowane rak ɗin akwai /26 tare da ajiyar ajiya. Za mu kasaftawa / 19 kowace DC da / 17 a kowace yanki.
2. Adireshin haɗi tsakanin Leaf/Tor da Spine.

   Ina so in sanya su algorithmically, wato, lissafin su daga sunayen na'urorin da ake buƙatar haɗawa.

   Bari ya kasance ... 169.254.0.0/16.
   Wato 169.254.00X.Y/31inda X - Lambar kashin baya, Y - P2P cibiyar sadarwa /31.
   Wannan zai ba ku damar ƙaddamar da har zuwa racks 128, kuma har zuwa Spines 10 a cikin DC. Ana iya maimaita adiresoshin haɗin (kuma za a) daga DC zuwa DC.

3. Muna tsara mahadar Spine-Edge-Leaf a kan subnets 169.254.10X.Y/31, inda daidai yake X - Lambar kashin baya, Y - P2P cibiyar sadarwa /31.
4. Haɗin adiresoshin daga Edge-Leaf zuwa MPLS kashin baya. Anan halin da ake ciki ya ɗan bambanta - wurin da aka haɗa dukkan sassan zuwa kek ɗaya, don haka sake amfani da adiresoshin iri ɗaya ba zai yi aiki ba - kuna buƙatar zaɓar subnet ɗin kyauta na gaba. Saboda haka, bari mu dauki a matsayin tushe 192.168.0.0/16 kuma za mu fitar da masu 'yanci daga gare ta.
5. Adiresoshin Loopback. Za mu ba su duka kewayon 172.16.0.0/12.
   • Leaf - / 25 a kowace DC - guda 128 racks. Za mu ware /23 kowane yanki.
   • Spine - / 28 a kowace DC - har zuwa 16 Spine. Mu ware /26 kowane yanki.
   • Edge-Leaf - / 29 kowace DC - har zuwa akwatuna 8. Mu ware /27 kowane yanki.

Idan ba mu da isassun keɓaɓɓun jeri a cikin DC (kuma ba za a sami wani ba - muna da'awar zama masu haɓakawa), kawai muna zaɓar toshe na gaba.

Wannan shine hoton tare da adireshin IP.

Nassosi:

Prefix
Matsayin na'ura
Yankin
DC

172.16.0.0/23
baki
 
 

172.16.0.0/27
ru
 

172.16.0.0/29
msk

172.16.0.8/29
kzn

172.16.0.32/27
sp
 

172.16.0.32/29
bcn

172.16.0.40/29
mlg

172.16.0.64/27
cn
 

172.16.0.64/29
sha

172.16.0.72/29
Sia

172.16.2.0/23
kashin baya
 
 

172.16.2.0/26
ru
 

172.16.2.0/28
msk

172.16.2.16/28
kzn

172.16.2.64/26
sp
 

172.16.2.64/28
bcn

172.16.2.80/28
mlg

172.16.2.128/26
cn
 

172.16.2.128/28
sha

172.16.2.144/28
Sia

172.16.8.0/21
Leaf
 
 

172.16.8.0/23
ru
 

172.16.8.0/25
msk

172.16.8.128/25
kzn

172.16.10.0/23
sp
 

172.16.10.0/25
bcn

172.16.10.128/25
mlg

172.16.12.0/23
cn
 

172.16.12.0/25
sha

172.16.12.128/25
Sia

Ƙarƙashin:

Prefix
Yankin
DC

10.0.0.0/17
ru
 

10.0.0.0/19
msk

10.0.32.0/19
kzn

10.0.128.0/17
sp
 

10.0.128.0/19
bcn

10.0.160.0/19
mlg

10.1.0.0/17
cn
 

10.1.0.0/19
sha

10.1.32.0/19
Sia

Laba

Dillalai biyu. Hanyar sadarwa daya. ADSM.

Juniper + Arista. Ubuntu. Hauwa'u mai kyau.

Adadin albarkatu akan sabar uwar garken mu a Mirana har yanzu yana iyakance, don haka don aiki za mu yi amfani da hanyar sadarwar da aka sauƙaƙa zuwa iyaka.

Cibiyoyin bayanai guda biyu: Kazan da Barcelona.

• Kashi biyu kowanne: Juniper da Arista.
• Ɗaya daga cikin torus (Leaf) a kowace - Juniper da Arista, tare da mai haɗin gwiwa guda ɗaya (bari mu ɗauki Cisco IOL mai nauyi don wannan).
• Kullin Edge-Leaf ɗaya kowanne (a yanzu Juniper kawai).
• Canjin Cisco ɗaya don sarrafa su duka.
• Baya ga akwatunan cibiyar sadarwa, injin sarrafa kama-da-wane yana aiki. Gudun Ubuntu.
  Yana da damar yin amfani da duk na'urori, zai gudanar da tsarin IPAM/DCIM, tarin rubutun Python, Mai yiwuwa da duk wani abu da za mu iya buƙata.

Cikakken tsari na duk na'urorin cibiyar sadarwa, waɗanda za mu yi ƙoƙarin haɓaka ta amfani da atomatik.

ƙarshe

Shin hakan ma karbabbe ne? Shin zan rubuta gajeriyar ƙarshe a ƙarƙashin kowane labarin?

Don haka muka zaba mataki uku Cibiyar sadarwa ta Kloz a cikin DC, tunda muna tsammanin yawancin zirga-zirgar Gabas-Yamma kuma muna son ECMP.

An raba hanyar sadarwar zuwa jiki (ƙasa) da kama-da-wane (mai rufi). A lokaci guda, mai rufi yana farawa daga mai watsa shiri - don haka sauƙaƙe abubuwan da ake buƙata don ƙaddamarwa.

Mun zaɓi BGP a matsayin ƙa'idar tuƙi don cibiyoyin sadarwa don haɓakawa da sassaucin manufofinta.

Za mu sami rabe-raben nodes don tsara DCI - Edge-leaf.
Kashin baya zai sami OSPF+LDP.
Za a aiwatar da DCI bisa MPLS L3VPN.
Don hanyoyin haɗin P2P, za mu ƙididdige adiresoshin IP na algorithm bisa ga sunayen na'ura.
Za mu sanya madauki bisa ga matsayin na'urorin da wurinsu a jere.
Ƙarƙashin ƙaƙƙarfan prefixes-kawai akan maɓallan Leaf akan layi bisa ga wurinsu.

Bari mu ɗauka cewa a yanzu ba mu shigar da kayan aikin ba tukuna.
Saboda haka, matakanmu na gaba za su kasance don ƙara su zuwa tsarin (IPAM, kaya), tsara damar shiga, samar da tsari da tura shi.

A cikin labarin na gaba za mu yi hulɗa da Netbox - tsarin ƙira da tsarin gudanarwa don sararin IP a cikin DC.

na gode

• Andrey Glazkov aka @glazgoo don gyarawa da gyarawa
• Alexander Klimenko aka @v00lk don gyarawa da gyarawa
• Artyom Chernobay na KDPV

source: www.habr.com