I-ProHoster > Блог > Ukuphatha > Ukwakhiwa kwengqalasizinda ye-IT ekwazi ukumelana namaphutha. Ingxenye 1 - Ukulungiselela Ukuphakela I-oVirt 4.3 Cluster

Ukwakhiwa kwengqalasizinda ye-IT ekwazi ukumelana namaphutha. Ingxenye 1 - Ukulungiselela Ukuphakela I-oVirt 4.3 Cluster

Abafundi bayamenywa ukuthi bazijwayeze nezimiso zokwakha ingqalasizinda ebekezelela amaphutha yebhizinisi elincane ngaphakathi kwesikhungo sedatha esisodwa, okuzoxoxwa ngakho ngokuningiliziwe ochungechungeni olufushane lwama-athikili.

Isingeniso

Ngaphansi isikhungo sedatha (Isikhungo Sokucubungula Idatha) singaqondwa ngokuthi:

  • i-rack yayo "egumbini leseva" layo endaweni yebhizinisi, ehlangabezana nezidingo ezincane zokuhlinzeka amandla nemishini yokupholisa, futhi inokufinyelela kwe-inthanethi ngabahlinzeki ababili abazimele;
  • i-rack eqashiwe enemishini yayo, etholakala esikhungweni sangempela sedatha - okuthiwa. Ukwabiwa kweTier III noma IV okuqinisekisa amandla athembekile, ukupholisa kanye nokuhluleka ukufinyelela ku-inthanethi;
  • imishini eqashwe ngokugcwele esikhungweni sedatha se-Tier III noma IV.

Iyiphi inketho yokuhlala ongayikhetha - esimweni ngasinye, yonke into ingumuntu ngamunye, futhi imvamisa incike ezintweni ezimbalwa ezibalulekile:

  • kungani ibhizinisi lidinga ingqalasizinda yalo ye-IT nhlobo;
  • lifunani ngempela ibhizinisi kungqalasizinda ye-IT (ukuthembeka, ukukala, ukulawuleka, njll.);
  • inani lokutshalwa kwezimali kokuqala kwingqalasizinda ye-IT, kanye nokuthi hlobo luni lwezindleko zayo - imali enkulu (okusho ukuthenga impahla yakho), noma ukusebenza (imishini ivame ukuqashwa);
  • umkhathizwe wokuhlela webhizinisi ngokwalo.

Ungabhala okuningi mayelana nezici ezinomthelela esinqumweni sebhizinisi sokudala nokusebenzisa ingqalasizinda yayo ye-IT, kodwa inhloso yethu ukukhombisa ngokusebenza indlela yokudala yona kanye le ngqalasizinda ukuze ikwazi ukubekezelela amaphutha futhi usengakwazi ukonga - ukunciphisa izindleko zokuthola isofthiwe yezohwebo, noma uzigweme ngokuphelele.

Njengoba umkhuba omude ubonisa, akufanelekile ukulondoloza ngensimbi, ngoba u-miser ukhokha kabili, ngisho nangaphezulu. Kodwa futhi - i-hardware enhle, lokhu kumane kuyisiphakamiso, futhi ekugcineni ukuthi yini ngempela ongayithenga nokuthi kungakanani kuncike emandleni ebhizinisi, kanye "nokuhaha" kwabaphathi bayo. Ngaphezu kwalokho, igama elithi "ukuhaha" kufanele liqondwe ngomqondo omuhle wegama, ngoba kungcono ukutshala i-hardware ngesikhathi sokuqala, ukuze kamuva ungabi nezinkinga ezinkulu ngokusekelwa kwayo okuqhubekayo nokukala, kusukela ekuqaleni okungalungile. ukuhlela nokonga ngokweqile kungaholela ezindlekweni eziphakeme kunalapho uqala umsebenzi.

Ngakho, idatha yokuqala yephrojekthi:

  • kukhona ibhizinisi elinqume ukudala ingosi yalo yewebhu futhi ilethe imisebenzi yalo ku-inthanethi;
  • inkampani yanquma ukuqasha irack ukuze ifake okokusebenza kwayo esikhungweni esihle sedatha esigunyazwe ngokwezinga le-Tier III;
  • inkampani yanquma ukungalondolozi okuningi ku-hardware, ngakho-ke yathenga imishini elandelayo eneziqinisekiso ezinwetshiwe nokusekelwa:

Uhlu lwezisetshenziswa

  • amaseva amabili omzimba weDell PowerEdge R640 kanje:
  • Amaphrosesa amabili we-Intel Xeon Gold 5120
  • 512 GB RAM
  • amadiski amabili e-SAS ku-RAID1, okufakwa kwe-OS
  • eyakhelwe ngaphakathi 4-port 1G ikhadi lenethiwekhi
  • amakhadi amabili enethiwekhi ye-2-port 10G
  • eyodwa 2-port FC HBA 16G.
  • Isitoreji sesilawuli se-Dell MD2f 3820 esixhunywe nge-FC 16G ngokuqondile kubasingathi be-Dell;
  • amaswishi amabili ezingeni lesibili - Cisco WS-C2960RX-48FPS-L estakiwe;
  • amaswishi amabili ezingeni lesithathu - Cisco WS-C3850-24T-E, kuhlanganiswe ibe isitaki;
  • I-Rack, i-UPS, i-PDU, amaseva e-console - anikezwe isikhungo sedatha.

Njengoba sibona, imishini ekhona inamathemba amahle okukalwa okuvundlile nokuma mpo, uma kwenzeka ibhizinisi lingaqhudelana nezinye izinkampani zephrofayili efanayo ku-inthanethi, bese liqala ukwenza inzuzo engatshalwa ekwandiseni izinsiza zokuncintisana okuqhubekayo. kanye nokukhula kwenzuzo.

Yiziphi izinto esingazingeza uma ibhizinisi linquma ukukhulisa ukusebenza kweqoqo lethu lekhompyutha:

  • sinokugcina okukhulu ngokwenani lezimbobo kumaswishi angu-2960X, okusho ukuthi singakwazi ukwengeza amaseva ezingxenyekazi zekhompuyutha;
  • thenga ama-switch amabili e-FC ukuze uxhume izinhlelo zokugcina kanye namaseva engeziwe kubo;
  • amaseva akhona angathuthukiswa - engeza inkumbulo, shintsha amaphrosesa ngabasebenzi abaphumelelayo, xhuma kunethiwekhi ye-10G enama-adaptha enethiwekhi akhona;
  • ungakwazi ukwengeza amashalofu ediski engeziwe ohlelweni lokugcina ngohlobo oludingekayo lwamadiski - i-SAS, i-SATA noma i-SSD, kuye ngomthwalo ohleliwe;
  • ngemuva kokwengeza amaswishi e-FC, ungathenga olunye uhlelo lokugcina ukuze wengeze umthamo wediski, futhi uma uthenga inketho ekhethekile yokuphindaphinda isilawuli kude kuyo, ungamisa ukuphindaphinda kwedatha phakathi kwezinhlelo zokugcina ngaphakathi kwemingcele yesikhungo esisodwa sedatha naphakathi kwezikhungo zedatha. (kodwa lokhu sekungaphezu kobubanzi besihloko);
  • kukhona nokushintshwa kwezinga lesithathu - i-Cisco 3850, engasetshenziswa njengomongo wenethiwekhi obekezelela amaphutha womzila wesivinini esikhulu phakathi kwamanethiwekhi angaphakathi. Lokhu kuzosiza kakhulu esikhathini esizayo, njengoba ingqalasizinda yangaphakathi ikhula. I-3850 futhi inezimbobo ze-10G ezingasetshenziswa kamuva lapho kuthuthukiswa imishini yenethiwekhi ibe yijubane le-10G.

Njengoba manje kungekho ndawo ngaphandle kwe-virtualization, ngokuqinisekile sizobe sisesimweni, ikakhulukazi njengoba lokhu kuyindlela enhle yokunciphisa izindleko zokuthola amaseva abizayo wezakhi zengqalasizinda ngayinye (amaseva ewebhu, imininingwane yolwazi, njll.), okungahlali kahle ngaso sonke isikhathi. esetshenziswa uma umthwalo uphansi, futhi yilokhu okuzoba khona ekuqaleni kokwethulwa kwephrojekthi.

Ngaphezu kwalokho, i-virtualization inezinye izinzuzo eziningi ezingaba usizo kakhulu kithi: Ukubekezelela iphutha le-VM kusuka ekuhlulekeni kweseva ye-hardware, Ukufuduka okubukhoma phakathi kwama-node we-cluster hardware wokugcinwa kwawo, ukusatshalaliswa komthwalo okwenziwa ngesandla noma okuzenzakalelayo phakathi kwama-cluster node, njll.

Ku-hardware ethengwe ibhizinisi, ukuthunyelwa kweqoqo elitholakala kakhulu le-VMware vSphere kuyazisikisela, kodwa njengoba noma iyiphi isofthiwe evela ku-VMware yaziwa ngamathegi entengo "yehhashi", sizosebenzisa isofthiwe yokuphatha i-virtualization mahhala ngokuphelele - OVirt, ngesisekelo lapho kwakhiwa umkhiqizo owaziwayo, kodwa osuvele uthengiswa - I-RHEV.

Isofthiwe OVirt kudingekile ukuhlanganisa zonke izakhi zengqalasizinda zibe yinto eyodwa ukuze ukwazi ukusebenza kalula ngemishini ebonakalayo etholakala kakhulu - lezi ziyisizindalwazi, izinhlelo zokusebenza zewebhu, amaseva wommeleli, ama-balancers, amaseva okuqoqa izingodo nokuhlaziya, njll., okungukuthi , ingosi yewebhu yebhizinisi lethu iqukethe ini.

Ukufingqa lesi singeniso, izindatshana ezilandelayo zisilindile, ezizokhombisa ngokusebenza ukuthi zifakwa kanjani zonke izingqalasizinda ze-hardware ne-software yebhizinisi:

Uhlu lwezihloko

  • Ingxenye 1. Ukulungiselela Ukuphakela I-oVirt Cluster 4.3.
  • Ingxenye 2. Ukufaka nokulungisa iqoqo le-oVirt 4.3.
  • Ingxenye 3. Ukusetha iqoqo le-VyOS, ukuhlela umzila wangaphandle obekezelela amaphutha.
  • Ingxenye 4. Isetha isitaki se-Cisco 3850, ihlela umzila we-intranet.

Ingxenye 1. Ukulungiselela Ukuphakela I-oVirt 4.3 Cluster

Ukusethwa komsingathi okuyisisekelo

Ukufaka nokumisa i-OS kuyisinyathelo esilula. Kunezindatshana eziningi zokuthi ungayifaka kanjani futhi uyilungiselele kanjani i-OS, ngakho-ke akunangqondo ukuzama ukunikeza okuthile okukhethekile ngalokhu.

Ngakho-ke, sinosokhaya ababili be-Dell PowerEdge R640 okudingeka sifake kubo i-OS futhi senze izilungiselelo zokuqala ukuze siwasebenzise njengama-hypervisors ukusebenzisa imishini ebonakalayo kuqoqo le-oVirt 4.3.

Njengoba sihlela ukusebenzisa i-oVirt yesofthiwe engeyona eyentengiso yamahhala, sikhethe i-OS yokuthumela ababungazi. I-CentOS 7.7, nakuba kungenzeka ukufaka ezinye izinhlelo zokusebenza kubasingathi be-oVirt:

  • ukwakhiwa okukhethekile okusekelwe ku-RHEL, okuthiwa. I-oVirt Node;
  • I-OS Oracle Linux Summer 2019 kwamenyezelwa mayelana nokugcina i-oVirt isebenza kuyo.

Ngaphambi kokufaka i-OS, kuyanconywa:

  • lungiselela inethiwekhi ye-iDRAC kubo bobabili abasingathi;
  • buyekeza i-firmware ye-BIOS ne-iDRAC ezinguqulweni zakamuva;
  • lungisa Iphrofayili Yesistimu yeseva, okungcono kumodi Yokusebenza;
  • lungisa i-RAID kumadiski endawo (kunconywa i-RAID1) ukuze ufake i-OS kuseva.

Bese sifaka i-OS kudiski edalwe ngaphambili nge-iDRAC - inqubo yokufaka ijwayelekile, azikho izikhathi ezikhethekile kuyo. Ungakwazi futhi ukufinyelela ikhonsoli yeseva ukuze uqale ukufakwa kwe-OS nge-iDRAC, nakuba kungekho okukuvimbelayo ukuthi uxhume imonitha, ikhibhodi negundane ngqo kuseva futhi ufake i-OS ku-flash drive.

Ngemva kokufaka i-OS, senza izilungiselelo zayo zokuqala:

systemctl enable network.service
systemctl start network.service
systemctl status network.service

systemctl stop NetworkManager
systemctl disable NetworkManager
systemctl status NetworkManager

yum install -y ntp
systemctl enable ntpd.service
systemctl start ntpd.service

cat /etc/sysconfig/selinux
SELINUX=disabled
SELINUXTYPE=targeted

cat /etc/security/limits.conf
 *               soft    nofile         65536
 *               hard   nofile         65536

cat /etc/sysctl.conf
vm.max_map_count = 262144
vm.swappiness = 1

Ifaka isethi eyisisekelo yesofthiwe

Ngokusetha kokuqala kwe-OS, udinga ukulungisa noma yisiphi isixhumi esibonakalayo senethiwekhi kuseva ukuze ukwazi ukufinyelela ku-inthanethi ukuze ubuyekeze i-OS futhi ufake amaphakheji esofthiwe adingekayo. Lokhu kungenziwa kokubili phakathi nenqubo yokufaka i-OS nangemva kwayo.

yum -y install epel-release
yum update
yum -y install bind-utils yum-utils net-tools git htop iotop nmon pciutils sysfsutils sysstat mc nc rsync wget traceroute gzip unzip telnet

Zonke lezi zilungiselelo ezingenhla kanye nesethi yesofthiwe kuyindaba yomuntu siqu, futhi leli sethi liyisincomo kuphela.

Njengoba umsingathi wethu ezodlala indima ye-hypervisor, sizovumela iphrofayili yokusebenza oyifunayo:

systemctl enable tuned 
systemctl start tuned 
systemctl status tuned 

tuned-adm profile 
tuned-adm profile virtual-host

Ungafunda kabanzi mayelana nephrofayili yokusebenza lapha:Isahluko 4".

Ngemva kokufaka i-OS, sidlulela engxenyeni elandelayo - ukulungisa ukuxhumana kwenethiwekhi kubasingathi, kanye nenqwaba yokushintsha kweCisco 2960X.

Ilungiselela i-Cisco 2960X Switch Stack

Kuphrojekthi yethu, izinombolo ezilandelayo ze-VLAN zizosetshenziswa - noma izizinda zokusakaza ezihlukanisiwe kwezinye, ukuze kuhlukaniswe izinhlobo ezahlukene zethrafikhi:

I-VLAN 10 - Inthanethi
I-VLAN 17 - Ukuphatha (iDRAC, isitoreji, ukushintshwa kokuphathwa)
I-VLAN 32 - Inethiwekhi yokukhiqiza ye-VM
I-VLAN 33 - inethiwekhi yokuxhumana (kosonkontileka bangaphandle)
I-VLAN 34 - Inethiwekhi yokuhlola ye-VM
I-VLAN 35 - Inethiwekhi yonjiniyela we-VM
I-VLAN 40 – ukuqapha inethiwekhi

Ngaphambi kokuqala umsebenzi, ake sinikeze umdwebo osezingeni le-L2, okufanele ekugcineni sifike kuwo:

Ukwakhiwa kwengqalasizinda ye-IT ekwazi ukumelana namaphutha. Ingxenye 1 - Ukulungiselela Ukuphakela I-oVirt 4.3 Cluster

Ngokusebenzelana kwenethiwekhi yabasingathi be-oVirt nemishini ebonakalayo yodwa, kanye nokuphatha uhlelo lwethu lokugcina, kuyadingeka ukulungisa isitaki sokushintshwa kwe-Cisco 2960X.

Abasubathi bakwaDell banamakhadi enethiwekhi ye-4-port eyakhelwe ngaphakathi, ngakho-ke, kuyalulekwa ukuthi bahlele ukuxhumana kwabo ne-Cisco 2960X kusetshenziswa ukuxhumana kwenethiwekhi okubekezelela iphutha, kusetshenziswa ukuqoqwa kwamachweba wenethiwekhi ebonakalayo ibe isixhumi esibonakalayo esinengqondo, kanye ne-LACP (802.3) ad) umthetho olandelwayo:

  • izimbobo ezimbili zokuqala kumsingathi zilungiswa kumodi yokubopha futhi zixhunywe ku-switch ye-2960X - lesi sikhombimsebenzisi esinengqondo sizolungiswa. bridge enekheli lokuphatha umphathi, ukuqapha, ukuxhumana nabanye ababungazi ku-oVirt cluster, izophinde isetshenziselwe ukufuduka okubukhoma kwemishini ebonakalayo;
  • amachweba amabili esibili kumsingathi nawo alungiselelwe kumodi yokubopha futhi axhunywe ku-2960X - kulesi sikhombimsebenzisi esinengqondo kusetshenziswa i-oVirt, amabhuloho azokwakhiwa kamuva (kuma-VLAN ahambisanayo) lapho imishini ebonakalayo izoxhumeka khona.
  • zombili izimbobo zenethiwekhi ngaphakathi kwesixhumi esibonakalayo esinengqondo esifanayo zizosebenza, i.e. ithrafikhi kuzo ingadluliselwa kanyekanye, ngemodi yokulinganisa.
  • izilungiselelo zenethiwekhi kuma-cluster node kufanele zifane ncamashi, ngaphandle kwamakheli e-IP.

Ukusetha isitaki esiyisisekelo sokushintsha 2960X namachweba alo

Ngaphambilini, ukushintsha kwethu kufanele kube:

  • i-rack ifakwe;
  • exhunywe izintambo ezimbili ezikhethekile zobude obudingekayo, isibonelo, i-CAB-STK-E-1M;
  • exhunywe kugesi;
  • ixhunywe endaweni yokusebenza yomlawuli ngembobo yekhonsoli yokucushwa kwayo kokuqala.

Isiqondiso esidingekayo salokhu sitholakala kokuthi ikhasi elisemthethweni umkhiqizi.

Ngemva kokuqeda lezi zinyathelo ezingenhla, silungisa amaswishi.
Okushiwo umyalo ngamunye akufanele kuchazwe ngaphakathi kohlaka lwalesi sihloko; uma kunesidingo, lonke ulwazi lungatholakala ngokuzimela.
Umgomo wethu uwukusetha ngokushesha isitaki sokushintshwa futhi sixhume ababungazi nezindawo zokuphatha isitoreji kuso.

1) Sixhuma ku-master switch, hamba kumodi enelungelo, bese uya kumodi yokumisa bese wenza izilungiselelo eziyisisekelo.

Ukulungiselelwa kokushintsha okuyisisekelo:

 enable
 configure terminal

 hostname 2960X

 no service pad
 service timestamps debug datetime msec
 service timestamps log datetime localtime show-timezone msec
 no service password-encryption
 service sequence-numbers

 switch 1 priority 15
 switch 2 priority 14
 stack-mac persistent timer 0

 clock timezone MSK 3
  vtp mode transparent
  ip subnet-zero

 vlan 17
  name Management

 vlan 32
  name PROD 

 vlan 33
  name Interconnect

 vlan 34
  name Test

 vlan 35
  name Dev

 vlan 40
  name Monitoring

 spanning-tree mode rapid-pvst
 spanning-tree etherchannel guard misconfig
 spanning-tree portfast bpduguard default
 spanning-tree extend system-id
 spanning-tree vlan 1-40 root primary
 spanning-tree loopguard default
 vlan internal allocation policy ascending
 port-channel load-balance src-dst-ip

 errdisable recovery cause loopback
 errdisable recovery cause bpduguard
 errdisable recovery interval 60

line con 0
 session-timeout 60
 exec-timeout 60 0
 logging synchronous
line vty 5 15
 session-timeout 60
 exec-timeout 60 0
 logging synchronous

 ip http server
 ip http secure-server
 no vstack

interface Vlan1
 no ip address
 shutdown

 exit

Londoloza ukulungiselelwa ngomyalo othi "wr um" bese uqala kabusha isitaki sokushintsha ngomyalo "Phinda ulayishe kabusha»ku-master switch 1.

2) Silungiselela izimbobo zenethiwekhi zokushintsha kumodi yokufinyelela (ukufinyelela) ku-VLAN 17, ukuze sixhume izixhumanisi zokulawula zezinhlelo zokugcina kanye namaseva e-iDRAC.

Ilungiselela Izimbobo Zokuphatha:

interface GigabitEthernet1/0/5
 description iDRAC - host1
 switchport access vlan 17
 switchport mode access
 spanning-tree portfast edge

interface GigabitEthernet1/0/6
 description Storage1 - Cntr0/Eth0
 switchport access vlan 17
 switchport mode access
 spanning-tree portfast edge

interface GigabitEthernet2/0/5
 description iDRAC - host2
 switchport access vlan 17
 switchport mode access
 spanning-tree portfast edge

interface GigabitEthernet2/0/6
 description Storage1 – Cntr1/Eth0
 switchport access vlan 17
 switchport mode access
 spanning-tree portfast edge
 exit

3) Ngemuva kokulayisha kabusha isitaki, hlola ukuthi sisebenza kahle:

Ukuhlola ukusebenza kwesitaki:

2960X#show switch stack-ring speed

Stack Ring Speed        : 20G
Stack Ring Configuration: Full
Stack Ring Protocol     : FlexStack

2960X#show switch stack-ports
  Switch #    Port 1       Port 2
  --------    ------       ------
    1           Ok           Ok
    2           Ok           Ok

2960X#show switch neighbors
  Switch #    Port 1       Port 2
  --------    ------       ------
      1         2             2
      2         1             1

2960X#show switch detail
Switch/Stack Mac Address : 0cd0.f8e4.ХХХХ
Mac persistency wait time: Indefinite
                                           H/W   Current
Switch#  Role   Mac Address     Priority Version  State
----------------------------------------------------------
*1       Master 0cd0.f8e4.ХХХХ    15     4       Ready
 2       Member 0029.c251.ХХХХ     14     4       Ready

         Stack Port Status             Neighbors
Switch#  Port 1     Port 2           Port 1   Port 2
--------------------------------------------------------
  1        Ok         Ok                2        2
  2        Ok         Ok                1        1

4) Ukusetha ukufinyelela kwe-SSH kusitaki se-2960X

Ukuze uphathe isitaki ukude nge-SSH, sizosebenzisa i-IP 172.20.1.10 elungiselelwe ku-SVI (shintsha isixhumi esibonakalayo esibonakalayo) I-VLAN17.

Nakuba kufiseleka ukusebenzisa ichweba elizinikele ekushintsheni ngezinjongo zokuphatha, lokhu kuyindaba yokuthanda komuntu nethuba.

Isetha ukufinyelela kwe-SSH kusitaki sokushintshwa:

ip default-gateway 172.20.1.2

interface vlan 17
 ip address 172.20.1.10 255.255.255.0

hostname 2960X
 ip domain-name hw.home-lab.ru
 no ip domain-lookup

clock set 12:47:04 06 Dec 2019

crypto key generate rsa

ip ssh version 2
ip ssh time-out 90

line vty 0 4
 session-timeout 60
 exec-timeout 60 0
 privilege level 15
 logging synchronous
 transport input ssh

line vty 5 15
 session-timeout 60
 exec-timeout 60 0
 privilege level 15
 logging synchronous
 transport input ssh

aaa new-model
aaa authentication login default local 
username cisco privilege 15 secret my_ssh_password

Setha iphasiwedi ukuze ufake imodi enelungelo:

enable secret *myenablepassword*
service password-encryption

Setha i-NTP:

ntp server 85.21.78.8 prefer
ntp server 89.221.207.113
ntp server 185.22.60.71
ntp server 192.36.143.130
ntp server 185.209.85.222

show ntp status
show ntp associations
show clock detail

5) Setha izixhumanisi ze-Etherchannel ezinengqondo nezimbobo ezibonakalayo ezixhunywe kubabungazi. Ukuze kube lula ukumisa, wonke ama-VLAN atholakalayo azovunyelwa kukho konke ukuxhumana okunengqondo, kodwa ngokuvamile kuyanconywa ukulungisa kuphela okudingekayo:

Ilungiselela ukuxhumana kwe-Etherchannel:

interface Port-channel1
 description EtherChannel with Host1-management
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 spanning-tree portfast edge trunk

interface Port-channel2
 description EtherChannel with Host2-management
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 spanning-tree portfast edge trunk

interface Port-channel3
 description EtherChannel with Host1-VM
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 spanning-tree portfast edge trunk

interface Port-channel4
 description EtherChannel with Host2-VM
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 spanning-tree portfast edge trunk

interface GigabitEthernet1/0/1
 description Host1-management
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 1 mode active

interface GigabitEthernet1/0/2
 description Host2-management
  switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 2 mode active

interface GigabitEthernet1/0/3
 description Host1-VM
  switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 3 mode active

interface GigabitEthernet1/0/4
 description Host2-VM
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 4 mode active

interface GigabitEthernet2/0/1
 description Host1-management
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 1 mode active

interface GigabitEthernet2/0/2
 description Host2-management
  switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 2 mode active

interface GigabitEthernet2/0/3
 description Host1-VM
  switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 3 mode active

interface GigabitEthernet2/0/4
 description Host2-VM
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 4 mode active

Ukucushwa kwasekuqaleni kokusebenzelana kwenethiwekhi kwemishini ebonakalayo, kubasingathi Umsingathi1 и Umsingathi2

Sibheka ubukhona bamamojula adingekayo ekusebenzeni kokubopha ohlelweni, faka imojula yokuphatha amabhuloho:

modinfo bonding
modinfo 8021q
yum install bridge-utils

Ilungiselela isixhumi esibonakalayo esinengqondo se-BOND1 semishini ebonakalayo kanye nokuxhumana kwayo okuphathekayo kubasingathi:

cat /etc/sysconfig/network-scripts/ifcfg-bond1
#DESCRIPTION - management
DEVICE=bond1
NAME=bond1
TYPE=Bond
IPV6INIT=no
ONBOOT=yes
USERCTL=no
NM_CONTROLLED=no
BOOTPROTO=none
BONDING_OPTS='mode=4 lacp_rate=1 xmit_hash_policy=2'

cat /etc/sysconfig/network-scripts/ifcfg-em2
#DESCRIPTION - management
DEVICE=em2
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MASTER=bond1
SLAVE=yes
USERCTL=no 
NM_CONTROLLED=no 

cat /etc/sysconfig/network-scripts/ifcfg-em3
#DESCRIPTION - management
DEVICE=em3
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MASTER=bond1
SLAVE=yes
USERCTL=no 
NM_CONTROLLED=no

Ngemva kokuqeda izilungiselelo ku-stack 2960H nabasingathi, qala kabusha inethiwekhi kubasingathi, futhi uhlole ukusebenza kwesixhumi esibonakalayo esinengqondo.

  • kumsingathi:

systemctl restart network

cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2+3 (2)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
...
802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
...
Slave Interface: em2
MII Status: up
Speed: 1000 Mbps
Duplex: full
...
Slave Interface: em3
MII Status: up
Speed: 1000 Mbps
Duplex: full

  • esitakini sokushintsha 2960H:

2960X#show lacp internal
Flags:  S - Device is requesting Slow LACPDUs
        F - Device is requesting Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode

Channel group 1
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi1/0/1   SA      bndl      32768         0x1       0x1     0x102       0x3D
Gi2/0/1   SA      bndl      32768         0x1       0x1     0x202       0x3D

2960X#sh etherchannel summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      N - not in use, no aggregation
        f - failed to allocate aggregator

        M - not in use, minimum links not met
        m - not in use, port not aggregated due to minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port

        A - formed by Auto LAG

Number of channel-groups in use: 11
Number of aggregators:           11

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         LACP      Gi1/0/1(P)  Gi2/0/1(P)

Ukulungiselelwa kokuqala kokuxhumana kwenethiwekhi kokuphatha izinsiza zeqoqo, kubasingathi Umsingathi1 и Umsingathi2

Ilungiselela i-BOND1 interface enengqondo yokuphatha kubasingathi, kanye nezindawo zayo ezibonakalayo:

cat /etc/sysconfig/network-scripts/ifcfg-bond0
#DESCRIPTION - management
DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
IPV6INIT=no
ONBOOT=yes
USERCTL=no
NM_CONTROLLED=no
BOOTPROTO=none
BONDING_OPTS='mode=4 lacp_rate=1 xmit_hash_policy=2'

cat /etc/sysconfig/network-scripts/ifcfg-em0
#DESCRIPTION - management
DEVICE=em0
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no 
NM_CONTROLLED=no 

cat /etc/sysconfig/network-scripts/ifcfg-em1
#DESCRIPTION - management
DEVICE=em1
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no 
NM_CONTROLLED=no

Ngemva kokuqeda izilungiselelo ku-stack 2960H nabasingathi, qala kabusha inethiwekhi kubasingathi, futhi uhlole ukusebenza kwesixhumi esibonakalayo esinengqondo.

systemctl restart network
cat /proc/net/bonding/bond1

2960X#show lacp internal
2960X#sh etherchannel summary

Setha ukuxhumana kwenethiwekhi yokuphatha kumsingathi ngamunye I-VLAN 17, futhi uyibophe kusixhumi esibonakalayo esinengqondo BOND1:

Ilungiselela i-VLAN17 ku-Host1:

cat /etc/sysconfig/network-scripts/ifcfg-bond1.17
DEVICE=bond1.17
NAME=bond1-vlan17
BOOTPROTO=none
ONBOOT=yes 
USERCTL=no 
NM_CONTROLLED=no 
VLAN=yes
MTU=1500  
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPADDR=172.20.17.163
NETMASK=255.255.255.0
GATEWAY=172.20.17.2
DEFROUTE=yes
DNS1=172.20.17.8
DNS2=172.20.17.9
ZONE=public

Ilungiselela i-VLAN17 ku-Host2:

cat /etc/sysconfig/network-scripts/ifcfg-bond1.17
DEVICE=bond1.17
NAME=bond1-vlan17
BOOTPROTO=none
ONBOOT=yes 
USERCTL=no 
NM_CONTROLLED=no 
VLAN=yes
MTU=1500  
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPADDR=172.20.17.164
NETMASK=255.255.255.0
GATEWAY=172.20.17.2
DEFROUTE=yes
DNS1=172.20.17.8
DNS2=172.20.17.9
ZONE=public

Siqala kabusha inethiwekhi kubasingathi futhi sihlole ukubonakala kwabo komunye nomunye.

Lokhu kuqeda ukucushwa kwesitaki sokushintshwa kwe-Cisco 2960X, futhi uma konke kwenziwe ngendlela efanele, manje sesinokuxhumana kwenethiwekhi yazo zonke izici zengqalasizinda komunye nomunye ezingeni le-L2.

Ukusethwa kwesitoreji se-Dell MD3820f

Ngaphambi kokuqala umsebenzi wokumisa isistimu yokugcina, kufanele isivele ixhunywe kusitaki sokushintshwa kwe-Cisco 2960H izixhumanisi zokuphatha, kanye nabasingathi Umsingathi1 и Umsingathi2 nge FC.

Uhlelo olujwayelekile lokuthi isistimu yesitoreji kufanele ixhunywe kanjani kusitaki sokushintsha sinikezwe esahlukweni esandulele.

Uhlelo lokuxhuma isitoreji nge-FC kubabungazi kufanele lubukeke kanje:

Ukwakhiwa kwengqalasizinda ye-IT ekwazi ukumelana namaphutha. Ingxenye 1 - Ukulungiselela Ukuphakela I-oVirt 4.3 Cluster

Ngesikhathi sokuxhuma, kuyadingeka ukubhala phansi amakheli e-WWPN kubasingathi be-FC HBA abaxhunywe kumachweba we-FC ohlelweni lwesitoreji - lokhu kuzodingeka ekucushweni okulandelayo kokubophezela komsingathi kuma-LUN ohlelweni lokugcina.

Landa futhi ufake i-Dell MD3820f yokuphatha isitoreji endaweni yokusebenza yomqondisi - I-PowerVault Modular Disk Storage Manager (I-MDSM).
Sixhuma kuye ngamakheli akhe e-IP azenzakalelayo, bese silungisa amakheli ethu ukusuka I-VLAN17, ukuphatha abalawuli nge-TCP/IP:

Isitoreji1:

ControllerA IP - 172.20.1.13, MASK - 255.255.255.0, Gateway - 172.20.1.2
ControllerB IP - 172.20.1.14, MASK - 255.255.255.0, Gateway - 172.20.1.2

Ngemuva kokusetha amakheli, siya ku-interface yokuphatha isitoreji bese usetha iphasiwedi, usethe isikhathi, ubuyekeze i-firmware yabalawuli namadiski, uma kunesidingo, njll.
Ukuthi lokhu kwenziwa kanjani kuchazwe ku umhlahlandlela wokuphatha isitoreji.

Ngemva kokwenza izilungiselelo ezingenhla, sidinga kuphela ukwenza izinto ezimbalwa:

  1. Lungiselela ama-ID embobo ye-FC host - Izihlonzi Zembobo Yokusingatha.
  2. Dala iqembu lokusingatha − iqembu lokusingatha bese wengeza abasingathi bethu ababili bakwaDell kuyo.
  3. Dala iqembu lediski kanye nama-virtual disks (noma ama-LUN) kuwo, azokwethulwa kubabungazi.
  4. Lungiselela ukwethulwa kwamadiski abonakalayo (noma ama-LUN) kubasingathi.

Ukwengeza ababungazi abasha nezihlonzi ezibophezelayo zamachweba e-FC kuzokwenziwa ngemenyu - I-Host Mappings -> Chaza -> Abasingathi…
Amakheli e-WWPN wabasingathi be-FC HBA angatholakala, isibonelo, ku-iDRAC yeseva.

Ngenxa yalokho, kufanele sithole okuthile okufana nalesi sithombe:

Ukwakhiwa kwengqalasizinda ye-IT ekwazi ukumelana namaphutha. Ingxenye 1 - Ukulungiselela Ukuphakela I-oVirt 4.3 Cluster

Ukwengeza iqembu elisha lokusingatha kanye nokubophezela ababungazi kulo kwenziwa ngemenyu - I-Host Mappings -> Chaza -> Iqembu Lokusingatha...
Kubaphathi, khetha uhlobo lwe-OS - I-Linux (DM-MP).

Ngemva kokudala iqembu lomsingathi, ngokusebenzisa ithebhu Isitoreji & Kopisha Amasevisi, dala iqembu lediski - Iqembu leDiski, enohlobo kuye ngezidingo zokubekezelela amaphutha, ngokwesibonelo, i-RAID10, futhi kuyo amadiski abonakalayo anosayizi odingekayo:

Ukwakhiwa kwengqalasizinda ye-IT ekwazi ukumelana namaphutha. Ingxenye 1 - Ukulungiselela Ukuphakela I-oVirt 4.3 Cluster

Futhi ekugcineni, isigaba sokugcina ukwethulwa kwamadiski abonakalayo (noma ama-LUN) kubasingathi.
Ukuze wenze lokhu, ngokusebenzisa imenyu - I-Host Mappings -> Lun mapping -> Engeza ... sibopha amadiski abonakalayo kubasingathi ngokubanika izinombolo.

Konke kufanele kubukeke njengalesi sithombe-skrini:

Ukwakhiwa kwengqalasizinda ye-IT ekwazi ukumelana namaphutha. Ingxenye 1 - Ukulungiselela Ukuphakela I-oVirt 4.3 Cluster

Lapha yilapho siqeda khona ngokusetha isitoreji, futhi uma yonke into yenziwe ngendlela efanele, abasingathi kufanele babone ama-LUN ethulwa kubo ngama-FC HBA abo.
Ake siphoqe isistimu ukuthi ibuyekeze ulwazi mayelana namadrayivu axhunyiwe:

ls -la /sys/class/scsi_host/
echo "- - -" > /sys/class/scsi_host/host[0-9]/scan

Ake sibone ukuthi imaphi amadivaysi abonakalayo kumaseva ethu:

cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 02 Id: 00 Lun: 00
  Vendor: DELL     Model: PERC H330 Mini   Rev: 4.29
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 00
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 01
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 04
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 11
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 31
  Vendor: DELL     Model: Universal Xport  Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 00
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 01
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 04
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 11
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 31
  Vendor: DELL     Model: Universal Xport  Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05

lsscsi
[0:2:0:0]    disk    DELL     PERC H330 Mini   4.29  /dev/sda
[15:0:0:0]   disk    DELL     MD38xxf          0825  -
[15:0:0:1]   disk    DELL     MD38xxf          0825  /dev/sdb
[15:0:0:4]   disk    DELL     MD38xxf          0825  /dev/sdc
[15:0:0:11]  disk    DELL     MD38xxf          0825  /dev/sdd
[15:0:0:31]  disk    DELL     Universal Xport  0825  -
 [18:0:0:0]   disk    DELL     MD38xxf          0825  -
[18:0:0:1]   disk    DELL     MD38xxf          0825  /dev/sdi
[18:0:0:4]   disk    DELL     MD38xxf          0825  /dev/sdj
[18:0:0:11]  disk    DELL     MD38xxf          0825  /dev/sdk
[18:0:0:31]  disk    DELL     Universal Xport  0825  -

Kubabungazi, ungakwazi futhi ukulungiselela ukuphindaphinda, futhi nakuba ikwazi ukuzenzela uma ifaka i-oVirt, kungcono ukuhlola ukulunga kwe-MP kusengaphambili.

Ukufaka nokumisa i-DM Multipath

yum install device-mapper-multipath
mpathconf --enable --user_friendly_names y

cat /etc/multipath.conf | egrep -v "^s*(#|$)"
defaults {
    user_friendly_names yes
            find_multipaths yes
}

blacklist {
  wwid 26353900f02796769
  devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"     
  devnode "^hd[a-z]"
 }

Setha isevisi ye-MP ukuthi iqale ngokuzenzakalelayo futhi uyiqale:

systemctl enable multipathd && systemctl restart multipathd

Ukuhlola ulwazi mayelana namamojula alayishiwe okusebenza kwe-MP:

lsmod | grep dm_multipath
dm_multipath           27792  6 dm_service_time
dm_mod                124407  139 dm_multipath,dm_log,dm_mirror

modinfo dm_multipath
filename:       /lib/modules/3.10.0-957.12.2.el7.x86_64/kernel/drivers/md/dm-multipath.ko.xz
license:        GPL
author:         Sistina Software <[email protected]>
description:    device-mapper multipath target
retpoline:      Y
rhelversion:    7.6
srcversion:     985A03DCAF053D4910E53EE
depends:        dm-mod
intree:         Y
vermagic:       3.10.0-957.12.2.el7.x86_64 SMP mod_unload modversions
signer:         CentOS Linux kernel signing key
sig_key:        A3:2D:39:46:F2:D3:58:EA:52:30:1F:63:37:8A:37:A5:54:03:00:45
sig_hashalgo:   sha256

Ukubuka isifinyezo sokucushwa kwe-multipath okukhona:

mpathconf
multipath is enabled
find_multipaths is disabled
user_friendly_names is disabled
dm_multipath module is loaded
multipathd is running

Ngemva kokwengeza i-LUN entsha kusistimu yokugcina nokuyethula kumsingathi, udinga ukuskena ama-HBA axhunywe kumsingathi kuwo. 

systemctl reload multipathd
multipath -v2

Futhi ekugcineni, sihlola ukuthi ingabe wonke ama-LUN ethulwe ohlelweni lokugcina lwababungazi, nokuthi ingabe zikhona yini izindlela ezimbili eziya kubo bonke.

Ukuhlola ukusebenza kwe-MP:

multipath -ll
3600a098000e4b4b3000003175cec1840 dm-2 DELL    ,MD38xxf
size=2.0T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 15:0:0:1  sdb 8:16  active ready running
`-+- policy='service-time 0' prio=9 status=enabled
  `- 18:0:0:1  sdi 8:128 active ready running
3600a098000e4b48f000002ab5cec1921 dm-6 DELL    ,MD38xxf
size=10T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 18:0:0:11 sdk 8:160 active ready running
`-+- policy='service-time 0' prio=9 status=enabled
  `- 15:0:0:11 sdd 8:48  active ready running
3600a098000e4b4b3000003c95d171065 dm-3 DELL    ,MD38xxf
size=150G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 15:0:0:4  sdc 8:32  active ready running
`-+- policy='service-time 0' prio=9 status=enabled
  `- 18:0:0:4  sdj 8:144 active ready running

Njengoba ubona, wonke ama-virtual disks ohlelweni lokugcina abonakala ngezindlela ezimbili. Ngakho-ke, wonke umsebenzi wokulungiselela usuqediwe, okusho ukuthi ungadlulela engxenyeni eyinhloko - ukusetha iqoqo le-oVirt, okuzoxoxwa ngalo esihlokweni esilandelayo.

Source: www.habr.com

Engeza amazwana