ProHoster > Блог > Administrasi > Nggawe infrastruktur IT sing tahan kesalahan. Bagean 1 - nyiapake kanggo nyebarake kluster oVirt 4.3

Nggawe infrastruktur IT sing tahan kesalahan. Bagean 1 - nyiapake kanggo nyebarake kluster oVirt 4.3

Para pamaca diundang supaya ngerti prinsip-prinsip mbangun infrastruktur sing tahan kesalahan kanggo perusahaan cilik ing pusat data siji, sing bakal dibahas kanthi rinci ing seri artikel cekak.

Bagean pambuko

Miturut Pusat data (Pusat Pangolahan Data) bisa dimangerteni minangka:

  • rak dhewe ing "kamar server" dhewe ing latar perusahaan, kang meets syarat minimal kanggo nyediakake sumber daya lan cooling peralatan, lan uga nduweni akses Internet liwat loro panyedhiya sawijining;
  • rak disewakake karo peralatan dhewe, dumunung ing pusat data nyata - sing disebut. kolokasi, sing tundhuk karo standar Tier III utawa IV, lan njamin sumber daya sing dipercaya, pendinginan lan akses Internet sing tahan kesalahan;
  • peralatan sing disewakake kanthi lengkap ing pusat data Tier III utawa IV.

Pilihan akomodasi sing dipilih yaiku individu ing saben kasus, lan biasane gumantung ing sawetara faktor utama:

  • Napa perusahaan mbutuhake infrastruktur IT dhewe?
  • apa persis sing dikarepake perusahaan saka infrastruktur IT (reliabilitas, skalabilitas, manajemen, lsp);
  • volume investasi awal ing infrastruktur IT, uga jinis biaya - ibukutha (sing tegese sampeyan tuku peralatan dhewe), utawa operasional (peralatan biasane disewakake);
  • planning cakrawala saka perusahaan dhewe.

Akeh sing bisa ditulis babagan faktor-faktor sing mengaruhi keputusan perusahaan kanggo nggawe lan nggunakake infrastruktur IT, nanging tujuane yaiku nuduhake prakteke carane nggawe prasarana iki supaya bisa tahan kesalahan lan uga bisa ngirit dhuwit. biaya tuku piranti lunak komersial, utawa nyingkiri kabeh.

Minangka laku long-term nuduhake, iku ora worth nyimpen ing hardware, wiwit pelit mbayar kaping pindho, lan malah luwih. Nanging maneh, hardware apik mung Rekomendasi, lan ing pungkasan apa persis kanggo tuku lan carane akeh gumantung ing Kapabilitas perusahaan lan "srakah" saka Manajemen. Kajaba iku, tembung "rakus" kudu dimangerteni ing pangertèn sing apik saka tembung kasebut, amarga luwih apik kanggo nandur modal ing hardware ing tahap wiwitan, supaya ora duwe masalah serius karo dhukungan lan skala luwih lanjut, amarga perencanaan lan skala sing ora bener. tabungan gedhe banget bisa mimpin kanggo biaya luwih saka nalika miwiti project.

Dadi, data awal kanggo proyek kasebut:

  • ana perusahaan sing mutusake nggawe portal web dhewe lan nggawa kegiatane menyang Internet;
  • perusahaan mutusaké kanggo nyewa rak kanggo nyeleh peralatan ing pusat data apik certified miturut standar Tier III;
  • perusahaan mutusaké ora kanggo nyimpen akeh ing hardware, lan mulane tuku peralatan ing ngisor iki karo babar pisan lan support:

dhaftar peralatan

  • loro server Dell PowerEdge R640 fisik minangka nderek:
  • loro prosesor Intel Xeon Gold 5120
  • RAM 512 Gb Kab
  • rong disk SAS ing RAID1, kanggo instalasi OS
  • dibangun ing 4-port kertu jaringan 1G
  • loro 2-port kertu jaringan 10G
  • siji 2-port FC HBA 16G.
  • 2-sistem panyimpenan controller Dell MD3820f, disambungake liwat FC 16G langsung kanggo sarwa dumadi Dell;
  • loro ngalih tingkat kapindho - Cisco WS-C2960RX-48FPS-L dibandhingke;
  • loro ngalih tingkat katelu - Cisco WS-C3850-24T-E, dibandhingke;
  • Rak, UPS, PDU, server konsol diwenehake dening pusat data.

Kaya sing dideleng, peralatan sing wis ana duwe prospek sing apik kanggo skala horisontal lan vertikal, yen perusahaan bisa bersaing karo perusahaan liyane kanthi profil sing padha ing Internet, lan wiwit entuk bathi, sing bisa nandur modal kanggo ngembangake sumber daya kanggo kompetisi luwih lanjut. lan wutah bathi.

Peralatan apa sing bisa ditambahake yen perusahaan mutusake nambah kinerja kluster komputasi:

  • kita duwe cadangan gedhe ing nomer bandar ing 2960X ngalih, kang tegese kita bisa nambah server hardware liyane;
  • tuku loro ngalih FC tambahan kanggo nyambungake sistem panyimpenan lan server tambahan kanggo wong-wong mau;
  • server ana bisa nganyari - nambah memori, ngganti prosesor karo luwih kuat, nyambung menyang jaringan 10G nggunakake adaptor jaringan ana;
  • Sampeyan bisa nambah rak disk tambahan menyang sistem panyimpenan kanthi jinis disk sing dibutuhake - SAS, SATA utawa SSD, gumantung saka beban sing direncanakake;
  • sawise nambah switch FC, sampeyan bisa tuku sistem panyimpenan liyane kanggo nambah kapasitas disk malah luwih, lan yen sampeyan tuku pilihan Replikasi Remote khusus, sampeyan bisa nyiyapake replikasi data antarane sistem panyimpenan loro ing pusat data padha lan antarane pusat data ( nanging iki wis ngluwihi ruang lingkup artikel);
  • Ana uga ngalih tingkat katelu - Cisco 3850, kang bisa digunakake minangka fault-tolerant inti jaringan kanggo kacepetan dhuwur nuntun antarane jaringan internal. Iki bakal mbiyantu banget ing mangsa ngarep amarga infrastruktur internal saya tambah akeh. 3850 uga duwe port 10G, sing bisa digunakake mengko nalika nganyarke peralatan jaringan menyang kecepatan 10G.

Wiwit saiki ora ana ngendi wae tanpa virtualisasi, mesthine kita bakal dadi tren, utamane amarga iki minangka cara sing apik kanggo nyuda biaya tuku server sing larang kanggo unsur infrastruktur individu (server web, database, lan liya-liyane), sing ora mesthi. optimal digunakake ing cilik mbukak kurang, lan iki persis apa sing bakal kelakon ing awal Bukak project.

Kajaba iku, virtualization wis akeh kaluwihan liyane sing bisa banget migunani kanggo kita: toleransi fault VM marang Gagal server hardware, Migration Live antarane kelenjar kluster hardware kanggo pangopènan sing, distribusi mbukak manual utawa otomatis antarane kelenjar kluster, etc.

Kanggo hardware sing dituku perusahaan, panyebaran kluster VMware vSphere sing kasedhiya banget, nanging amarga piranti lunak apa wae saka VMware dikenal kanthi label rega "jaran", kita bakal nggunakake piranti lunak gratis kanggo ngatur virtualisasi - oVirt, ing basis saka produk sing kondhang nanging wis komersial digawe - rhev.

Piranti lunak oVirt perlu kanggo nggabungake kabeh unsur infrastruktur dadi siji supaya bisa kanthi gampang nggarap mesin virtual sing kasedhiya banget - iki minangka basis data, aplikasi web, server proxy, penyeimbang, server kanggo ngumpulake log lan analytics, lsp. portal web perusahaan kita kasusun saka.

Kanggo ngringkes introduksi iki, kita bisa ngarep-arep artikel ing ngisor iki, sing bakal nuduhake kanthi bener babagan cara masang kabeh infrastruktur hardware lan piranti lunak perusahaan:

Dhaptar artikel

  • Bagean 1. Nyiyapake kanggo nyebarake kluster oVirt 4.3.
  • Bagean 2. Nginstal lan ngatur kluster oVirt 4.3.
  • Bagean 3. Nyetel kluster VyOS, ngatur rute eksternal sing tahan kesalahan.
  • Bagean 4. Nggawe Cisco 3850 tumpukan, ngatur intranet nuntun.

Part 1. Nyiapake kanggo masang oVirt 4.3 kluster

Persiyapan host dhasar

Nginstal lan ngatur OS minangka langkah paling gampang. Ana akeh artikel babagan carane nginstal lan ngatur OS kanthi bener, mula ora ana gunane kanggo nyoba menehi eksklusif babagan iki.

Dadi, kita duwe loro host Dell PowerEdge R640 sing kudu diinstal OS lan nindakake setelan awal supaya bisa digunakake minangka hypervisors kanggo mesin virtual sing mlaku ing kluster oVirt 4.3.

Amarga kita rencana nggunakake piranti lunak oVirt non-komersial gratis, OS dipilih kanggo nyebarake host CentOS 7.7, sanajan OS liyane bisa diinstal ing host kanggo oVirt:

  • mbangun khusus adhedhasar RHEL, sing disebut. oVirt Node;
  • OS Oracle Linux, musim panas 2019 iku diumumake babagan ndhukung karya oVirt ing.

Sadurunge nginstal OS dianjurake:

  • ngatur antarmuka jaringan iDRAC ing loro sarwa dumadi;
  • nganyari firmware BIOS lan iDRAC menyang versi paling anyar;
  • ngatur Profil Sistem server, luwih apik ing mode Performance;
  • ngatur RAID saka disk lokal (RAID1 dianjurake) kanggo nginstal OS ing server.

Banjur kita nginstal OS ing disk sing digawe sadurunge liwat iDRAC - proses instalasi iku normal, ora ana wektu khusus. Akses menyang konsol server kanggo miwiti instalasi OS uga bisa dipikolehi liwat iDRAC, sanajan ora ana sing ngalangi sampeyan nyambungake monitor, keyboard lan mouse langsung menyang server lan nginstal OS saka flash drive.

Sawise nginstal OS, kita nindakake setelan awal:

systemctl enable network.service
systemctl start network.service
systemctl status network.service

systemctl stop NetworkManager
systemctl disable NetworkManager
systemctl status NetworkManager

yum install -y ntp
systemctl enable ntpd.service
systemctl start ntpd.service

cat /etc/sysconfig/selinux
SELINUX=disabled
SELINUXTYPE=targeted

cat /etc/security/limits.conf
 *               soft    nofile         65536
 *               hard   nofile         65536

cat /etc/sysctl.conf
vm.max_map_count = 262144
vm.swappiness = 1

Nginstal piranti lunak dhasar

Kanggo pisanan ngatur OS, sampeyan kudu ngatur antarmuka jaringan apa wae ing server supaya sampeyan bisa ngakses Internet kanggo nganyari OS lan nginstal paket piranti lunak sing dibutuhake. Iki bisa ditindakake sajrone proses instalasi OS lan sawise.

yum -y install epel-release
yum update
yum -y install bind-utils yum-utils net-tools git htop iotop nmon pciutils sysfsutils sysstat mc nc rsync wget traceroute gzip unzip telnet

Kabeh setelan lan piranti lunak ing ndhuwur minangka pilihan pribadi, lan set iki mung minangka rekomendasi.

Amarga host kita bakal dadi hypervisor, kita bakal ngaktifake profil kinerja sing dibutuhake:

systemctl enable tuned 
systemctl start tuned 
systemctl status tuned 

tuned-adm profile 
tuned-adm profile virtual-host

Sampeyan bisa maca liyane babagan profil kinerja ing kene: "Bab 4. laras lan laras-adm".

Sawise nginstal OS, kita pindhah menyang bagean sabanjure - nyetel antarmuka jaringan ing host lan tumpukan Cisco 2960X ngalih.

Konfigurasi Cisco 2960X Ngalih Stack

Proyek kita bakal nggunakake nomer VLAN ing ngisor iki - utawa domain siaran, sing diisolasi saka siji liyane, kanggo misahake macem-macem jinis lalu lintas:

VLAN 10 - Internet
VLAN 17 - Manajemen (iDRAC, sistem panyimpenan, manajemen switch)
VLAN 32 - Jaringan produksi VM
VLAN 33 - jaringan interkoneksi (kanggo kontraktor eksternal)
VLAN 34 - Jaringan tes VM
VLAN 35 - Jaringan pangembang VM
VLAN 40 – Jaringan monitoring

Sadurunge miwiti karya, iki diagram ing tingkat L2 sing pungkasane kudu ditindakake:

Nggawe infrastruktur IT sing tahan kesalahan. Bagean 1 - nyiapake kanggo nyebarake kluster oVirt 4.3

Kanggo interaksi jaringan sarwa dumadi oVirt lan mesin virtual karo saben liyane, uga kanggo ngatur sistem panyimpenan, iku perlu kanggo ngatur tumpukan Cisco 2960X ngalih.

Dell sarwa dumadi wis dibangun ing kertu jaringan 4-port, Mulane, iku saranake kanggo ngatur sambungan menyang Cisco 2960X nggunakake sambungan jaringan fault-toleran, nggunakake klompok bandar jaringan fisik menyang antarmuka logis, lan protokol LACP ( 802.3ad):

  • rong port pisanan ing host dikonfigurasi ing mode ikatan lan disambungake menyang switch 2960X - antarmuka logis iki bakal dikonfigurasi bridge karo alamat kanggo Manajemen inang, ngawasi, komunikasi karo sarwa dumadi liyane ing kluster oVirt, uga bakal digunakake kanggo Live migration saka mesin virtual;
  • loro bandar kaloro ing inang uga diatur ing mode iketan lan disambungake menyang 2960X - ing antarmuka logis iki nggunakake oVirt, kreteg bakal digawe ing mangsa (ing VLANs cocog) kang mesin virtual bakal disambungake.
  • loro bandar jaringan, ing antarmuka logis padha, bakal aktif, i.e. lalu lintas liwat mau bisa ditularaké bebarengan, ing mode wawas.
  • setelan jaringan ing kelenjar kluster kudu pancen padha, kajaba alamat IP.

Persiyapan tumpukan ngalih dhasar 2960X lan lelabuhane

Saklar kita kudu pisanan:

  • rak dipasang;
  • disambungake liwat rong kabel khusus saka dawa sing dibutuhake, contone, CAB-STK-E-1M;
  • disambungake menyang sumber daya;
  • disambungake menyang workstation administrator liwat port console kanggo konfigurasi dhisikan.

Panuntun perlu kanggo iki kasedhiya ing halaman resmi pabrikan.

Sawise ngrampungake langkah-langkah ing ndhuwur, kita ngatur switch.
Apa tegese saben printah ora dimaksudake kanggo dideskripsikake ing kerangka artikel iki; yen perlu, kabeh informasi bisa ditemokake kanthi mandiri.
Tujuane yaiku ngatur tumpukan switch kanthi cepet lan nyambungake host lan antarmuka manajemen panyimpenan.

1) Sambungake menyang switch master, pindhah menyang mode istimewa, banjur pindhah menyang mode konfigurasi lan nggawe setelan dhasar.

Konfigurasi switch dhasar:

 enable
 configure terminal

 hostname 2960X

 no service pad
 service timestamps debug datetime msec
 service timestamps log datetime localtime show-timezone msec
 no service password-encryption
 service sequence-numbers

 switch 1 priority 15
 switch 2 priority 14
 stack-mac persistent timer 0

 clock timezone MSK 3
  vtp mode transparent
  ip subnet-zero

 vlan 17
  name Management

 vlan 32
  name PROD 

 vlan 33
  name Interconnect

 vlan 34
  name Test

 vlan 35
  name Dev

 vlan 40
  name Monitoring

 spanning-tree mode rapid-pvst
 spanning-tree etherchannel guard misconfig
 spanning-tree portfast bpduguard default
 spanning-tree extend system-id
 spanning-tree vlan 1-40 root primary
 spanning-tree loopguard default
 vlan internal allocation policy ascending
 port-channel load-balance src-dst-ip

 errdisable recovery cause loopback
 errdisable recovery cause bpduguard
 errdisable recovery interval 60

line con 0
 session-timeout 60
 exec-timeout 60 0
 logging synchronous
line vty 5 15
 session-timeout 60
 exec-timeout 60 0
 logging synchronous

 ip http server
 ip http secure-server
 no vstack

interface Vlan1
 no ip address
 shutdown

 exit

Kita nyimpen konfigurasi kanthi printah "wr mem"lan urip maneh tumpukan switch nganggo printah"muat maneh» ing saklar master 1.

2) We ngatur bandar jaringan ngalih ing mode akses ing VLAN 17, kanggo nyambungake antarmuka Manajemen sistem panyimpenan lan server iDRAC.

Nyetel port manajemen:

interface GigabitEthernet1/0/5
 description iDRAC - host1
 switchport access vlan 17
 switchport mode access
 spanning-tree portfast edge

interface GigabitEthernet1/0/6
 description Storage1 - Cntr0/Eth0
 switchport access vlan 17
 switchport mode access
 spanning-tree portfast edge

interface GigabitEthernet2/0/5
 description iDRAC - host2
 switchport access vlan 17
 switchport mode access
 spanning-tree portfast edge

interface GigabitEthernet2/0/6
 description Storage1 – Cntr1/Eth0
 switchport access vlan 17
 switchport mode access
 spanning-tree portfast edge
 exit

3) Sawise ngisi ulang tumpukan, priksa manawa bisa digunakake kanthi bener:

Priksa fungsi tumpukan:

2960X#show switch stack-ring speed

Stack Ring Speed        : 20G
Stack Ring Configuration: Full
Stack Ring Protocol     : FlexStack

2960X#show switch stack-ports
  Switch #    Port 1       Port 2
  --------    ------       ------
    1           Ok           Ok
    2           Ok           Ok

2960X#show switch neighbors
  Switch #    Port 1       Port 2
  --------    ------       ------
      1         2             2
      2         1             1

2960X#show switch detail
Switch/Stack Mac Address : 0cd0.f8e4.ХХХХ
Mac persistency wait time: Indefinite
                                           H/W   Current
Switch#  Role   Mac Address     Priority Version  State
----------------------------------------------------------
*1       Master 0cd0.f8e4.ХХХХ    15     4       Ready
 2       Member 0029.c251.ХХХХ     14     4       Ready

         Stack Port Status             Neighbors
Switch#  Port 1     Port 2           Port 1   Port 2
--------------------------------------------------------
  1        Ok         Ok                2        2
  2        Ok         Ok                1        1

4) Nggawe akses SSH menyang tumpukan 2960X

Kanggo mbatalake ngatur tumpukan liwat SSH, kita bakal nggunakake IP 172.20.1.10 diatur kanggo SVI (ngalih antarmuka virtual) VLAN17.

Senajan disaranake nggunakake port darmabakti darmabakti ing ngalih kanggo tujuan Manajemen, iki bab pilihan pribadi lan Kapabilitas.

Konfigurasi akses SSH menyang tumpukan switch:

ip default-gateway 172.20.1.2

interface vlan 17
 ip address 172.20.1.10 255.255.255.0

hostname 2960X
 ip domain-name hw.home-lab.ru
 no ip domain-lookup

clock set 12:47:04 06 Dec 2019

crypto key generate rsa

ip ssh version 2
ip ssh time-out 90

line vty 0 4
 session-timeout 60
 exec-timeout 60 0
 privilege level 15
 logging synchronous
 transport input ssh

line vty 5 15
 session-timeout 60
 exec-timeout 60 0
 privilege level 15
 logging synchronous
 transport input ssh

aaa new-model
aaa authentication login default local 
username cisco privilege 15 secret my_ssh_password

Setel sandhi kanggo mlebu mode istimewa:

enable secret *myenablepassword*
service password-encryption

Setelan NTP:

ntp server 85.21.78.8 prefer
ntp server 89.221.207.113
ntp server 185.22.60.71
ntp server 192.36.143.130
ntp server 185.209.85.222

show ntp status
show ntp associations
show clock detail

5) Ngatur antarmuka Etherchannel logis lan port fisik sing disambungake menyang host. Kanggo gampang konfigurasi, kabeh VLAN sing kasedhiya bakal diaktifake ing kabeh antarmuka logis, nanging umume dianjurake kanggo ngatur mung sing dibutuhake:

Konfigurasi antarmuka Etherchannel:

interface Port-channel1
 description EtherChannel with Host1-management
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 spanning-tree portfast edge trunk

interface Port-channel2
 description EtherChannel with Host2-management
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 spanning-tree portfast edge trunk

interface Port-channel3
 description EtherChannel with Host1-VM
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 spanning-tree portfast edge trunk

interface Port-channel4
 description EtherChannel with Host2-VM
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 spanning-tree portfast edge trunk

interface GigabitEthernet1/0/1
 description Host1-management
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 1 mode active

interface GigabitEthernet1/0/2
 description Host2-management
  switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 2 mode active

interface GigabitEthernet1/0/3
 description Host1-VM
  switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 3 mode active

interface GigabitEthernet1/0/4
 description Host2-VM
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 4 mode active

interface GigabitEthernet2/0/1
 description Host1-management
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 1 mode active

interface GigabitEthernet2/0/2
 description Host2-management
  switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 2 mode active

interface GigabitEthernet2/0/3
 description Host1-VM
  switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 3 mode active

interface GigabitEthernet2/0/4
 description Host2-VM
 switchport trunk allowed vlan 10,17,30-40
 switchport mode trunk
 channel-protocol lacp
 channel-group 4 mode active

Konfigurasi awal antarmuka jaringan kanggo mesin virtual ing host Tuan rumah1 и Tuan rumah2

Kita mriksa anané modul sing perlu kanggo ikatan bisa digunakake ing sistem, nginstal modul kanggo ngontrol jembatan:

modinfo bonding
modinfo 8021q
yum install bridge-utils

Konfigurasi antarmuka logis BOND1 kanggo mesin virtual lan antarmuka fisik ing host:

cat /etc/sysconfig/network-scripts/ifcfg-bond1
#DESCRIPTION - management
DEVICE=bond1
NAME=bond1
TYPE=Bond
IPV6INIT=no
ONBOOT=yes
USERCTL=no
NM_CONTROLLED=no
BOOTPROTO=none
BONDING_OPTS='mode=4 lacp_rate=1 xmit_hash_policy=2'

cat /etc/sysconfig/network-scripts/ifcfg-em2
#DESCRIPTION - management
DEVICE=em2
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MASTER=bond1
SLAVE=yes
USERCTL=no 
NM_CONTROLLED=no 

cat /etc/sysconfig/network-scripts/ifcfg-em3
#DESCRIPTION - management
DEVICE=em3
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MASTER=bond1
SLAVE=yes
USERCTL=no 
NM_CONTROLLED=no

Sawise ngrampungake setelan ing tumpukan 2960H lan sarwa dumadi, kita miwiti maneh jaringan ing sarwa dumadi lan mriksa fungsi antarmuka logis.

  • ing tuan rumah:

systemctl restart network

cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2+3 (2)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
...
802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
...
Slave Interface: em2
MII Status: up
Speed: 1000 Mbps
Duplex: full
...
Slave Interface: em3
MII Status: up
Speed: 1000 Mbps
Duplex: full

  • ing tumpukan ngalih 2960H:

2960X#show lacp internal
Flags:  S - Device is requesting Slow LACPDUs
        F - Device is requesting Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode

Channel group 1
                            LACP port     Admin     Oper    Port        Port
Port      Flags   State     Priority      Key       Key     Number      State
Gi1/0/1   SA      bndl      32768         0x1       0x1     0x102       0x3D
Gi2/0/1   SA      bndl      32768         0x1       0x1     0x202       0x3D

2960X#sh etherchannel summary
Flags:  D - down        P - bundled in port-channel
        I - stand-alone s - suspended
        H - Hot-standby (LACP only)
        R - Layer3      S - Layer2
        U - in use      N - not in use, no aggregation
        f - failed to allocate aggregator

        M - not in use, minimum links not met
        m - not in use, port not aggregated due to minimum links not met
        u - unsuitable for bundling
        w - waiting to be aggregated
        d - default port

        A - formed by Auto LAG

Number of channel-groups in use: 11
Number of aggregators:           11

Group  Port-channel  Protocol    Ports
------+-------------+-----------+-----------------------------------------------
1      Po1(SU)         LACP      Gi1/0/1(P)  Gi2/0/1(P)

Konfigurasi wiwitan antarmuka jaringan kanggo ngatur sumber daya kluster ing host Tuan rumah1 и Tuan rumah2

Konfigurasi antarmuka logis BOND1 kanggo manajemen lan antarmuka fisik ing host:

cat /etc/sysconfig/network-scripts/ifcfg-bond0
#DESCRIPTION - management
DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
IPV6INIT=no
ONBOOT=yes
USERCTL=no
NM_CONTROLLED=no
BOOTPROTO=none
BONDING_OPTS='mode=4 lacp_rate=1 xmit_hash_policy=2'

cat /etc/sysconfig/network-scripts/ifcfg-em0
#DESCRIPTION - management
DEVICE=em0
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no 
NM_CONTROLLED=no 

cat /etc/sysconfig/network-scripts/ifcfg-em1
#DESCRIPTION - management
DEVICE=em1
TYPE=Ethernet
BOOTPROTO=none
ONBOOT=yes
MASTER=bond0
SLAVE=yes
USERCTL=no 
NM_CONTROLLED=no

Sawise ngrampungake setelan ing tumpukan 2960H lan sarwa dumadi, kita miwiti maneh jaringan ing sarwa dumadi lan mriksa fungsi antarmuka logis.

systemctl restart network
cat /proc/net/bonding/bond1

2960X#show lacp internal
2960X#sh etherchannel summary

We ngatur antarmuka jaringan kontrol ing saben host ing VLAN 17, lan ikatan menyang antarmuka logis BOND1:

Konfigurasi VLAN17 ing Host1:

cat /etc/sysconfig/network-scripts/ifcfg-bond1.17
DEVICE=bond1.17
NAME=bond1-vlan17
BOOTPROTO=none
ONBOOT=yes 
USERCTL=no 
NM_CONTROLLED=no 
VLAN=yes
MTU=1500  
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPADDR=172.20.17.163
NETMASK=255.255.255.0
GATEWAY=172.20.17.2
DEFROUTE=yes
DNS1=172.20.17.8
DNS2=172.20.17.9
ZONE=public

Konfigurasi VLAN17 ing Host2:

cat /etc/sysconfig/network-scripts/ifcfg-bond1.17
DEVICE=bond1.17
NAME=bond1-vlan17
BOOTPROTO=none
ONBOOT=yes 
USERCTL=no 
NM_CONTROLLED=no 
VLAN=yes
MTU=1500  
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
IPADDR=172.20.17.164
NETMASK=255.255.255.0
GATEWAY=172.20.17.2
DEFROUTE=yes
DNS1=172.20.17.8
DNS2=172.20.17.9
ZONE=public

Kita miwiti maneh jaringan ing host lan mriksa visibilitas kanggo saben liyane.

Iki ngrampungake konfigurasi tumpukan Cisco 2960X ngalih, lan yen kabeh wis rampung bener, banjur saiki kita duwe panyambungan jaringan kabeh unsur infrastruktur kanggo saben liyane ing tingkat L2.

Nyiyapake sistem panyimpenan Dell MD3820f

Sadurunge miwiti karya ing nyetel sistem panyimpenan, iku kudu wis disambungake menyang tumpukan ngalih Cisco 2960H antarmuka kontrol, uga kanggo sarwa dumadi Tuan rumah1 и Tuan rumah2 liwat FC.

Diagram umum babagan carane sistem panyimpenan kudu disambungake menyang tumpukan switch diwenehi ing bab sadurunge.

Diagram kanggo nyambungake sistem panyimpenan liwat FC menyang host kudu katon kaya iki:

Nggawe infrastruktur IT sing tahan kesalahan. Bagean 1 - nyiapake kanggo nyebarake kluster oVirt 4.3

Sajrone sambungan, sampeyan kudu nulis alamat WWPN kanggo sarwa dumadi FC HBA sing disambungake menyang port FC ing sistem panyimpenan - iki bakal dibutuhake kanggo mengko nyetel ikatan host menyang LUN ing sistem panyimpenan.

Ing stasiun kerja administrator, download lan instal sarana kanggo ngatur sistem panyimpenan Dell MD3820f - PowerVault Modular Disk Storage Manager (MDSM).
Kita nyambungake liwat alamat IP standar, banjur atur alamat saka VLAN17, kanggo ngatur pengontrol liwat TCP/IP:

Panyimpen1:

ControllerA IP - 172.20.1.13, MASK - 255.255.255.0, Gateway - 172.20.1.2
ControllerB IP - 172.20.1.14, MASK - 255.255.255.0, Gateway - 172.20.1.2

Sawise nyetel alamat, pindhah menyang antarmuka manajemen panyimpenan lan atur sandhi, nyetel wektu, nganyari perangkat kukuh kanggo pengontrol lan disk, yen perlu, lsp.
Carane iki rampung diterangake ing pedoman administrasi Sistem panyimpenan

Sawise ngrampungake setelan ing ndhuwur, kita mung kudu nindakake sawetara langkah:

  1. Konfigurasi ID port FC host - Host Port Identifiers.
  2. Nggawe grup host - Klompok tuan rumah lan nambah loro sarwa dumadi Dell kita.
  3. Nggawe grup disk lan disk virtual (utawa LUNs) ing sing bakal diwenehi kanggo host.
  4. Konfigurasi presentasi disk virtual (utawa LUN) kanggo host.

Nambahake host anyar lan ngiket pengenal port FC host menyang wong-wong mau wis rampung liwat menu - Host Mappings -> Netepake -> Tuan rumah…
Alamat WWPN saka host FC HBA bisa ditemokake, contone, ing server iDRAC.

Akibaté, kita kudu njaluk kaya iki:

Nggawe infrastruktur IT sing tahan kesalahan. Bagean 1 - nyiapake kanggo nyebarake kluster oVirt 4.3

Nambahake klompok host anyar lan ikatan host kasebut ditindakake liwat menu - Host Mappings -> Netepake -> Grup host…
Kanggo host, pilih jinis OS - Linux (DM-MP).

Sawise nggawe grup inang, liwat tab Layanan Panyimpenan & Salin, nggawe grup disk - Grup Disk, kanthi jinis gumantung saka syarat kanggo toleransi kesalahan, contone, RAID10, lan ing disk virtual ukuran sing dibutuhake:

Nggawe infrastruktur IT sing tahan kesalahan. Bagean 1 - nyiapake kanggo nyebarake kluster oVirt 4.3

Lan pungkasane, tahap pungkasan yaiku presentasi disk virtual (utawa LUN) menyang host.
Kanggo nindakake iki, liwat menu - Host Mappings -> Pemetaan rembulan -> Tambah… Kita nggandhengake disk virtual karo host kanthi menehi nomer.

Kabeh kudu katon kaya gambar iki:

Nggawe infrastruktur IT sing tahan kesalahan. Bagean 1 - nyiapake kanggo nyebarake kluster oVirt 4.3

Iki ngendi kita rampung nyetel sistem panyimpenan, lan yen kabeh wis rampung bener, banjur sarwa dumadi kudu ndeleng LUNs presented kanggo wong liwat FC HBA.
Ayo meksa sistem nganyari informasi babagan disk sing disambungake:

ls -la /sys/class/scsi_host/
echo "- - -" > /sys/class/scsi_host/host[0-9]/scan

Ayo ndeleng piranti apa sing katon ing server kita:

cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 02 Id: 00 Lun: 00
  Vendor: DELL     Model: PERC H330 Mini   Rev: 4.29
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 00
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 01
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 04
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 11
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi15 Channel: 00 Id: 00 Lun: 31
  Vendor: DELL     Model: Universal Xport  Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 00
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 01
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 04
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 11
  Vendor: DELL     Model: MD38xxf          Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05
Host: scsi18 Channel: 00 Id: 00 Lun: 31
  Vendor: DELL     Model: Universal Xport  Rev: 0825
  Type:   Direct-Access                    ANSI  SCSI revision: 05

lsscsi
[0:2:0:0]    disk    DELL     PERC H330 Mini   4.29  /dev/sda
[15:0:0:0]   disk    DELL     MD38xxf          0825  -
[15:0:0:1]   disk    DELL     MD38xxf          0825  /dev/sdb
[15:0:0:4]   disk    DELL     MD38xxf          0825  /dev/sdc
[15:0:0:11]  disk    DELL     MD38xxf          0825  /dev/sdd
[15:0:0:31]  disk    DELL     Universal Xport  0825  -
 [18:0:0:0]   disk    DELL     MD38xxf          0825  -
[18:0:0:1]   disk    DELL     MD38xxf          0825  /dev/sdi
[18:0:0:4]   disk    DELL     MD38xxf          0825  /dev/sdj
[18:0:0:11]  disk    DELL     MD38xxf          0825  /dev/sdk
[18:0:0:31]  disk    DELL     Universal Xport  0825  -

Ing sarwa dumadi sampeyan uga bisa ngatur tambahan multipath, lan sanajan nalika nginstal oVirt bisa nindakake iki dhewe, iku luwih apik kanggo mriksa operasi bener MP ing advance dhewe.

Instalasi lan konfigurasi DM Multipath

yum install device-mapper-multipath
mpathconf --enable --user_friendly_names y

cat /etc/multipath.conf | egrep -v "^s*(#|$)"
defaults {
    user_friendly_names yes
            find_multipaths yes
}

blacklist {
  wwid 26353900f02796769
  devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"     
  devnode "^hd[a-z]"
 }

Setel layanan MP kanggo miwiti otomatis lan miwiti:

systemctl enable multipathd && systemctl restart multipathd

Priksa informasi babagan modul dimuat kanggo operasi MP:

lsmod | grep dm_multipath
dm_multipath           27792  6 dm_service_time
dm_mod                124407  139 dm_multipath,dm_log,dm_mirror

modinfo dm_multipath
filename:       /lib/modules/3.10.0-957.12.2.el7.x86_64/kernel/drivers/md/dm-multipath.ko.xz
license:        GPL
author:         Sistina Software <[email protected]>
description:    device-mapper multipath target
retpoline:      Y
rhelversion:    7.6
srcversion:     985A03DCAF053D4910E53EE
depends:        dm-mod
intree:         Y
vermagic:       3.10.0-957.12.2.el7.x86_64 SMP mod_unload modversions
signer:         CentOS Linux kernel signing key
sig_key:        A3:2D:39:46:F2:D3:58:EA:52:30:1F:63:37:8A:37:A5:54:03:00:45
sig_hashalgo:   sha256

Ayo goleki informasi ringkesan babagan konfigurasi multipath sing ana:

mpathconf
multipath is enabled
find_multipaths is disabled
user_friendly_names is disabled
dm_multipath module is loaded
multipathd is running

Sawise nambahake LUN anyar menyang sistem panyimpenan lan nampilake menyang host, sampeyan kudu mindhai HBA sing disambungake menyang host kasebut. 

systemctl reload multipathd
multipath -v2

Lan pungkasanipun, kita mriksa apa kabeh LUNs padha presented ing sistem panyimpenan kanggo sarwa dumadi, lan apa ana loro dalan kanggo kabeh.

Priksa operasi MP:

multipath -ll
3600a098000e4b4b3000003175cec1840 dm-2 DELL    ,MD38xxf
size=2.0T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 15:0:0:1  sdb 8:16  active ready running
`-+- policy='service-time 0' prio=9 status=enabled
  `- 18:0:0:1  sdi 8:128 active ready running
3600a098000e4b48f000002ab5cec1921 dm-6 DELL    ,MD38xxf
size=10T features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 18:0:0:11 sdk 8:160 active ready running
`-+- policy='service-time 0' prio=9 status=enabled
  `- 15:0:0:11 sdd 8:48  active ready running
3600a098000e4b4b3000003c95d171065 dm-3 DELL    ,MD38xxf
size=150G features='3 queue_if_no_path pg_init_retries 50' hwhandler='1 rdac' wp=rw
|-+- policy='service-time 0' prio=14 status=active
| `- 15:0:0:4  sdc 8:32  active ready running
`-+- policy='service-time 0' prio=9 status=enabled
  `- 18:0:0:4  sdj 8:144 active ready running

Kaya sing sampeyan ngerteni, kabeh telung disk virtual ing sistem panyimpenan katon ing rong dalan. Mangkono, kabeh karya persiapan wis rampung, tegese sampeyan bisa nerusake menyang bagean utama - nyetel kluster oVirt, sing bakal dibahas ing artikel sabanjure.

Source: www.habr.com

Add a comment