Ntau tus lej ntawm Enterprise daim ntawv thov thiab cov tshuab virtualization muaj lawv tus kheej cov txheej txheem los tsim cov kev daws teeb meem txhaum cai. Tshwj xeeb, Oracle RAC (Oracle Real Application Cluster) yog pawg ntawm ob lossis ntau dua Oracle database servers ua haujlwm ua ke kom sib npaug ntawm kev thauj khoom thiab muab kev ua txhaum cai ntawm qib server / daim ntawv thov. Txhawm rau ua haujlwm hauv hom no, koj xav tau kev sib koom ua ke, uas feem ntau yog lub kaw lus cia.
Raws li peb twb tau tham hauv ib qho ntawm peb , lub kaw lus cia nws tus kheej, txawm tias muaj cov khoom sib tshooj (nrog rau cov tswj hwm), tseem muaj cov ntsiab lus tsis ua haujlwm - feem ntau yog nyob rau hauv daim ntawv ntawm ib txheej ntawm cov ntaub ntawv. Yog li ntawd, txhawm rau tsim kom muaj kev daws teeb meem Oracle nrog cov kev xav tau ntawm kev ntseeg siab, "N servers - ib qho chaw khaws cia" cov tswv yim yuav tsum tau ua kom nyuaj.
Ua ntej, tau kawg, peb yuav tsum txiav txim siab seb qhov kev pheej hmoo twg peb tab tom sim ua pov thawj tawm tsam. Hauv tsab xov xwm no, peb yuav tsis xav txog kev tiv thaiv kev hem thawj zoo li "meteorite tau los txog." Yog li kev tsim kho thaj chaw uas muaj kev puas tsuaj rov qab los daws teeb meem yuav nyob twj ywm lub ntsiab lus rau ib qho ntawm cov kab lus hauv qab no. Ntawm no peb yuav saib qhov hu ua Cross-Rack kev puas tsuaj rov qab los daws teeb meem, thaum kev tiv thaiv tau tsim nyob rau theem ntawm cov khoom siv rau cov neeg rau zaub mov. Cov txee lawv tus kheej tuaj yeem nyob hauv tib chav los yog sib txawv, tab sis feem ntau nyob hauv tib lub tsev.
Cov txee no yuav tsum muaj tag nrho cov cuab yeej tsim nyog thiab software uas yuav tso cai rau kev ua haujlwm ntawm Oracle databases tsis hais lub xeev ntawm "neeg nyob ze". Hauv lwm lo lus, siv Cross-Rack kev daws teeb meem rov qab los, peb tshem tawm cov kev pheej hmoo ntawm kev ua tsis tiav:
- Oracle Application Servers
- Cia lub nruab
- Hloov tshuab
- Ua tsis tiav ntawm tag nrho cov khoom siv hauv lub txee:
- Lub zog tsis kam
- Lub tshuab cua txias tsis ua haujlwm
- Sab nraud yam (tib neeg, xwm, thiab lwm yam)
Duplication of Oracle servers implies lub hauv paus ntsiab lus kev khiav hauj lwm ntawm Oracle RAC thiab yog siv los ntawm ib daim ntawv thov. Duplication ntawm switching chaw kuj tsis muaj teeb meem. Tab sis nrog duplication ntawm qhov system cia, txhua yam tsis yog li ntawd yooj yim.
Qhov kev xaiv yooj yim tshaj plaws yog cov ntaub ntawv rov ua dua los ntawm qhov system cia tseem ceeb mus rau qhov thaub qab. Synchronous los yog asynchronous, nyob ntawm seb lub peev xwm ntawm lub kaw lus cia. Nrog asynchronous replication, cov lus nug tam sim ntawd tshwm sim ntawm kev ua kom cov ntaub ntawv sib xws hauv kev sib raug zoo rau Oracle. Tab sis txawm hais tias muaj kev sib koom ua ke software nrog rau daim ntawv thov, txawm li cas los xij, yog tias muaj kev tsis ua haujlwm ntawm lub ntsiab cia, kev cuam tshuam los ntawm cov thawj coj yuav xav tau txhawm rau hloov cov pawg mus rau qhov chaw khaws cia.
Ib qho kev xaiv nyuaj dua yog software thiab / lossis kho vajtse cia "virtualizers" uas yuav tshem tawm cov teeb meem sib xws thiab kev cuam tshuam ntawm phau ntawv. Tab sis qhov nyuaj ntawm kev xa mus thiab kev tswj hwm tom ntej, nrog rau cov nqi tsis zoo ntawm cov kev daws teeb meem, ntshai ntau yam.
AccelStor NeoSapphire ™ Txhua qhov kev daws teeb meem Flash yog zoo meej rau cov xwm txheej xws li Hla-Rack kev puas tsuaj rov qab siv Shared-Nothing architecture. Cov qauv no yog ob-node cia system uas siv cov cuab yeej FlexiRemap® thev naus laus zis los ua haujlwm nrog flash drives. Ua tsaug rau NeoSapphire™ H710 muaj peev xwm xa cov kev ua tau zoo txog 600K IOPS @ 4K random sau thiab 1M + IOPS @ 4K random nyeem, uas tsis tuaj yeem ua tiav thaum siv classic RAID-raws li kev khaws cia.
Tab sis qhov tseem ceeb ntawm NeoSapphire ™ H710 yog kev ua tiav ntawm ob lub nodes nyob rau hauv daim ntawv sib cais, txhua tus muaj nws tus kheej cov ntaub ntawv luam tawm. Synchronization ntawm nodes yog nqa tawm los ntawm sab nraud InfiniBand interface. Ua tsaug rau cov qauv no, nws muaj peev xwm faib cov nodes mus rau qhov chaw sib txawv ntawm qhov deb txog li 100m, yog li muab Cross-Rack kev daws teeb meem rov qab. Ob lub nodes ua haujlwm tag nrho synchronously. Los ntawm tus tswv tsev, H710 zoo li zoo tib yam dual-controller cia system. Yog li ntawd, tsis tas yuav ua ib qho software ntxiv lossis kev xaiv kho vajtse lossis tshwj xeeb cov chaw teeb tsa.
Yog tias peb sib piv tag nrho Cross-Rack kev puas tsuaj rov qab daws teeb meem tau piav qhia saum toj no, ces qhov kev xaiv los ntawm AccelStor sawv tawm pom zoo los ntawm tus so:
AccelStor NeoSapphire™ Sib koom tsis muaj dab tsi Architecture
Software lossis hardware "virtualizer" cia system
Replication raws li kev daws teeb meem
Muaj Txaus
Server tsis ua haujlwm
Tsis Muaj Downtime
Tsis Muaj Downtime
Tsis Muaj Downtime
Hloov tsis ua haujlwm
Tsis Muaj Downtime
Tsis Muaj Downtime
Tsis Muaj Downtime
Cia qhov system tsis ua hauj lwm
Tsis Muaj Downtime
Tsis Muaj Downtime
Nruab nrab
Tag nrho lub txee tsis ua haujlwm
Tsis Muaj Downtime
Tsis Muaj Downtime
Nruab nrab
Nqi thiab complexity
Kev daws nqi
Tsawg*
Tsis yog
Tsis yog
Deployment complexity
Tsawg
Tsis yog
Tsis yog
* AccelStor NeoSapphire ™ tseem yog Tag Nrho Flash array, uas txhais tau tias tsis raug nqi "3 kopecks," tshwj xeeb tshaj yog vim nws muaj peev xwm cia ob npaug. Txawm li cas los xij, thaum sib piv tus nqi kawg ntawm kev daws teeb meem raws li nws nrog cov zoo sib xws los ntawm lwm tus neeg muag khoom, tus nqi yuav suav tias yog tsawg.
Lub topology rau kev sib txuas daim ntawv thov servers thiab Tag nrho Flash array nodes yuav zoo li no:
Thaum npaj cov topology, nws kuj tau pom zoo kom muab cov kev tswj xyuas qhov sib npaug thiab sib txuas cov servers.
Ntawm no thiab ntxiv peb yuav tham txog kev sib txuas ntawm Fiber Channel. Yog tias koj siv iSCSI, txhua yam yuav zoo ib yam, hloov kho rau hom keyboards siv thiab sib txawv me ntsis array nqis.
Kev npaj ua haujlwm ntawm array
Cov khoom siv thiab software siv
Server thiab Switch Specifications
Cheebtsam
piav qhia
Oracle Database 11g servers
Ob
Server operating system
Oracle Linux
Oracle database version
11g (RAC)
Processors rau server
Ob 16 cores Intel® Xeon® CPU E5-2667 v2 @ 3.30GHz
Lub cev nco ib lub server
128GB
FC network
16Gb/s FC nrog multipathing
FC HAB
Emulex Lpe-16002B
Dedicated pej xeem 1GbE ports rau kev tswj pawg
Intel ethernet adapter RJ45
16Gb/s FC hloov
Phaib 6505
Dedicated private 10GbE ports rau cov ntaub ntawv synchonization
Intel X520
AccelStor NeoSapphire ™ Tag nrho Flash Array Specification
Cheebtsam
piav qhia
Cia qhov system
NeoSapphire ™ siab muaj qauv: H710
Duab version
4.0.1
Tag nrho cov tsav tsheb
48
Tsav loj
1.92TB
Tsav hom
SSD
FC lub hom phiaj ports
16 x 16Gb ports (8 per node)
Tswj cov chaw nres nkoj
Lub 1GbE ethernet cable txuas rau cov tswv los ntawm kev hloov ethernet
Lub plawv dhia chaw nres nkoj
1GbE ethernet cable txuas ntawm ob qhov chaw cia khoom
Cov ntaub ntawv synchronization chaw nres nkoj
56Gb/s InfiniBand cable
Ua ntej koj tuaj yeem siv array, koj yuav tsum pib nws. Los ntawm lub neej ntawd, qhov chaw tswj ntawm ob lub nodes yog tib yam (192.168.1.1). Koj yuav tsum txuas mus rau lawv ib qho los ntawm ib qho thiab teeb tsa tshiab (twb txawv) tswj chaw nyob thiab teeb tsa lub sijhawm synchronization, tom qab ntawd Cov Chaw Tswj Xyuas tuaj yeem txuas nrog ib lub network. Tom qab ntawd, cov nodes tau muab tso rau hauv ib khub HA los ntawm kev muab cov subnets rau kev sib txuas ntawm Interlink.
Tom qab pib ua tiav, koj tuaj yeem tswj hwm cov array los ntawm ib qho ntawm txhua qhov.
Tom ntej no, peb tsim cov ntim tsim nyog thiab tshaj tawm rau cov ntawv thov servers.
Nws raug nquahu kom tsim ntau qhov ntim rau Oracle ASM vim qhov no yuav ua rau muaj ntau lub hom phiaj rau cov servers, uas thaum kawg yuav txhim kho kev ua tau zoo tag nrho (ntau dua ntawm cov kab hauv lwm qhov. ).
Test configuration
Lub Npe Lub Npe
Ntim Loj
Cov ntaub ntawv 01
200GB
Cov ntaub ntawv 02
200GB
Cov ntaub ntawv 03
200GB
Cov ntaub ntawv 04
200GB
Cov ntaub ntawv 05
200GB
Cov ntaub ntawv 06
200GB
Cov ntaub ntawv 07
200GB
Cov ntaub ntawv 08
200GB
Cov ntaub ntawv 09
200GB
Cov ntaub ntawv 10
200GB
Grid01
1GB
Grid02
1GB
Grid03
1GB
Grid04
1GB
Grid05
1GB
Grid06
1GB
Rov 01
100GB
Rov 02
100GB
Rov 03
100GB
Rov 04
100GB
Rov 05
100GB
Rov 06
100GB
Rov 07
100GB
Rov 08
100GB
Rov 09
100GB
Rov 10
100GB
Qee cov lus piav qhia txog kev ua haujlwm ntawm cov array thiab cov txheej txheem tshwm sim thaum muaj xwm txheej ceev
Cov ntaub ntawv txheej txheem ntawm txhua lub node muaj "tus lej version" parameter. Tom qab pib pib, nws yog tib yam thiab sib npaug rau 1. Yog tias vim li cas tus lej version txawv, ces cov ntaub ntawv yeej ib txwm synchronized los ntawm cov laus version mus rau tus yau, tom qab uas tus naj npawb ntawm cov hluas version yog aligned, i.e. qhov no txhais tau tias cov ntawv luam yog tib yam. Yog vim li cas vim li cas versions yuav txawv:
- Teem sijhawm reboot ntawm ib qho ntawm cov nodes
- Ib qho kev huam yuaj ntawm ib qho ntawm cov nodes vim yog kev kaw tam sim ntawd (lub hwj chim mov, overheating, thiab lwm yam).
- Poob InfiniBand kev sib txuas nrog tsis muaj peev xwm synchronize
- Kev sib tsoo ntawm ib qho ntawm cov nodes vim cov ntaub ntawv kev noj nyiaj txiag. Ntawm no koj yuav tau tsim ib pab pawg HA tshiab thiab ua kom tiav synchronization ntawm cov ntaub ntawv teev.
Txawm li cas los xij, cov node uas tseem nyob hauv online nce nws tus lej version los ntawm ib qho txhawm rau txhawm rau synchronize nws cov ntaub ntawv teev tom qab kev sib txuas nrog cov khub rov qab los.
Yog tias qhov kev sib txuas dhau ntawm Ethernet txuas tau ploj lawm, Lub Plawv Ua Si ib ntus hloov mus rau InfiniBand thiab rov qab los hauv 10 vib nas this thaum nws rov qab los.
Teeb tsa hosts
Txhawm rau ua kom muaj kev zam txim txhaum cai thiab txhim kho kev ua haujlwm, koj yuav tsum ua kom MPIO kev txhawb nqa rau cov array. Txhawm rau ua qhov no, koj yuav tsum ntxiv cov kab rau /etc/multipath.conf cov ntaub ntawv, thiab tom qab ntawd rov pib qhov kev pabcuam multipath
Cov ntawv zaiskhoom siv {
ntaus ntawv {
tus neeg muag khoom "AStor"
path_grouping_policy "pab_by_prio"
path_selector "queue-length 0"
path_checker "tur"
nta "0"
hardware_handler "0"
prio "const"
tsis ua hauj lwm tam sim
fast_io_fail_tmo 5
dev_loss_tmo 60
user_friendly_names yog
detect_prio yog
rr_min_io_rq 1
no_path_retry 0
}
}
Tom ntej no, txhawm rau ASM ua haujlwm nrog MPIO ntawm ASMLib, koj yuav tsum hloov cov ntaub ntawv /etc/sysconfig/oracleasm thiab ces khiav /etc/init.d/oracleasm scandisks
Cov ntawv zais
# ORACLEASM_SCANORDER: Sib piv cov qauv los txiav txim disk scanning
ORACLEASM_SCANORDER = "dm"
# ORACLEASM_SCANEXCLUDE: Sib piv cov qauv kom tsis suav cov disks los ntawm kev luam theej duab
ORACLEASM_SCANEXCLUDE = "sd"
Примечание
Yog tias koj tsis xav siv ASMLib, koj tuaj yeem siv UDEV cov cai, uas yog lub hauv paus rau ASMLib.
Pib nrog version 12.1.0.2 ntawm Oracle Database, qhov kev xaiv muaj rau kev teeb tsa ua ib feem ntawm ASMFD software.
Nws yog ib qho tseem ceeb los xyuas kom meej tias cov disks tsim rau Oracle ASM yog ua raws li cov block loj uas cov array lub cev ua haujlwm nrog (4K). Txwv tsis pub, teeb meem kev ua haujlwm yuav tshwm sim. Yog li ntawd, nws yog ib qho tsim nyog los tsim cov ntim nrog qhov tsim nyog:
parted /dev/mapper/device-name mklabel gpt mkpart primary 2048s 100% align-check optimal 1
Kev faib tawm ntawm cov ntaub ntawv thoob plaws cov ntim tsim rau peb qhov kev sim teeb tsa
Lub Npe Lub Npe
Ntim Loj
Volume LUNs daim ntawv qhia
ASM Volume Device Detail
Qhov Loj Qhov Loj
Cov ntaub ntawv 01
200GB
Daim ntawv qhia tag nrho cov ntim cia rau qhov system cia tag nrho cov ntaub ntawv chaw nres nkoj
Redundancy: Ib txwm
Lub npe: DGDATA
Hom phiaj: Cov ntaub ntawv
4MB
Cov ntaub ntawv 02
200GB
Cov ntaub ntawv 03
200GB
Cov ntaub ntawv 04
200GB
Cov ntaub ntawv 05
200GB
Cov ntaub ntawv 06
200GB
Cov ntaub ntawv 07
200GB
Cov ntaub ntawv 08
200GB
Cov ntaub ntawv 09
200GB
Cov ntaub ntawv 10
200GB
Grid01
1GB
Redundancy: Ib txwm
Lub npe: DGGRID1
Hom phiaj: Daim phiaj: CRS thiab pov npav
4MB
Grid02
1GB
Grid03
1GB
Grid04
1GB
Redundancy: Ib txwm
Lub npe: DGGRID2
Hom phiaj: Daim phiaj: CRS thiab pov npav
4MB
Grid05
1GB
Grid06
1GB
Rov 01
100GB
Redundancy: Ib txwm
Lub npe: DGREDO1
Lub hom phiaj: Redo cav ntawm xov 1
4MB
Rov 02
100GB
Rov 03
100GB
Rov 04
100GB
Rov 05
100GB
Rov 06
100GB
Redundancy: Ib txwm
Lub npe: DGREDO2
Lub hom phiaj: Redo cav ntawm xov 2
4MB
Rov 07
100GB
Rov 08
100GB
Rov 09
100GB
Rov 10
100GB
Database Settings
- Block loj = 8K
- Hloov chaw = 16 GB
- Disable AMM (Automatic Memory Management)
- Disable Transparent Huge Pages
Lwm qhov chaw
# vi /etc/sysctl.conf
✓ fs.aio-max-nr = 1048576
✓ fs.file-max = 6815744
✓ Kernel.shmmax 103079215104
✓ Kernel.shmall 31457280
✓ Kernel.shmmn 4096
✓ kernel.sem = 250 32000 100 128
✓ net.ipv4.ip_local_port_range = 9000 65500
✓ net.core.rmem_default = 262144
✓ net.core.rmem_max = 4194304
✓ net.core.wmem_default = 262144
✓ net.core.wmem_max = 1048586
✓vm.swappiness=10
✓ vm.min_free_kbytes=524288 # tsis txhob teeb qhov no yog tias koj siv Linux x86
✓ vm.vfs_cache_pressure=200
✓ vm.nr_hugepages = 57000
# vi /etc/security/limits.conf
✓ grid soft nproc 2047
✓ grid hard nproc 16384
✓ daim phiaj mos nofile 1024
✓ daim phiaj nyuaj nofile 65536
✓ daim phiaj mos pawg 10240
✓ daim phiaj nyuaj 32768
✓ oracle soft nproc 2047
✓ oracle hard nproc 16384
✓ oracle soft nofile 1024
✓ oracle nyuaj nofile 65536
✓ oracle mos pawg 10240
✓ oracle nyuaj pawg 32768
✓ mos memlock 120795954
✓ mob siab 120795954
sqlplus “/as sysdba”
Hloov cov txheej txheem txheej txheem = 2000 scope = spfile;
alter system teem open_cursors = 2000 scope = spfile;
alter system teem session_cached_cursors = 300 scope = spfile;
alter system teeb db_files=8192 scope=spfile;
Kev xeem tsis tiav
Rau kev ua qauv qhia, HammerDB tau siv los ua raws li OLTP load. HammerDB configuration:
Number of Warehouses
256
Tag nrho cov kev ua lag luam rau ib tus neeg siv
1000000000000
Cov neeg siv virtual
256
Qhov tshwm sim yog 2.1M TPM, uas nyob deb ntawm array qhov kev txwv tsis pub dhau , tab sis yog "qab nthab" rau kev teeb tsa kho vajtse tam sim no ntawm cov servers (feem ntau vim yog processors) thiab lawv tus lej. Lub hom phiaj ntawm qhov kev sim no tseem yog ua kom pom qhov ua tsis raug ntawm kev daws teeb meem tag nrho, thiab tsis ua kom tiav qhov siab tshaj plaws. Yog li ntawd, peb yuav tsuas tsim ntawm daim duab no.
Kuaj rau kev ua tsis tiav ntawm ib qho ntawm cov nodes
Cov tswv tau poob ib feem ntawm txoj hauv kev mus rau qhov chaw cia, txuas ntxiv mus ua haujlwm los ntawm cov seem nrog rau qhov thib ob. Kev ua tau zoo poob rau ob peb lub vib nas this vim txoj hauv kev tau rov ua dua, thiab tom qab ntawd rov qab mus rau qhov qub. Tsis muaj kev cuam tshuam hauv kev pabcuam.
Txee tsis ua hauj lwm kuaj nrog txhua yam khoom siv
Nyob rau hauv cov ntaub ntawv no, kev ua tau zoo kuj poob rau ob peb lub vib nas this vim qhov restructuring ntawm txoj kev, thiab ces rov qab mus rau ib nrab ntawm tus nqi qub. Qhov tshwm sim tau raug muab faib ua ib nrab ntawm qhov pib vim yog kev cais tawm ntawm ib daim ntawv thov server los ntawm kev ua haujlwm. Tsis muaj kev cuam tshuam hauv kev pabcuam ib yam.
Yog tias muaj qhov xav tau los siv qhov kev ua txhaum cai Cross-Rack kev puas tsuaj rov qab los daws teeb meem rau Oracle ntawm tus nqi tsim nyog thiab nrog kev siv me me / kev tswj hwm, ces Oracle RAC thiab architecture ua haujlwm ua ke yuav yog ib qho kev xaiv zoo tshaj plaws. Hloov chaw Oracle RAC, tuaj yeem muaj lwm yam software uas muab kev sib koom ua ke, tib DBMS lossis virtualization systems, piv txwv li. Lub hauv paus ntsiab lus ntawm kev tsim cov kev daws teeb meem yuav nyob twj ywm tib yam. Thiab cov kab hauv qab yog xoom rau RTO thiab RPO.
Tau qhov twg los: www.hab.com