Palo e kholo ea lits'ebetso tsa Khoebo le lits'ebetso tsa virtualization li na le mekhoa ea tsona ea ho aha litharollo tse mamellang liphoso. Ka ho khetheha, Oracle RAC (Oracle Real Application Cluster) ke sehlopha sa li-server tse peli kapa ho feta tsa Oracle tse sebetsang hammoho ho leka-lekanya mojaro le ho fana ka mamello ea phoso boemong ba seva / kopo. Ho sebetsa ka mokhoa ona, o hloka polokelo e arolelanoang, eo hangata e leng mokhoa oa ho boloka.
Joalokaha re se re tšohlile ho e 'ngoe ea rona , tsamaiso ea polokelo ka boeona, ho sa tsotellehe boteng ba likarolo tse kopitsoang (ho akarelletsa le balaoli), e ntse e e-na le lintlha tsa ho hlōleha - haholo-holo ka mokhoa oa sete e le 'ngoe ea data. Ka hona, ho aha tharollo ea Oracle e nang le litlhoko tse eketsehileng tsa ts'epo, morero oa "N server - one storage system" o hloka ho rarahana.
Ha e le hantle, pele, re lokela ho etsa qeto ea hore na re leka ho itšireletsa likotsing life. Sehloohong sena, re ke ke ra nahana ka tšireletso khahlanong le litšokelo tse kang “meteorite e fihlile.” Kahoo ho aha tharollo ea ho khutlisa likoluoa sebakeng se hasantsoeng e tla lula e le sehlooho sa se seng sa lihlooho tse latelang. Mona re tla sheba seo ho thoeng ke tharollo ea koluoa ea Cross-Rack, ha tšireletso e hahiloe boemong ba lik'habote tsa seva. Likhabinete ka botsona li ka ba ka phapusing e le 'ngoe kapa ka tse fapaneng, empa hangata ka har'a moaho o le mong.
Likhabinete tsena li tlameha ho ba le lisebelisoa tsohle tse hlokahalang le software e tla lumella ts'ebetso ea database ea Oracle ho sa tsotelehe boemo ba "moahelani". Ka mantsoe a mang, re sebelisa tharollo ea ho hlaphoheloa ha likoluoa tsa Cross-Rack, re tlosa likotsi tsa ho hloleha:
- Li-server tsa Oracle Application
- Litsamaiso tsa polokelo
- Litsamaiso tsa ho fetoha
- Ho hloleha ho felletseng ha lisebelisoa tsohle tse ka hara kabinete:
- Ho hana matla
- Ho hloleha ha sistimi ea ho futhumatsa
- Lintlha tsa ka ntle (motho, tlhaho, joalo-joalo)
Ho pheta-pheta ha li-server tsa Oracle ho bolela molao-motheo oa ts'ebetso oa Oracle RAC mme o kenngoa ts'ebetsong ka kopo. Ho pheta-pheta ha lisebelisoa tsa ho fetola lisebelisoa le hona ha se bothata. Empa ka phetisetso ea sistimi ea polokelo, tsohle ha li bonolo haholo.
Khetho e bonolo ka ho fetisisa ke ho pheta-pheta data ho tloha tsamaisong e kholo ea polokelo ho ea ho ea ho boloka. Synchronous kapa asynchronous, ho itšetlehile ka bokhoni ba tsamaiso ea polokelo. Ka phetisetso ea asynchronous, potso e hlaha hanghang ea ho netefatsa ho tsitsa ha data mabapi le Oracle. Empa le haeba ho na le kopanyo ea software le kopo, ho sa tsotellehe boemo leha e le bofe, ha ho e-na le ho hlōleha tsamaisong e kholo ea polokelo, ho tla hlokahala hore ho kenelle ka letsoho ke batsamaisi e le ho fetola sehlopha ho boloka polokelo.
Khetho e rarahaneng haholoanyane ke "software" le / kapa "hardware" ea polokelo ea "virtualizer" e tla felisa mathata a tsitsitseng le ho kenella ka letsoho. Empa ho rarahana ha tsamaiso le tsamaiso e latelang, hammoho le litšenyehelo tse sa hlompheheng haholo tsa tharollo e joalo, li tšosa ba bangata.
AccelStor NeoSapphire™ All Flash array solution e nepahetse bakeng sa maemo a kang ho hlaphoheloa ha koluoa ea Cross-Rack. sebelisa kaho ea Shared-Nothing. Moetso ona ke sistimi ea polokelo ea li-node tse peli e sebelisang theknoloji ea FlexiRemap® ho sebetsa ka li-flash drive. Kea leboha NeoSapphire™ H710 e khona ho fana ka ts'ebetso e fihlang ho 600K IOPS@4K e ngotsoeng ka tšohanyetso le 1M+ IOPS@4K e baloang ka tšohanyetso, e ke keng ea fihlelleha ha u sebelisa lisebelisoa tsa khale tsa polokelo tse thehiloeng ho RAID.
Empa tšobotsi e ka sehloohong ea NeoSapphire™ H710 ke ho etsoa ha li-node tse peli ka mokhoa oa linyeoe tse arohaneng, tseo e 'ngoe le e 'ngoe e nang le kopi ea eona ea data. Kamahanyo ea li-node e etsoa ka sebopeho sa kantle sa InfiniBand. Ka lebaka la mohaho ona, hoa khoneha ho aba li-node libakeng tse fapaneng ho tloha ho 100m, kahoo ho fana ka tharollo ea ho hlaphoheloa ha likoluoa tsa Cross-Rack. Li-node ka bobeli li sebetsa ka mokhoa o lumellanang ka ho feletseng. Ho tloha lehlakoreng la moamoheli, H710 e shebahala joalo ka sistimi e tloaelehileng ea li-controller tse peli. Ka hona, ha ho na tlhoko ea ho etsa khetho efe kapa efe e eketsehileng ea software kapa hardware kapa litlhophiso tse rarahaneng haholo.
Haeba re bapisa litharollo tsohle tsa ho hlaphoheloa ha likoluoa tsa Cross-Rack tse hlalositsoeng ka holimo, joale khetho e tsoang ho AccelStor e hlahella haholo ho tse ling kaofela:
AccelStor NeoSapphire™ Ha e arolelane Kaho ea Mehaho
Software kapa hardware "virtualizer" tsamaiso ea polokelo
Phetolelo e thehiloeng tharollo
Ho fumaneha
Ho hloleha ha seva
Ha ho Nako ea Phomolo
Ha ho Nako ea Phomolo
Ha ho Nako ea Phomolo
Ho hloleha ha switjha
Ha ho Nako ea Phomolo
Ha ho Nako ea Phomolo
Ha ho Nako ea Phomolo
Ho hloleha ha sistimi ea polokelo
Ha ho Nako ea Phomolo
Ha ho Nako ea Phomolo
Phomolo
Ho hloleha ha khabinete ka botlalo
Ha ho Nako ea Phomolo
Ha ho Nako ea Phomolo
Phomolo
Litšenyehelo le ho rarahana
Theko ea tharollo
Tlase*
Высокая
Высокая
Mathata a ho tsamaisa
Tlaase
Высокая
Высокая
*AccelStor NeoSapphire™ e ntse e le All Flash array, eo ka tlhaloso e sa bitseng "3 kopecks," haholo kaha e na le polokelo ea matla a mabeli. Leha ho le joalo, ha ho bapisoa litšenyehelo tsa ho qetela tsa tharollo e thehiloeng ho eona le tse tšoanang ho tsoa ho barekisi ba bang, theko e ka nkoa e le tlaase.
The topology bakeng sa ho hokahanya li-server tsa kopo le All Flash array node e tla shebahala tjena:
Ha u rera topology, ho boetse ho khothaletsoa haholo ho etsa li-switches tsa taolo le li-server tse hokahanyang.
Mona le ho feta re tla bua ka ho hokahana ka Fiber Channel. Haeba u sebelisa iSCSI, ntho e 'ngoe le e' ngoe e tla tšoana, e lokisoe bakeng sa mefuta ea li-switches tse sebelisoang le litlhophiso tse fapaneng tse fapaneng.
Mosebetsi oa ho itokisa ho sehlopha
Lisebelisoa le lisebelisoa tsa software
Litlhaloso tsa Seva le Switjha
Lihlopha
tlhaloso
Li-server tsa Oracle Database 11g
Tse peli
Sistimi e sebetsang ea seva
Linux tsa Oracle
Mofuta oa database oa Oracle
11g (RAC)
Li-processor ka seva
Li-cores tse peli tse 16 Intel® Xeon® CPU E5-2667 v2 @ 3.30GHz
Mehopolo ea 'mele ka seva
128GB
FC marangrang
16Gb/s FC e nang le multipathing
FC HBA
Emulex Lpe-16002B
Likou tse inehetseng tsa sechaba tsa 1GbE bakeng sa taolo ea lihlopha
Adapter ea ethernet ea Intel RJ45
16Gb/s FC switch
EA-6505-Brocade
Likou tse ikemetseng tsa 10GbE bakeng sa khokahano ea data
Intel X520
AccelStor NeoSapphire™ All Flash Array Specification
Lihlopha
tlhaloso
Sistimi ea polokelo
NeoSapphire™ ea mofuta o phahameng oa ho fumaneha: H710
Mofuta oa setšoantšo
4.0.1
Kakaretso ea palo ea likoloi
48
Boholo ba koloi
1.92TB
Mofuta oa Drive
SSD
FC sepheo sa likou
16x 16Gb likou (8 ka node)
Boemakepe ba tsamaiso
Mohala oa ethernet oa 1GbE o hokelang ho li-hosts ka switjha ea ethernet
Ho otla ha pelo
Mohala oa ethernet oa 1GbE o hokelang lipakeng tsa node tse peli tsa polokelo
Kou ea khokahano ea data
56Gb/s thapo ya InfiniBand
Pele o ka sebelisa sehlopha, o tlameha ho se qala. Ka ho sa feleng, aterese ea taolo ea li-node ka bobeli e tšoana (192.168.1.1). U hloka ho hokahanya le bona ka bonngoe mme u behe liaterese tse ncha (tse seng li fapane) tsa tsamaiso le ho theha nako ea ho lumellana, ka mor'a moo likou tsa Tsamaiso li ka kopanngoa le marang-rang a le mong. Kamora moo, li-node li kopantsoe ho ba para ea HA ka ho fana ka li-subnets bakeng sa likhokahano tsa Interlink.
Ka mor'a hore ho qalisoa ho phethoe, u ka khona ho laola sehlopha ho tloha node efe kapa efe.
Ka mor'a moo, re theha meqolo e hlokahalang ebe re e phatlalatsa ho li-server tsa kopo.
Ho khothaletsoa haholo ho theha li-volumes tse ngata bakeng sa Oracle ASM kaha sena se tla eketsa palo ea lipehelo bakeng sa li-server, e leng hore qetellong e tla ntlafatsa ts'ebetso ka kakaretso (ho feta ka mela ho e 'ngoe. ).
Tlhophiso ea teko
Lebitso la Bolumo ea polokelo
Boholo ba Buka
Boitsebiso01
200GB
Boitsebiso02
200GB
Boitsebiso03
200GB
Boitsebiso04
200GB
Boitsebiso05
200GB
Boitsebiso06
200GB
Boitsebiso07
200GB
Boitsebiso08
200GB
Boitsebiso09
200GB
Boitsebiso10
200GB
Grid01
1GB
Grid02
1GB
Grid03
1GB
Grid04
1GB
Grid05
1GB
Grid06
1GB
Etsa hape01
100GB
Etsa hape02
100GB
Etsa hape03
100GB
Etsa hape04
100GB
Etsa hape05
100GB
Etsa hape06
100GB
Etsa hape07
100GB
Etsa hape08
100GB
Etsa hape09
100GB
Etsa hape10
100GB
Litlhaloso tse ling mabapi le mekhoa ea ts'ebetso ea sehlopha le lits'ebetso tse etsahalang maemong a tšohanyetso
Sehlopha sa data sa node ka 'ngoe se na le parameter ea "nomoro ea phetolelo". Ka mor'a ho qala ho qala, e tšoana ebile e lekana le 1. Haeba ka lebaka le itseng nomoro ea phetolelo e fapane, joale data e lula e lumellana ho tloha phetolelong ea khale ho ea ho e monyenyane, ka mor'a moo palo ea phetolelo e nyenyane e lumellane, ke hore. sena se bolela hore likopi li tšoana. Mabaka a etsang hore liphetolelo li fapane:
- Reboots e hlophisitsoeng ea e 'ngoe ea li-node
- Kotsi ho e 'ngoe ea li-node ka lebaka la ho tima ka tšohanyetso (phepelo ea matla, ho chesa haholo, joalo-joalo).
- Khokahano ea InfiniBand e lahlehile ka ho se khone ho hokahanya
- Ho oa ho e 'ngoe ea li-node ka lebaka la bobolu ba data. Mona o tla hloka ho theha sehlopha se secha sa HA le khokahano e felletseng ea sete ea data.
Ho sa tsotellehe boemo leha e le bofe, node e setseng Inthaneteng e eketsa palo ea eona ea phetolelo ka e le 'ngoe e le hore e lumellane le data ea eona ka mor'a hore ho tsosolosoa ha khokahano le para.
Haeba khokahanyo holim'a sehokelo sa Ethernet e lahlehile, Heartbeat ka nakoana e fetohela ho InfiniBand ebe e khutla ka mor'a metsotsoana e 10 ha e khutlisoa.
Ho theha mabotho
Ho netefatsa mamello ea liphoso le ho ntlafatsa ts'ebetso, o tlameha ho nolofalletsa tšehetso ea MPIO bakeng sa sehlopha. Ho etsa sena, o hloka ho kenya mela faeleng ea /etc/multipath.conf, ebe o qala hape tšebeletso ea multipath.
Mongolo o patiloenglisebelisoa {
sesebelisoa {
morekisi "Astor"
path_grouping_policy "group_by_prio"
path_selector "queue-length 0"
path_checker "tur"
likarolo "0"
hardware_handler "0"
pele "const"
hloleha hanghang
fast_io_fail_tmo 5
dev_loss_tmo 60
user_friendly_names e
detect_prio e
rr_min_io_rq 1
no_path_retry 0
}
}
E latelang, e le hore ASM e sebetse le MPIO ka ASMLib, o hloka ho fetola faele ea /etc/sysconfig/oracleasm ebe o matha /etc/init.d/oracleasm scandisks
Mongolo o patiloeng
# ORACLEASM_SCANORDER: Ho tsamaisana le lipaterone ho odara ho hlahlojoa ha disk
ORACLEASM_SCANORDER="dm"
# ORACLEASM_SCANEXCLUDE: Ho bapisa lipaterone ho qhelela li-disk ho scan
ORACLEASM_SCANEXCLUDE="sd"
mantsoe
Haeba u sa batle ho sebelisa ASMLib, u ka sebelisa melao ea UDEV, e leng motheo oa ASMLib.
Ho qala ka mofuta oa 12.1.0.2 oa Oracle Database, khetho e fumaneha bakeng sa ho kenngoa e le karolo ea software ea ASMFD.
Hoa hlokahala ho etsa bonnete ba hore li-disk tse entsoeng bakeng sa Oracle ASM li tsamaisana le boholo ba li-block tseo sehlopha se sebetsang le tsona (4K). Ho seng joalo, mathata a ts'ebetso a ka hlaha. Ka hona, hoa hlokahala ho theha li-volumes ka li-parameter tse nepahetseng:
e arotsoe /dev/mapper/lebitso la sesebelisoa mklabel gpt mkpart primary 2048s 100% ikamahanya le maemo-hlahloba optimal 1
Phatlalatso ea li-database ho pholletsa le mefuta e entsoeng bakeng sa tlhophiso ea rona ea liteko
Lebitso la Bolumo ea polokelo
Boholo ba Buka
Ho etsa 'mapa oa li-LUN tsa Bolumo
Lintlha tsa Sesebelisoa sa Molumo oa ASM
Kabo ea boholo ba Yuniti
Boitsebiso01
200GB
'Mapa mefuta eohle ea polokelo ho sistimi ea polokelo likoung tsohle tsa data
Ho hloka mosebetsi: Ho tloaelehile
Lebitso:DGDATA
Morero: Lifaele tsa data
4MB
Boitsebiso02
200GB
Boitsebiso03
200GB
Boitsebiso04
200GB
Boitsebiso05
200GB
Boitsebiso06
200GB
Boitsebiso07
200GB
Boitsebiso08
200GB
Boitsebiso09
200GB
Boitsebiso10
200GB
Grid01
1GB
Ho hloka mosebetsi: Ho tloaelehile
Lebitso: DGGRID1
Morero: Grid: CRS le Voting
4MB
Grid02
1GB
Grid03
1GB
Grid04
1GB
Ho hloka mosebetsi: Ho tloaelehile
Lebitso: DGGRID2
Morero: Grid: CRS le Voting
4MB
Grid05
1GB
Grid06
1GB
Etsa hape01
100GB
Ho hloka mosebetsi: Ho tloaelehile
Lebitso: DGREDO1
Morero: Khutlisa log ea khoele 1
4MB
Etsa hape02
100GB
Etsa hape03
100GB
Etsa hape04
100GB
Etsa hape05
100GB
Etsa hape06
100GB
Ho hloka mosebetsi: Ho tloaelehile
Lebitso: DGREDO2
Morero: Khutlisa log ea khoele 2
4MB
Etsa hape07
100GB
Etsa hape08
100GB
Etsa hape09
100GB
Etsa hape10
100GB
Litlhophiso tsa Database
- Block boholo = 8K
- Sebaka sa phapanyetsano = 16GB
- Thibela AMM (Automatic Memory Management)
- Thibela Maqephe a Transparent Huge
Litlhophiso tse ling
# vi /etc/sysctl.conf
✓ fs.aio-max-nr = 1048576
✓ fs.file-max = 6815744
✓ kernel.shmmax 103079215104
✓ kernel.shmall 31457280
✓ kernel.shmmn 4096
✓ kernel.sem = 250 32000 100 128
✓ net.ipv4.ip_local_port_range = 9000 65500
✓ net.core.rmem_default = 262144
✓ net.core.rmem_max = 4194304
✓ net.core.wmem_default = 262144
✓ net.core.wmem_max = 1048586
✓vm.swappiness=10
✓ vm.min_free_kbytes=524288 # seke oa beha sena haeba u sebelisa Linux x86
✓ vm.vfs_cache_pressure=200
✓ vm.nr_hugepages = 57000
# vi /etc/security/limits.conf
✓ grid soft nproc 2047
✓ grid e thata nproc 16384
✓ nofile e bonolo ea grid 1024
✓ grid hard nofile 65536
✓ stack e bonolo ea grid 10240
✓ setlankane se thata sa grid 32768
✓ oracle soft nproc 2047
✓ oracle e thata nproc 16384
✓ nofile e bonolo ea oracle 1024
✓ oracle hard nofile 65536
✓ stack e bonolo ea oracle 10240
✓ stack e thata ea oracle 32768
✓ memlock e bonolo 120795954
✓ memlock e thata 120795954
sqlplus "/ as sysdba"
alter system set systems=2000 scope=spfile;
alter system set open_cursors=2000 scope=spfile;
fetola sete ya sistimi session_cached_cursors=300 scope=spfile;
alter system set db_files=8192 scope=spfile;
Teko ea ho hloleha
Bakeng sa merero ea lipontšo, HammerDB e sebelisitsoe ho etsisa mojaro oa OLTP. Litlhophiso tsa HammerDB:
Palo ea Mabolokelo
256
Kakaretso ea Lichelete ka Mosebelisi
1000000000000
Basebelisi ba Virtual
256
Sephetho e bile 2.1M TPM, e hole le moeli oa tšebetso oa sehlopha , empa ke "siling" bakeng sa tlhophiso ea lisebelisoa tsa hajoale tsa li-server (haholo-holo ka lebaka la li-processor) le palo ea tsona. Sepheo sa teko ena e ntse e le ho bontša mamello ea phoso ea tharollo ka kakaretso, eseng ho finyella tshebetso e phahameng. Ka hona, re tla haha holima setšoantšo sena feela.
Teko ea ho hloleha ho e 'ngoe ea li-node
Mabotho a ile a lahleheloa ke karolo ea litsela tse eang polokelong, a tsoela pele ho sebetsa ka ba setseng ka node ea bobeli. Ts'ebetso e ile ea theoha ka metsotsoana e seng mekae ka lebaka la litsela tse ntseng li tsosolosoa, ebe e khutlela ho tloaelehileng. Ho ne ho se na tšitiso tšebeletsong.
Teko ea ho hloleha ha Kabinete ka lisebelisoa tsohle
Tabeng ena, ts'ebetso e boetse e theohile ka metsotsoana e seng mekae ka lebaka la ho tsosolosoa ha litsela, ebe e khutlela ho halofo ea boleng ba pele. Sephetho se ile sa fokotsoa ka halofo ho tloha qalong ka lebaka la ho qheleloa ka thoko ha seva e le 'ngoe ea kopo ts'ebetsong. Hape ho ne ho se na tšitiso tšebeletsong.
Haeba ho na le tlhokahalo ea ho kenya ts'ebetsong tharollo ea ho hlaphoheloa ha likoluoa tsa Cross-Rack bakeng sa Oracle ka litšenyehelo tse lekaneng le ka boiteko bo fokolang ba ho tsamaisa / tsamaiso, joale Oracle RAC le meralo ea meralo li sebetsa hammoho. e tla ba e 'ngoe ea likhetho tse ntle ka ho fetisisa. Sebakeng sa Oracle RAC, ho ka ba le software efe kapa efe e fanang ka lihlopha, DBMS e tšoanang kapa lits'ebetso tsa virtualization, mohlala. Molao-motheo oa ho haha tharollo o tla lula o tšoana. 'Me ntlha ea bohlokoa ke zero bakeng sa RTO le RPO.
Source: www.habr.com