Ukusukela ngo-1999, ukunika inkonzo kwi-ofisi yangasemva, ibhanki yethu isebenzise inkqubo yebhanki edibeneyo ye-BISKVIT kwi-Progress OpenEdge platform, esetyenziswa ngokubanzi kwihlabathi liphela, kubandakanywa necandelo lezemali. Ukusebenza kwale DBMS kukuvumela ukuba ufunde ukuya kuthi ga kwisigidi okanye ngaphezulu kweerekhodi ngesibini kwisiseko sedatha enye (DB). Iinkonzo zethu zeProgress OpenEdge malunga ne-1,5 yezigidi zeediphozithi zomntu ngamnye kunye nezivumelwano ezimalunga nezigidi ezingama-22,2 zeemveliso ezisebenzayo (imali-mboleko yemoto kunye neemali mboleko), kwaye ikwanoxanduva lwazo zonke iindawo zokuhlala kunye nomlawuli (iBhanki Embi) kunye ne-SWIFT.
Ukusebenzisa iProgress OpenEdge, sasijongene nesidingo sokuyenza isebenze kunye ne-Oracle DBMS. Ekuqaleni, le ngxowa yayiyi-bottleneck yeziseko zethu - de kube sifake kwaye silungiselele i-Pro2 CDC - imveliso yeProgress evumela ukuba uthumele idatha kwi-DBMS yeNkqubela kwi-Oracle DBMS ngokuthe ngqo, kwi-intanethi. Kule post siza kukuxelela ngokweenkcukacha, ngayo yonke imigibe, indlela yokwenza abahlobo ngempumelelo phakathi kwe-OpenEdge kunye ne-Oracle.
Kwenzeka njani: ukulayisha idatha kwi-QCD ngokwabelana ngefayile
Okokuqala, ezinye iinyani malunga nezibonelelo zethu. Inani labasebenzisi abasebenzayo be-database malunga ne-15 lamawaka. Umthamo wazo zonke ii-database zemveliso, kubandakanywa i-replica kunye nokulinda, yi-600 TB, i-database enkulu yi-16,5 TB. Ngexesha elifanayo, ii-database zihlala zihlaziywa: kulo nyaka uphelileyo kuphela, malunga ne-120 TB yedatha evelisayo yongezwa. Inkqubo inikwe amandla ngamaseva angaphambili angama-150 kwiqonga le-x86. Oovimba beenkcukacha babanjwe kwiiseva zeqonga le-IBM ezingama-21.
Iinkqubo zangaphambili, iinkqubo zebhanki eziphambili kunye neenkonzo zebhanki zihlanganiswe ne-OpenEdge Progress (BISCUIT IBS) ngebhasi ye-Sonic ESB. Ukulayisha idatha kwi-QCD kwenzeka ngokutshintshiselana kwefayile. Kuze kube yithuba elithile, esi sisombululo sineengxaki ezimbini ezinkulu ngexesha elinye - ukusebenza okuphantsi kokulayisha ulwazi kwindawo yokugcina idatha yenkampani (CDW) kunye nexesha elide lokwenza ulungelelwaniso lwedatha (uxolelwaniso) kunye nezinye iinkqubo.
Ngoko ke, saqala ukukhangela isixhobo esinokukhawulezisa ezi nkqubo. Isisombululo kuzo zombini iingxaki yayiyimveliso entsha yeProgress OpenEdge - Pro2 CDC (Tshintsha i-Data Capture). Ngoko, masiqale.
Faka iProgress OpenEdge kunye nePro2Oracle
Ukusebenzisa iPro2 Oracle kwikhompyuter yomlawuli weWindows, kwanele ukufaka iProgress OpenEdge Developer Kit Classroom Edition, enokuthi simahla. Abalawuli bokufako be-OpenEdge abamiselweyo:
DLC: C: ProgressOpenEdge
UMSEBENZI: C:OpenEdgeWRK
Iinkqubo ze-ETL zifuna iProgress OpenEdge iilayisensi zenguqulo 11.7+ - ezizezi i-OE DataServer ye-Oracle kunye ne-4GL Development System. Ezi layisenisi zibandakanyiwe nePro2. Ukusebenza ngokupheleleyo kwe-DataServer ye-Oracle ene-database ye-Oracle ekude, i-Full Oracle Client ifakiwe.
Kwiseva ye-Oracle kufuneka ufake i-Oracle Database 12+, yenza i-database engenanto kwaye wongeze umsebenzisi (masimbize cdc).
Ukufakela iPro2Oracle, khuphela usasazo lwamva nje kwiziko lokukhuphela. Khupha indawo yokugcina kulawulo C:Pro2 (Ukuqwalasela i-Pro2 kwi-Unix, ukuhanjiswa okufanayo kusetyenziswa kwaye imigaqo efanayo yokumisela iyasebenza).
Ukudala idatabase yophindaphindo lweCDc
Isiseko sedatha yokuphindaphinda cdc (impendulo) I-Pro2 isetyenziselwa ukugcina ulwazi loqwalaselo, kubandakanywa imephu yophindaphindo, amagama eedatha eziphindiweyo kunye neetafile zazo. Kwakhona iqulethe umgca wokuphindaphinda, oquka amanqaku malunga nenyaniso yokuba umgca wetheyibhile kwisiseko sedatha yomthombo utshintshile. Idatha esuka kumgca wokuphindaphinda isetyenziswe yienkqubo ze-ETL ukuchonga imiqolo ekufuneka ikhutshelwe kwi-Oracle kwisiseko sedatha yemvelaphi.
Senza idatabase yecdc eyahlukileyo.
Inkqubo yokuyila uvimba weenkcukacha
- Kwiseva yedatha senza uluhlu lwedatha yecdc - umzekelo, kumncedisi /isiseko sedatha/cdc/.
- Yenza i-dummy ye-database ye-cdc: khuphela i-$DLC/engenanto cdc
- Nika inkxaso kwiifayile ezinkulu: proutil cdc -C EnableLargeFiles
- Silungiselela iskripthi sokuqalisa i-database ye-cdc. Iiparamitha zokuqalisa kufuneka zifane neeparitha zokuqala zesiseko sedatha esiphindiweyo.
- Siqala i-database ye-cdc.
- Qhagamshela kwisiseko sedatha yecdc kwaye ulayishe ischema sePro2 kwifayile cdc.df, efakwe kunye nePro2.
- Senza aba basebenzisi balandelayo kuvimba weenkcukacha zeCDc:
ipro2adm - yokudibanisa kwiphaneli yolawulo yePro2;
pro2etl - yokudibanisa iinkqubo ze-ETL (ReplBatch);
ipro2cdc β yokudibanisa iinkqubo zeCDC (CDCBatch);
Ivula i-OpenEdge Guqula i-Data Capture
Ngoku makhe sivule inkqubo yeCDC ngokwayo, ngoncedo apho idatha iya kuphinda iphindwe kwindawo yetekhnoloji eyongezelelweyo. Kwisiseko sesiseko sedatha yeNkqubela ye-OpenEdge nganye, kufuneka udibanise iindawo zokugcina ezahlukileyo apho idatha yemvelaphi iya kuphindwa phindwa, kwaye ivule isixhobo ngokwaso usebenzisa umyalelo. imveliso.
Umzekelo wenkqubo yesiseko sedatha ye-bisquit
- Ukukopa kwikhathalogu C:Pro2db ifayile cdcadd.st kuluhlu lwesiseko sesiseko sebisquit.
- Sichaza kwi cdcadd.st ubungakanani obumiselweyo bemimandla "ReplCDCArea" ΠΈ "ReplCDCArea_IDX". Unokongeza iindawo ezintsha zokugcina kwi-intanethi: prostrct addonline bisquit cdcadd.st
- Vula i-OpenEdge CDC:
proutil bisquit -C enablecdc indawo "ReplCDCArea" indexarea "ReplCDCArea_IDX" - Aba basebenzisi balandelayo kufuneka benziwe kwisiseko sedatha ukuchonga iinkqubo ezisebenzayo:
a. pro2adm - yokudibanisa ukusuka kwiphaneli yolawulo yePro2.
b. pro2etl - yokudibanisa iinkqubo ze-ETL (ReplBatch).
c. pro2cdc β yokudibanisa iinkqubo zeCDC (CDCBatch).
Ukudala iSchema Holder ye-DataServer ye-Oracle
Emva koko, kufuneka senze i-database ye-Schema Holder kumncedisi apho idatha evela kwi-Progress DBMS iya kuphinda iphindwe kwi-Oracle DBMS. I-DataServer Schema Holder yiNkcukacha engenanto ye-OpenEdge database ngaphandle kwabasebenzisi okanye idatha yesicelo, equlethe imephu yembalelwano phakathi kweetafile zomthombo kunye neetafile ze-Oracle zangaphandle.
I-database ye-Schema Holder ye-Progress OpenEdge DataServer ye-Oracle ye-Pro2 kufuneka ibekwe kwi-server yenkqubo ye-ETL; yenziwe ngokwahlukileyo kwisebe ngalinye.
Uyenza njani iSibambi seSchema
- Khupha unikezelo lwePro2 kulawulo /pro2
- Yenza kwaye uye kulawulo /pro2/dbsh
- Yenza isiseko sedatha SoMphathi weSchema usebenzisa umyalelo ikopi $ DLC / bisquitsh engenanto
- Ukwenza uguqulo ibhisquits kwi-encoding efunekayo-umzekelo, kwi-UTF-8 ukuba ii-database ze-Oracle zine-UTF-8 encoding: proutil bisquitsh -C convchar ukuguqula UTF-8
- Emva kokudala i-database engenanto ibhisquits qhagamshela kuyo ngendlela yomsebenzisi omnye: pro bisquits
- Makhe siye kwiDatha yesichazi-magama: Izixhobo-> Isichazi-magama seDatha->Iseva yedatha-> Izinto eziluncedo ze-ORACLE-> Yenza iSchema seServer yeDatha
- Qalisa uMphathi weSchema
- Ukuseta i-Oracle DataServer broker:
a. Qalisa i-AdminServer.
proadsv -qalisa
b. Ukuqala kwe-Oracle DataServer broker
oraman -igama orabroker1 -qalisa
Ukumisela ipaneli yolawulo kunye nesikimu sophindaphindo
Ukusebenzisa ipaneli yolawulo yePro2, iiparamitha zePro2 ziqwalaselwe, kubandakanywa ukusekwa kweskimu sokuphindaphinda kunye nokuvelisa iinkqubo ze-ETL (Ilayibrari yeProcessor), iinkqubo eziphambili zokuvumelanisa (iBulk-Copy Processor), i-replication triggers kunye nemigaqo-nkqubo ye-CDC ye-OpenEdge. Kukwakho nezixhobo eziphambili zokubeka iliso kunye nokulawula iinkqubo ze-ETL kunye neCDC. Okokuqala, siseta iifayile zeparameter.
Indlela yokuqwalasela iifayile zeparameter
- Yiya kwikhathalogu C:Pro2bpreplScripts
- Vula ifayile ukuze ihlelwe replProc.pf
- Yongeza iiparamitha zoqhagamshelo kwisiseko sedatha seCDc yophindaphindo:
# Idatabase yokuphindaphinda
-db cdc -ld repl -H <igama lenginginya yedatha engundoqo> -S <database broker port cdc>
-U pro2admin -P <password> - Faka kwi replProc.pf iparameters zoqhagamshelo kumthombo wolwazi kunye noMphathi weSchema ngendlela yeefayile zeeparamitha. Igama lefayile yeeparamitha kufuneka lihambelane negama lesiseko sedatha esidityanisiweyo.
# Qhagamshela kuyo yonke imithombo ephindaphindwayo ye-BISQUIT
-pf bpreplscriptsbisquit.pf - Faka kwi replProc.pf iiparamitha zokuqhagamshela kuMphathi weSchema.
#OkuJoliswe kwi-Pro DB Schema Holder
-db bisquitsh -ld bisquitsh
-H <ETL inkqubo yegama lomninimzi>
-S <biskuitsh broker port>
-db bisquitsql
-ld bisquitsql
-dt ORACLE
-S 5162 -H <Igama lomninimzi we-Oracle broker>
-DataService orabroker1 - Gcina ifayile yeparameters replProc.pf
- Okulandelayo, kufuneka udale kwaye uvule ukuhlela iifayile zeparameter kwisiseko sedatha nganye eqhagamshelweyo kuluhlu C:Pro2bpreplScripts: bisquit.pf. Ifayile nganye ye-pf iqulethe iiparamitha zokuqhagamshela kwisiseko sedatha esihambelanayo, umzekelo:
-db bisquit -ld bisquit -H <igama lomninimzi> -S <broker port>
-U pro2admin -P <password>
Ukuqwalasela iindlela ezimfutshane zeWindows, kufuneka uye kulawulo C:Pro2bpreplScripts kwaye uhlele "Pro2 - Administration" indlela emfutshane. Ukwenza oku, vula iipropati zendlela emfutshane kunye nomgca Qala nge bonisa uvimba wofakelo wePro2. Umsebenzi ofanayo kufuneka wenziwe kwiindlela ezimfutshane "Pro2 - Editor" kunye ne "RunBulkLoader".
ULungiselelo loLawulo lwePro2: Kulayishwa ubumbeko lokuqala
Masiqalise i-console.
Yiya kwi "DB Map".
Ukudibanisa i-database kwi-Pro2 - Ulawulo, yiya kwithebhu Imephu yeDB. Yongeza imephu yemithombo yolwazi - Umphathi weSchema-Oracle.
Yiya kwisithuba neemaphu. Idwelisiwe Uvimba weDatha onguMthombo Ngokungagqibekanga, isiseko sedatha esiqhagamshelwe sokuqala sikhethiwe. Ekunene koluhlu kufuneka kubekho umbhalo Zonke iiDatha eziQhagamshelwe β oovimba beenkcukacha abakhethiweyo zidityanisiwe. Ezantsi ekhohlo kufuneka ubone uluhlu lwe Progress tables from bisquit. Ekunene kukho uluhlu lweetafile ezivela kwisiseko sedatha se-Oracle.
Ukudala i-SQL schemas kunye nedathabheyisi kwi-Oracle
Ukwenza imephu yophindaphindo, kufuneka uqale uvelise SQL iskimu kwi-Oracle. Kulawulo lwePro2 senza into yemenyu Izixhobo -> Yenza iKhowudi -> I-Schema ekujoliswe kuyo, emva koko kwibhokisi yencoko yababini Khetha ugcino lwedatha khetha uvimba wedata omnye okanye ngaphezulu kwaye uwahambise ekunene.
Cofa u-OK kwaye ukhethe uvimba ukugcina i-SQL schemas.
Okulandelayo senza isiseko. Oku kunokwenziwa, umzekelo, ngokusebenzisa Umyili we-Oracle SQL. Ukwenza oku, sidibanisa kwi-database ye-Oracle kwaye silayishe i-schema yokongeza iitafile. Emva kokutshintsha ukubunjwa kweetafile ze-Oracle, kufuneka uhlaziye i-schemas ye-SQL kwi-Schema Holder.
Emva kokuba ukukhuphela kugqitywe ngempumelelo, phuma kwi-database ye-bisquitsh kwaye uvule iphaneli yokulawula ye-Pro2. Iitheyibhile ezivela kwisiseko sedatha se-Oracle kufuneka zivele kwiMaphu isithuba ekunene.
Imephu yetafile
Ukwenza imephu yokuphindaphinda, kwiphaneli yolawulo yePro2, yiya kwiMaphu ithebhu kwaye ukhethe isiseko sedatha. Cofa kwiiThebhile zeMaphu, khetha Khetha uTshintsho ngasekhohlo kweetafile ekufuneka ziphindaphindwe kwi-Oracle, zihambise ngasekunene kwaye uqinisekise ukhetho. Imephu iyakwenziwa ngokuzenzekelayo kwiitafile ezikhethiweyo. Siphinda umsebenzi ukwenza imephu yophindaphindo yolunye uvimba weenkcukacha wemithombo.
Ukuvelisa iThala leencwadi lePro2 yokuReplication Processor kunye neBulk-Copy Processor Programme
Ilayibrari yeReplication Processor yenzelwe iinkqubo zokuphindaphinda ngokwesiko (ETLs) eqhuba umgca wokuphindaphinda wePro2 kwaye ityhale utshintsho kwisiseko sedatha se-Oracle. Iinkqubo zethala leencwadi lomqhubekekisi wokuphindaphinda zigcinwa ngokuzenzekelayo kulawulo emva kwesizukulwana bprepl/repl_proc (PROC_DIRECTORY iparamitha). Ukwenza ilayibrari yeprosesa yophindaphindo, yiya ku Izixhobo -> Yenza iKhowudi -> Ilayibrari yeProsesa. Emva kokuba isizukulwana sigqityiwe, iinkqubo ziya kuvela kulawulo bprepl/repl_proc.
Iinkqubo zeProsesa yoMlayisho omninzi zisetyenziselwa ukungqamanisa oovimba beenkcukacha bomthombo weNkqubela kunye nesiseko sedatha ye-Oracle ekujoliswe kuyo esekwe kwiNkqubela i-ABL (4GL) yolwimi lwenkqubo. Ukuzivelisa, yiya kwinto yemenyu Izixhobo -> Veza iKhowudi -> Iprosesa ye-Bulk-Copy. Kwi Khetha iDatha yebhokisi yencoko yababini, khetha imithombo yolwazi, uwase kwicala lasekunene lefestile kwaye ucofe OK. Emva kokuba isizukulwana sigqityiwe, iinkqubo ziya kuvela kulawulo bpreplrepl_mproc.
Ukumisela iinkqubo zokuphindaphinda kwi-Pro2
Ukwahlula iitafile kwiiseti ezinikezelwa ngumsonto ophindaphindayo owahlukileyo kuphucula ukusebenza kunye nokusebenza kakuhle kwePro2 Oracle. Ngokungagqibekanga, lonke uqhakamshelwano olwenziwe kwimephu yophindaphindo yeetheyibhile ezintsha zokuphindaphinda zinxulunyaniswa nenombolo yomsonto 1. Kuyacetyiswa ukwahlula iitafile zibe yimisonto eyahlukeneyo.
Ulwazi malunga nobume bemisonto yophindaphindo luboniswa kwiscreen soLawulo lwePro2 kwi Monitor thebhu kwicandelo leSimo sokuPhinda. Inkcazo eneenkcukacha yamaxabiso eparameter inokufumaneka kuxwebhu lwePro2 (uluhlu C:Pro2Docs).
Yenza kwaye uvule imigaqo-nkqubo ye-CDC
Imigaqo-nkqubo yiseti yemithetho ye-OpenEdge CDC injini ukujonga utshintsho kwiitafile. Ngexesha lokubhalwa, iPro2 ixhasa kuphela imigaqo-nkqubo yeCDC enenqanaba 0, oko kukuthi, yinyani kuphela ebekwe iliso irekhodi utshintsho.
Ukwenza umgaqo-nkqubo we-CDC, kwiqela lenjongo yolawulo, yiya kwiMaphu isithuba, khetha isiseko sedatha yemvelaphi kwaye nqakraza kwi Yongeza/Susa iiPolisi iqhosha. Kwi Khetha Utshintsho ifestile evulayo, khetha kwicala lasekhohlo kwaye uye ngasekunene iitafile ofuna ukuyila okanye ukuyicima inkqubo yeCDC.
Ukuvula ithebhu yeMaphu kwakhona, khetha isiseko sedatha kwaye ucofe iqhosha (kwi)Vuselela Imigaqo-nkqubo. Khetha kwaye uye kwicala lasekunene letheyibhile imigaqo-nkqubo efuna ukwenziwa isebenze, cofa Lungile. Emva koku ziphawulwa ngombala oluhlaza. Ngokusebenzisa (kwi)Vuselela Imigaqo-nkqubo Unako kwakhona ukwenza iipolisi zeCDC zingasebenzi. Zonke iintengiselwano zenziwa kwi-intanethi.
Emva kokuba umgaqo-nkqubo we-CDC usebenze, amanqaku malunga neerekhodi ezilungisiweyo zigcinwa kwindawo yokugcina "ReplCDCArea" ngokwesiseko sedatha yemvelaphi. La manqaku aya kuqwalaselwa ngenkqubo eyodwa CDCBatch, esekelwe kuzo iya kudala amanqaku kumgca wokuphindaphinda we-Pro2 kwisiseko sedatha cdc (impendulo).
Ngoko ke, sinemigca emibini yokuphindaphinda. Inqanaba lokuqala yi-CDCBatch: ukusuka kwisiseko sedatha yomthombo, idatha kuqala iya kwindawo yedatha ye-CDC ephakathi. Inqanaba lesibini kuxa idatha idluliselwa kwi-database ye-CDC kwi-Oracle. Olu luphawu loyilo lwangoku kunye nemveliso ngokwayo - ukuza kuthi ga ngoku abaphuhlisi abazange bakwazi ukuseka ukuphindaphinda ngokuthe ngqo.
Ungqamaniso oluphambili
Emva kokuvumela indlela ye-CDC kunye nokuseta iseva yokuphindaphinda ye-Pro2, kufuneka siqale ungqamaniso lokuqala. Umyalelo wongqamaniso lokuqala:
/pro2/bprepl/Script/replLoad.sh bisquit table-name
Emva kokuba ungqamaniso lokuqala lugqityiwe, iinkqubo zokuphindaphinda zinokuqalwa.
Ukuqala kweenkqubo zokuphindaphinda
Ukuqala iinkqubo zophindaphindo kufuneka usebenzise iscript replbatch.sh. Phambi kokuba uqalise, qiniseka ukuba kukho imibhalo yereplbatch yayo yonke imisonto- replbatch1, replbatch2, njl. Ukuba yonke into isendaweni, vula umgca womyalelo (umzekelo, proenv), yiya kuluhlu /bprepl/scripts kwaye uqale iskripthi. Kwiphaneli yolawulo, sijonga ukuba inkqubo ehambelanayo ifumene i-RUNNING status.
Iziphumo
Emva kokuphunyezwa, siye sakukhawulezisa kakhulu ukufakwa kolwazi kwindawo yokugcina idatha yenkampani. Idatha ingena ngokuzenzekelayo kwi-Oracle kwi-intanethi. Akukho mfuneko yokuchitha ixesha usebenzisa imibuzo yexesha elide ukuqokelela idatha kwiinkqubo ezahlukeneyo. Ukongeza, kwesi sisombululo inkqubo yokuphindaphinda inokucinezela idatha, enefuthe elihle kwisantya. Ngoku uxolelwaniso lwemihla ngemihla lwenkqubo yeBISKVIT kunye nezinye iinkqubo zaqala ukuthatha imizuzu eyi-15-20 endaweni yeeyure ezingama-2-2,5, kwaye uxolelwaniso olupheleleyo lwathatha iiyure ezininzi endaweni yeentsuku ezimbini.
umthombo: www.habr.com