Kusukela ngo-1999, ukuze kusebenze ihhovisi elingemuva, ibhange lethu liye lasebenzisa uhlelo lwamabhange oludidiyelwe i-BISKVIT ku-Progress OpenEdge yesikhulumi, esisetshenziswa kabanzi emhlabeni wonke, kuhlanganise nomkhakha wezezimali. Ukusebenza kwale DBMS kukuvumela ukuthi ufunde amarekhodi afinyelela esigidini noma ngaphezulu ngomzuzwana kusizindalwazi esisodwa (DB). Izinsizakalo zethu ze-Progress OpenEdge cishe eziyizigidi ezingu-1,5 kanye nezinkontileka ezibalelwa ezigidini ezingu-22,2 zemikhiqizo esebenzayo (imalimboleko yezimoto kanye nezindlu ezibolekiwe), futhi ibhekele zonke izinkokhelo nomlawuli (Ibhange Elimaphakathi) kanye ne-SWIFT.
Sisebenzisa i-Progress OpenEdge, besibhekene nesidingo sokuyenza isebenze ne-Oracle DBMS. Ekuqaleni, le nqwaba ibiyibhodlela lengqalasizinda yethu - size sifake futhi silungise i-Pro2 CDC - umkhiqizo we-Progress okuvumela ukuthi uthumele idatha isuka ku-DBMS yokuqhubeka iye ku-Oracle DBMS ngokuqondile, ku-inthanethi. Kulokhu okuthunyelwe sizokutshela ngokuningiliziwe, nazo zonke izingibe, ukuthi ungenza kanjani ubungane ngempumelelo phakathi kwe-OpenEdge ne-Oracle.
Kwenzeke kanjani: ukulayisha idatha ku-QCD ngokwabelana ngamafayela
Okokuqala, amanye amaqiniso mayelana nengqalasizinda yethu. Inani labasebenzisi abasebenzayo be-database cishe izinkulungwane eziyi-15. Ivolumu yazo zonke izingobo zolwazi ezikhiqizayo, okuhlanganisa i-replica nokubekwe eceleni, ingama-600 TB, isizindalwazi esikhulu kunazo zonke singu-16,5 TB. Ngesikhathi esifanayo, imininingwane yolwazi ihlale igcwaliswa: ngonyaka odlule nje kuphela, cishe i-120 TB yedatha ekhiqizayo yengeziwe. Uhlelo lunikwa amandla amaseva angaphambili ayi-150 kupulatifomu ye-x86. Izingosi zolwazi zisingathwa kumaseva angama-21 epulatifomu ye-IBM.
Izinhlelo zangaphambili, amasistimu amabhange ayisisekelo ahlukahlukene kanye nezinsizakalo zasebhange zihlanganiswe ne-OpenEdge Progress (BISCUIT IBS) ngebhasi le-Sonic ESB. Ukulayisha idatha ku-QCD kwenzeka ngokushintshaniswa kwefayela. Kuze kube yisikhathi esithile, lesi sixazululo sasinezinkinga ezimbili ezinkulu ngesikhathi esisodwa - ukusebenza okuphansi kokulayisha ulwazi endaweni yokugcina idatha yenkampani (CDW) kanye nesikhathi eside sokwenza ukubuyisana kwedatha (ukubuyisana) nezinye izinhlelo.
Ngakho-ke, saqala ukufuna ithuluzi elingasheshisa lezi zinqubo. Isixazululo sazo zombili izinkinga kwaba umkhiqizo omusha we-Progress OpenEdge - Pro2 CDC (Shintsha Ukuthwebula Kwedatha). Ngakho, ake siqale.
Faka i-Progress OpenEdge ne-Pro2Oracle
Ukuze usebenzise i-Pro2 Oracle kukhompuyutha ye-Windows yomlawuli, kwanele ukufaka i-Progress OpenEdge Developer Kit Classroom Edition, okungaba Mahhala. Imibhalo yokufaka ye-OpenEdge ezenzakalelayo:
I-DLC: C:ProgressOpenEdge
UMSEBENZI: C:OpenEdgeWRK
Izinqubo ze-ETL zidinga inguqulo yamalayisensi e-Progress OpenEdge 11.7+ - okuyi-OE DataServer ye-Oracle kanye ne-4GL Development System. Lawa malayisensi afakwe ne-Pro2. Ukuze isebenze ngokugcwele i-DataServer ye-Oracle enesizindalwazi se-Oracle esikude, i-Full Oracle Client ifakiwe.
Kuseva ye-Oracle udinga ukufaka i-Oracle Database 12+, udale i-database engenalutho bese wengeza umsebenzisi (asimbize cdc).
Ukuze ufake i-Pro2Oracle, landa ukusatshalaliswa kwakamuva esikhungweni sokulanda. Khipha ingobo yomlando kuhla lwemibhalo C:Pro2 (Ukuze ulungiselele i-Pro2 ku-Unix, kusetshenziswa ukusatshalaliswa okufanayo futhi kuyasebenza imigomo efanayo yokumisa).
Ukudala isizindalwazi sokuphindaphinda kwe-cdc
Isizindalwazi sokuphindaphinda cdc (impendulo) I-Pro2 isetshenziselwa ukugcina imininingwane yokumisa, okuhlanganisa imephu yokuphindaphinda, amagama esizindalwazi esiphindaphindekile kanye namathebula azo. Futhi iqukethe umugqa wokuphindaphinda, ohlanganisa amanothi mayelana neqiniso lokuthi umugqa wethebula kusizindalwazi somthombo ushintshile. Idatha esuka kumugqa wokuphindaphinda isetshenziswa izinqubo ze-ETL ukuze kuhlonzwe imigqa okudingeka ikopishelwe ku-Oracle kusukela kusizindalwazi somthombo.
Sakha isizindalwazi se-cdc esihlukile.
Inqubo yokwakha isizindalwazi
- Kuseva yedathabhesi sakha uhla lwemibhalo lwesizindalwazi se-cdc - isibonelo, kuseva /isizindalwazi/cdc/.
- Dala i-dummy yesizindalwazi se-cdc: kopisha i-$DLC/empty cdc
- Nika amandla usekelo lwamafayela amakhulu: proutil cdc -C EnableLargeFiles
- Silungisa umbhalo wokuqala isizindalwazi se-cdc. Imingcele yokuqala kufanele ifane nemingcele yokuqala yesizindalwazi esiphindaphindwe.
- Siqala i-database ye-cdc.
- Xhuma kusizindalwazi se-cdc bese ulayisha i-schema se-Pro2 efayeleni cdc.df, efakwe ne-Pro2.
- Sakha abasebenzisi abalandelayo kusizindalwazi se-cdc:
i-pro2adm - yokuxhuma kuphaneli yokuphatha ye-Pro2;
pro2etl - yokuxhuma izinqubo ze-ETL (ReplBatch);
pro2cdc - yokuxhuma izinqubo ze-CDC (CDCBatch);
Ivula i-OpenEdge Guqula Ukuthwebula Idatha
Manje ake sivule indlela ye-CDC ngokwayo, ngosizo lokuthi idatha izophindwaphindwa endaweni yobuchwepheshe eyengeziwe. Kusizindalwazi ngasinye somthombo we-Progress OpenEdge, udinga ukwengeza izindawo ezihlukene zokugcina lapho idatha yomthombo izophindwaphindwa khona, futhi wenze kusebenze umshini ngokwawo usebenzisa umyalo. umkhiqizo.
Isibonelo senqubo yesizindalwazi se-bisquit
- Ikopisha kukhathalogi C:Pro2db ifayela cdcadd.st kusizindalwazi somthombo we-bisquit.
- Sichaza ku cdcadd.st usayizi omisiwe wezifunda "ReplCDCArea" ΠΈ "ReplCDCArea_IDX". Ungangeza izindawo ezintsha zokugcina ku-inthanethi: prostrct addonline bisquit cdcadd.st
- Vula i-OpenEdge CDC:
proutil bisquit -C enablecdc indawo "ReplCDCArea" indexarea "ReplCDCArea_IDX" - Abasebenzisi abalandelayo kufanele badalwe kusizindalwazi somthombo ukuhlonza izinqubo ezisebenzayo:
a. i-pro2adm - yokuxhuma kuphaneli yokuphatha ye-Pro2.
b. pro2etl - yokuxhuma izinqubo ze-ETL (ReplBatch).
c. pro2cdc - yokuxhuma izinqubo ze-CDC (CDCBatch).
Ukudala Isibambi Se-Schema se-DataServer se-Oracle
Okulandelayo, sidinga ukwakha isizindalwazi Somnikazi We-Schema kuseva lapho idatha evela ku-Progress DBMS izophindwaphindwa ku-Oracle DBMS. I-DataServer Schema Holder iyisizindalwazi se-Progress OpenEdge esingenalutho esingenabasebenzisi noma idatha yohlelo lokusebenza, equkethe imephu yokuxhumana phakathi kwamathebula omthombo namathebula e-Oracle angaphandle.
Isizindalwazi Sesibambi Se-Schema se-Progress OpenEdge DataServer se-Oracle ye-Pro2 kufanele sibekwe kuseva yenqubo ye-ETL; sidalelwa ngokwehlukana kwegatsha ngalinye.
Usenza kanjani isibambi se-Schema
- Khipha ukusatshalaliswa kwe-Pro2 kuhla lwemibhalo /pro2
- Dala futhi uye kunkomba /pro2/dbsh
- Dala i-database ye-Schema Holder usebenzisa umyalo kopisha i-$DLC/i-bisquitsh engenalutho
- Ukwenza ukuguqulwa ibhisikidi ekubhalweni kwekhodi okudingekile - isibonelo, ku-UTF-8 uma imininingwane ye-Oracle inombhalo wekhodi we-UTF-8: i-proutil bisquitsh -C convchar iguqulela i-UTF-8
- Ngemva kokudala i-database engenalutho ibhisikidi xhuma kuyo ngemodi yomsebenzisi oyedwa: pro bisquits
- Ake siye kusichazamazwi sedatha: Amathuluzi -> Isichazamazwi Sedatha -> Iseva Yedatha -> Izinsiza ze-ORACLE -> Dala i-DataServer Schema
- Yethula Isibambi Se-Schema
- Ukusetha i-Oracle DataServer broker:
a. Qala i-AdminServer.
proadsv -qalisa
b. Ukuqala komthengisi we-Oracle DataServer
oraman -name orabroker1 -start
Ukusetha iphaneli yokulawula nesikimu sokuphindaphinda
Kusetshenziswa iphaneli yokulawula ye-Pro2, amapharamitha we-Pro2 ayalungiswa, okuhlanganisa ukusetha isikimu sokuphindaphinda nokukhiqiza izinqubo ze-ETL (I-Processor Library), izinhlelo eziyinhloko zokuvumelanisa (I-Bulk-Copy Processor), izibangeli zokuphindaphinda nezinqubomgomo ze-OpenEdge CDC. Kukhona futhi amathuluzi ayinhloko okuqapha nokuphatha izinqubo ze-ETL ne-CDC. Okokuqala, simisa amafayela wepharamitha.
Ungawamisa kanjani amafayela wepharamitha
- Iya kukhathalogi C:Pro2bpreplScripts
- Vula ifayela ukuze lihlelwe replProc.pf
- Engeza izinhlaka zokuxhuma kusizindalwazi sokuphindaphinda kwe-cdc:
# Isizindalwazi sokuphindaphinda
-db cdc -ld repl -H <igama lomphathi wesizindalwazi esikhulu> -S <i-database broker port cdc>
-U pro2admin -P <iphasiwedi> - Engeza ku replProc.pf amapharamitha wokuxhuma kusizindalwazi somthombo kanye nesibambi se-Schema ngendlela yamafayela epharamitha. Igama lefayela lepharamitha kufanele lifane negama lesizindalwazi somthombo esixhunyiwe.
# Xhuma kuyo yonke imithombo ephindaphindiwe ye-BISQUIT
-pf bpreplscriptsbisquit.pf - Engeza ku replProc.pf amapharamitha wokuxhuma Kumphathi We-Schema.
#Target Pro DB Schema Holder
-db bisquitsh -ld bisquitsh
-H <ETL inqubo yegama lomsingathi>
-S <biskuitsh broker port>
-db bisquitsql
-ld bisquitsql
-dt ORACLE
-S 5162 -H <Igama lomphathi we-Oracle broker>
-DataService orabroker1 - Londoloza ifayela lepharamitha replProc.pf
- Okulandelayo, udinga ukudala futhi uvule ukuze uhlele amafayela epharamitha kusizindalwazi somthombo ngamunye oxhumekile kumkhombandlela C:Pro2bpreplScripts: bisquit.pf. Ifayela ngalinye le-pf liqukethe imingcele yokuxhuma kusizindalwazi esihambisanayo, isibonelo:
-db bisquit -ld bisquit -H <igama lomsingathi> -S <broker port>
-U pro2admin -P <iphasiwedi>
Ukuze ulungise izinqamuleli ze-Windows, udinga ukuya kuhla lwemibhalo C:Pro2bpreplScripts bese uhlela βPro2 β Administrationβ isinqamuleli. Ukuze wenze lokhu, vula izakhiwo zesinqamuleli nasemgqeni Qala ku- khombisa umkhombandlela wokufaka we-Pro2. Umsebenzi ofanayo kufanele wenziwe kuzinqamuleli ze-βPro2 β Editorβ kanye ne-βRunBulkLoaderβ.
Ukusethwa Kwe-Pro2 Administration: Ilayisha Ukucushwa Kwasekuqaleni
Masiqalise ikhonsoli.
Iya ku-βDB Mapβ.
Ukuze uxhumanise imininingwane yolwazi ku-Pro2 β Administration, iya kuthebhu Imephu ye-DB. Engeza imephu yolwazi lwemithombo - Isibambi se-Schema - Oracle.
Iya kuthebhu Mapping. Kusohlwini Isizindalwazi Somthombo Ngokuzenzakalelayo, isizindalwazi sokuqala esixhunyiwe somthombo siyakhethwa. Ngakwesokudla sohlu kufanele kube nombhalo Wonke Ama-database Axhunyiwe β isizindalwazi esikhethiwe sixhunyiwe. Ngezansi kwesokunxele kufanele ubone uhlu lwamathebula okuthi Inqubekela-phambili kusuka ku-bisquit. Ngakwesokudla kunohlu lwamathebula avela kusizindalwazi se-Oracle.
Ukudala izikimu ze-SQL nezizindalwazi ku-Oracle
Ukuze udale imephu yokuphindaphinda, kufanele uqale ukhiqize I-SQL schema e-Oracle. Ku-Pro2 Administration senza into yemenyu Amathuluzi -> Khiqiza Ikhodi -> I-Target Schema, bese kuba sebhokisini lengxoxo Khetha i-Database khetha isizindalwazi somthombo owodwa noma ngaphezulu bese uwahambisa kwesokudla.
Chofoza okuthi KULUNGILE bese ukhetha inkomba ukuze ulondoloze izikimu ze-SQL.
Okulandelayo sakha isisekelo. Lokhu kungenziwa, isibonelo, ngokusebenzisa Unjiniyela we-Oracle SQL. Ukuze wenze lokhu, sixhuma ku-database ye-Oracle futhi silayishe i-schema yokwengeza amatafula. Ngemva kokushintsha ukwakheka kwamathebula e-Oracle, udinga ukubuyekeza izikimu ze-SQL Kumphathi We-Schema.
Ngemuva kokuthi ukulanda kuqedwe ngempumelelo, phuma kusizindalwazi se-bisquitsh bese uvula iphaneli yokuphatha ye-Pro2. Amathebula asuka kusizindalwazi se-Oracle kufanele avele kuthebhu yemephu kwesokudla.
Imephu yethebula
Ukuze udale imephu yokuphindaphinda, kuphaneli yokuphatha ye-Pro2, hamba kuthebhu ye-Mapping bese ukhetha isizindalwazi somthombo. Chofoza Amathebula Emephu, khetha Khetha Izinguquko kwesokunxele samathebula okufanele aphindaphindwe ku-Oracle, awahambise kwesokudla bese uqinisekise okukhethiwe. Imephu izodalwa ngokuzenzakalelayo kumathebula akhethiwe. Siphinda umsebenzi ukuze sakhe imephu yokuphindaphinda yezinye izingosi zolwazi zomthombo.
Ikhiqiza Umtapo Wezincwadi Wokuphindaphinda we-Pro2 kanye nezinhlelo ze-Bulk-Copy Processor
I-Replication Processor Library yakhelwe izinqubo zokuphindaphinda ngokwezifiso (ama-ETL) acubungula ulayini wokuphindaphinda we-Pro2 futhi aphushe izinguquko kusizindalwazi se-Oracle. Izinhlelo zomtapo wolwazi wokuphindaphinda zigcinwa ngokuzenzakalelayo kuhlu lwemibhalo ngemva kwesizukulwane bprepl/repl_proc (PROC_DIRECTORY ipharamitha). Ukuze ukhiqize ilabhulali yokucubungula ukuphindaphinda, yiya ku Amathuluzi -> Khiqiza Ikhodi -> Umtapowolwazi Wokuphrosesa. Ngemva kokuqedwa kokukhiqiza, izinhlelo zizovela ohlwini lwemibhalo bprepl/repl_proc.
Izinhlelo ze-Bulk Processor zisetshenziselwa ukuvumelanisa imithombo yolwazi yenqubekela-phambili nesizindalwazi esiqondiwe se-Oracle ngokusekelwe olimini lokuhlela lwe-Progress ABL (4GL). Ukuze uzikhiqize, hamba entweni yemenyu Amathuluzi -> Khiqiza Ikhodi -> I-Bulk-Copy Processor. Ebhokisini lengxoxo le-Khetha Isizindalwazi, khetha imithombo yolwazi, ihambise ngakwesokudla kwewindi bese uchofoza OK. Ngemva kokuqedwa kokukhiqiza, izinhlelo zizovela ohlwini lwemibhalo bpreplrepl_mproc.
Ukusetha izinqubo zokuphindaphinda ku-Pro2
Ukuhlukanisa amathebula abe amasethi anikezwa umucu wokuphindaphinda ohlukile kuthuthukisa ukusebenza nokusebenza kahle kwe-Pro2 Oracle. Ngokuzenzakalelayo, konke ukuxhumana okudalwe kumephu yokuphindaphinda yamathebula amasha okuphindaphinda kuhlotshaniswa nenombolo yochungechunge 1. Kunconywa ukuhlukanisa amatafula abe imicu ehlukene.
Ulwazi mayelana nesimo sezintambo eziphindaphindayo luboniswa esikrinini Sokuphatha se-Pro2 kuthebhu ye-Monitor esigabeni Sesimo Sokuphindaphinda. Incazelo enemininingwane yamanani epharamitha ingatholakala kumadokhumenti e-Pro2 (uhla lwemibhalo C:Pro2Docs).
Dala futhi wenze kusebenze izinqubomgomo ze-CDC
Izinqubomgomo ziyisethi yemithetho yenjini ye-OpenEdge CDC yokuqapha izinguquko kumathebula. Ngesikhathi sokubhala, i-Pro2 isekela kuphela izinqubomgomo ze-CDC ezineleveli 0, okungukuthi, iqiniso kuphela eligadwayo qopha izinguquko.
Ukuze udale inqubomgomo ye-CDC, kuphaneli yokulawula, hamba kuthebhu ye-Mapping, khetha isizindalwazi somthombo bese uchofoza inkinobho ethi Engeza/Susa Izinqubomgomo. Kuwindi elithi Khetha Izinguquko elivulekayo, khetha ohlangothini lwesobunxele bese uya kwesokudla amathebula okudingeka uwadale noma ususe inqubomgomo ye-CDC.
Ukuze wenze kusebenze, vula ithebhu ye-Mapping futhi, khetha isizindalwazi somthombo bese uchofoza inkinobho (Ku)Sebenzisa Izinqubomgomo. Khetha futhi uye kwesokudla sethebula izinqubomgomo ezidinga ukwenziwa zisebenze, chofoza okuthi KULUNGILE. Ngemva kwalokhu ziphawulwa ngokuluhlaza. Ngokusebenzisa (Ku)Sebenzisa Izinqubomgomo Ungakwazi futhi ukuvala izinqubomgomo ze-CDC. Konke ukuthengiselana kwenziwa ku-inthanethi.
Ngemuva kokuthi inqubomgomo ye-CDC yenziwe yasebenza, amanothi mayelana namarekhodi ashintshiwe alondolozwa endaweni yokugcina "ReplCDCArea" ngokusho kwesizindalwazi somthombo. Lawa manothi azocutshungulwa ngenqubo ekhethekile I-CDCBatch, okusekelwe kuzo kuzodala amanothi kulayini wokuphindaphinda we-Pro2 kusizindalwazi cdc (impendulo).
Ngakho, sinemigqa emibili yokuphindaphinda. Isigaba sokuqala yi-CDCBatch: kusuka kusizindalwazi somthombo, idatha kuqala iya kusizindalwazi se-CDC esimaphakathi. Isigaba sesibili yilapho idatha idluliswa isuka kusizindalwazi se-CDC iye ku-Oracle. Lesi isici sesakhiwo samanje nomkhiqizo ngokwawo - kuze kube manje abathuthukisi abakwazanga ukusungula ukuphindaphinda okuqondile.
Ukuvumelanisa okuyisisekelo
Ngemva kokunika amandla indlela ye-CDC nokusetha iseva yokuphindaphinda ye-Pro2, sidinga ukuqalisa ukuvumelanisa kokuqala. Umyalo wokuvumelanisa wokuqala:
/pro2/bprepl/Script/replLoad.sh bisquit table-name
Ngemva kokuqedwa kokuvumelanisa kokuqala, izinqubo zokuphindaphinda zingaqalwa.
Ukuqala kwezinqubo zokuphindaphinda
Ukuze uqale izinqubo zokuphindaphinda udinga ukusebenzisa iskripthi replbatch.sh. Ngaphambi kokuqala, qiniseka ukuthi kukhona imibhalo ye-replbatch yayo yonke imicu - replbatch1, replbatch2, njll. Uma konke sekumi ngomumo, vula umugqa womyalo (isibonelo, proenv), hamba ohlwini lwemibhalo /bprepl/scripts bese uqala iskripthi. Kuphaneli yokulawula, sihlola ukuthi inqubo ehambisanayo ithole isimo se-RUNNING.
Imiphumela
Ngemva kokuqaliswa, sikusheshise kakhulu ukulayishwa kolwazi endaweni yokugcina idatha yebhizinisi. Idatha ingena ngokuzenzakalelayo ku-Oracle ku-inthanethi. Asikho isidingo sokumosha isikhathi usebenzisa imibuzo ehlala isikhathi eside ukuqoqa idatha kumasistimu ahlukene. Ngaphezu kwalokho, kulesi sixazululo inqubo yokuphindaphinda ingacindezela idatha, ebuye ibe nomthelela omuhle ngesivinini. Manje ukubuyisana kwansuku zonke kwesistimu ye-BISKVIT nezinye izinhlelo kwaqala ukuthatha imizuzu engu-15-20 esikhundleni samahora angu-2-2,5, futhi ukubuyisana okuphelele kwathatha amahora ambalwa esikhundleni sezinsuku ezimbili.
Source: www.habr.com