Kubva 1999, kushandira hofisi yekumashure, bhangi redu rakashandisa iyo yakasanganiswa yebhangi system BISKVIT paProgress OpenEdge papuratifomu, iyo inoshandiswa zvakanyanya pasirese, kusanganisira mune yezvemari. Kuita kweDBMS iyi kunoita kuti uverenge marekodhi anosvika miriyoni kana kupfuura pasekondi mune imwe dhatabhesi (DB). Yedu Progress OpenEdge masevhisi anosvika miriyoni imwe chete yedhipoziti uye angangoita 1,5 miriyoni zvibvumirano zvezvigadzirwa zvinoshanda (zvikwereti zvemotokari uye mbavha), uye zvakare ine basa rekugadziriswa kwese nemutongi (Central Bank) uye SWIFT.
Tichishandisa Progress OpenEdge, takatarisana nekudiwa kwekuita kuti ishande neOracle DBMS. Pakutanga, bundle iri raive bhodhoro rezvivakwa zvedu - kudzamara taisa nekugadzirisa Pro2 CDC - Progress chigadzirwa chinokutendera kuti utumire data kubva kuBudiriro DBMS kuenda kuOracle DBMS zvakananga, online. Mune ino positi isu tichakuudza zvakadzama, nemakomba ese, maitiro ekuita shamwari zvinobudirira pakati peOpenEdge neOracle.
Zvakaitika sei: kurodha data kuQCD kuburikidza nekugovana faira
Chekutanga, zvimwe chokwadi nezvezvivakwa zvedu. Nhamba yevashandisi vanoshingaira ve database inenge 15 zviuru. Huwandu hweese madhatabhesi anogadzira, kusanganisira replica uye yekumira, i600 TB, iyo yakakura dhatabhesi ndeye 16,5 TB. Panguva imwecheteyo, dhatabhesi iri kugara ichizadzikiswa: pamusoro pegore rapfuura chete, inenge 120 TB yedata inobereka yakawedzerwa. Iyo sisitimu inofambiswa ne150 kumberi maseva pane x86 papuratifomu. Iwo dhatabhesi anogarwa pamakumi maviri nembiri IBM papuratifomu maseva.
Kumberi-kumagumo masisitimu, akasiyana epakati mabhengi masisitimu uye masevhisi ekubhengi akabatanidzwa neOpenEdge Progress (BISCUIT IBS) kuburikidza nebhazi reSonic ESB. Kuisa data kuQCD kunoitika kuburikidza nekuchinjana kwefaira. Kusvika pane imwe nguva, mhinduro iyi yaive nematambudziko maviri makuru panguva imwechete - kuderera kwekuita kwekuisa ruzivo mucorporate data warehouse (CDW) uye nguva yakareba yekuita data reconciliation (kuyananisa) nemamwe masisitimu.
Naizvozvo, takatanga kutsvaga chishandiso chaigona kukurumidzira maitiro aya. Mhinduro kumatambudziko ese ari maviri yaive itsva Progress OpenEdge chigadzirwa - Pro2 CDC (Change Data Capture). Saka, ngatitange.
Isa Progress OpenEdge uye Pro2Oracle
Kuti umhanye Pro2 Oracle pakombuta yeWindows, zvakakwana kuisa Progress OpenEdge Developer Kit Classroom Edition, inogona zvemahara. Default OpenEdge yekuisa madhairekitori:
DLC: C: ProgressOpenEdge
WRK: C:OpenEdgeWRK
Maitiro eETL anoda Progress OpenEdge rezinesi shanduro 11.7+ - iyo OE DataServer yeOracle uye 4GL Development System. Aya marezinesi anosanganisirwa nePro2. Kushanda kwakazara kweDataServer yeOracle ine kure kure Oracle dhatabhesi, Yakazara Oracle Client inoiswa.
PaOracle server iwe unofanirwa kuisa Oracle Database 12+, gadzira dhatabhesi risina chinhu uye wedzera mushandisi (ngatimufonerei. cdc).
Kuisa Pro2Oracle, dhawunirodha yazvino kugovera kubva kunzvimbo yekurodha. Vhura iyo archive mudhairekitori C:Pro2 (Kugadzirisa Pro2 paUnix, kugovera kwakafanana kunoshandiswa uye misimboti imwechete yekumisikidza inoshanda).
Kugadzira cdc replication database
Replication database cdc (repl) Pro2 inoshandiswa kuchengetedza ruzivo rwekugadzirisa, kusanganisira iyo mepu yekudzokorora, mazita eakadzokororwa dhatabhesi uye matafura avo. Iyo ine zvakare mutsara wekudzokorora, unosanganisira manotsi nezve chokwadi chekuti tafura mutsara mune sosi dhatabhesi yachinja. Dhata kubva kumutsara wekudzokorora inoshandiswa neETL maitiro kuona mitsara inoda kukopwa kuOracle kubva kudhatabhesi.
Isu tiri kugadzira yakaparadzana cdc dhatabhesi.
Maitiro ekugadzira database
- Pa database server tinogadzira dhairekitori reiyo cdc dhatabhesi - semuenzaniso, pane server /database/cdc/.
- Gadzira dummy ye cdc dhatabhesi: procopy $DLC/empty cdc
- Gonesa rutsigiro rwemafaira makuru: proutil cdc -C EnableLargeFiles
- Isu tinogadzirira script yekutanga cdc database. Mamiriro ekutanga anofanirwa kunge akafanana neanotanga maparamita edhatabhesi rakadzokororwa.
- Isu tinotanga iyo cdc database.
- Batanidza kune cdc dhatabhesi uye rodha iyo Pro2 schema kubva pafaira cdc.df, iyo inosanganisirwa nePro2.
- Isu tinogadzira vashandisi vanotevera mune cdc dhatabhesi:
pro2adm - yekubatanidza kubva kuPro2 administrative panel;
pro2etl - yekubatanidza ETL maitiro (ReplBatch);
pro2cdc - yekubatanidza CDC maitiro (CDCBatch);
Kuita OpenEdge Shandura Dhata Capture
Zvino ngatibatidzei iyo CDC meshini pachayo, nerubatsiro rweiyo data ichadzokororwa kune imwezve tekinoroji nzvimbo. Kune yega yega Progress OpenEdge sosi dhatabhesi, iwe unofanirwa kuwedzera dzakasiyana nzvimbo dzekuchengetera mukati umo iyo sosi data ichadzokororwa, uye shandisa iyo mechanisi pachayo uchishandisa murairo. proutil.
Muenzaniso maitiro eiyo bisquit dhatabhesi
- Kukopa kubva kukhathalogi C:Pro2db faira cdcadd.st kune bisquit source database directory.
- Isu tinotsanangura mu cdcadd.st yakagadziriswa saizi zviyero zvematunhu "ReplCDCArea" ΠΈ "ReplCDCArea_IDX". Iwe unogona kuwedzera nzvimbo nyowani dzekuchengetera online: prostrct addonline bisquit cdcadd.st
- Shandisa OpenEdge CDC:
proutil bisquit -C inogonesa cdc nharaunda "ReplCDCArea" indexarea "ReplCDCArea_IDX" - Vanotevera vashandisi vanofanirwa kugadzirwa mune dhatabhesi kuti vaone mafambiro ekuita:
a. pro2adm - yekubatanidza kubva kuPro2 administrative panel.
b. pro2etl - yekubatanidza ETL maitiro (ReplBatch).
c. pro2cdc - yekubatanidza CDC maitiro (CDCBatch).
Kugadzira Schema Holder yeDataServer yeOracle
Tevere, isu tinofanirwa kugadzira dhatabhesi reSchema Holder pane sevha uko data kubva kuProgress DBMS ichadzokororwa kuOracle DBMS. DataServer Schema Holder iDataServer OpenEdge dhatabhesi isina vashandisi kana data rekushandisa, ine mepu yekunyorerana pakati pematafura matafura uye ekunze Oracle matafura.
Iyo Schema Holder dhatabhesi yeKufambira mberi OpenEdge DataServer yeOracle yePro2 inofanirwa kunge iri paETL process server; inogadzirwa zvakasiyana kune rimwe nerimwe bazi.
Maitiro ekugadzira Schema Holder
- Bvisa kugovera kwePro2 mudhairekitori /pro2
- Gadzira uye enda kune dhairekitori /pro2/dbsh
- Gadzira dhatabhesi reSchema Holder uchishandisa rairo kopi $DLC/empty bisquitsh
- Kuita kutendeuka bisquits mune inodiwa encoding - semuenzaniso, muUTF-8 kana Oracle dhatabhesi ine UTF-8 encoding: proutil bisquitsh -C convchar shandura UTF-8
- Mushure mekugadzira database isina chinhu bisquits batanidza kwairi mune imwechete-mushandisi modhi: pro bisquits
- Ngatiendei kuData Dictionary: Zvishandiso -> Dhata Dhata -> DataServer -> ORACLE Zvishandiso -> Gadzira DataServer Schema
- Tanga Schema Holder
- Kumisikidza iyo Oracle DataServer broker:
a. Tanga AdminServer.
proadsv -start
b. Kutanga kweOracle DataServer broker
oraman -zita orabroker1 -kutanga
Kumisikidza iyo administrative panel uye replication scheme
Uchishandisa Pro2 administrative panel, Pro2 paramita inogadziriswa, kusanganisira kuseta chirongwa chekudzokorora uye kugadzira ETL maitiro (Processor Library), mapurogiramu ekutanga ekuyananisa (Bulk-Copy processor), zvinokonzeresa uye OpenEdge CDC marongero. Kune zvakare maturusi ekutanga ekutarisa uye kubata ETL neCDC maitiro. Chokutanga pane zvose, tinogadzirisa mafaira eparameter.
Maitiro ekugadzirisa parameter mafaera
- Enda kukhathalogi C:Pro2bpreplScripts
- Vhura faira rekugadzirisa replProc.pf
- Wedzera ma paramita ekubatanidza kune cdc replication database:
# Replication Database
-db cdc -ld repl -H <main database hostname> -S <database broker port cdc>
-U pro2admin -P <password> - Wedzera kune replProc.pf yekubatanidza ma paramita kune dhatabhesi uye Schema Holder muchimiro chemafaira emafaira. Iro zita rema parameter faira rinofanira kuenderana nezita reiyo source database iri kubatana.
# Batanidza kune ese akadzokororwa masosi BISQUIT
-pf bpreplscriptsbisquit.pf - Wedzera kune replProc.pf parameters yekubatanidza kune Schema Holder.
#Target Pro DB Schema Holder
-db bisquitsh -ld bisquitsh
-H <ETL process host zita>
-S <biskuitsh broker port>
-db bisquitsql
-ld bisquitsql
-dt ORACLE
-S 5162 -H <Oracle broker hostname>
-DataService orabroker1 - Sevha iyo parameter faira replProc.pf
- Tevere, iwe unofanirwa kugadzira uye kuvhura ekugadzirisa parameter mafaera kune yega yega yakabatana sosi dhatabhesi mudhairekitori C:Pro2bpreplScripts: bisquit.pf. Imwe neimwe pf faira ine ma paramita ekubatanidza kune inoenderana dhatabhesi, semuenzaniso:
-db bisquit -ld bisquit -H <hostname> -S <broker port>
-U pro2admin -P <password>
Kuti ugadzirise Windows mapfupi, unofanirwa kuenda kune dhairekitori C:Pro2bpreplScripts uye gadzirisa "Pro2 - Administration" nzira yekudimbudzira. Kuti uite izvi, vhura zvimiro zvekudimbudzira uye mumutsara Tanga mukati ratidza Pro2 yekuisa dhairekitori. Kushanda kwakafanana kunofanirwa kuitwa kune "Pro2 - Mharidzo" uye "RunBulkLoader" mapfupi.
Pro2 Administration Setup: Loading Initial Configuration
Ngatitangei console.
Enda ku "DB Mepu".
Kuti ubatanidze dhatabhesi muPro2 - Kutonga, enda kune iyo tab DB Mepu. Wedzera mepu yezvinyorwa zvinyorwa - Schema Holder - Oracle.
Enda kune tab Mepu. Akanyorwa Source Database Nekumisikidza, yekutanga yakabatana source database inosarudzwa. Kurudyi kwechinyorwa panofanira kunge kune chinyorwa All Databases Akabatanidzwa - iyo dhatabhesi yakasarudzwa yakabatana. Pazasi kuruboshwe iwe unofanirwa kuona runyoro rweKufambira mberi kwematafura kubva kubisquit. Kurudyi kune rondedzero yematafura kubva kuOracle dhatabhesi.
Kugadzira SQL schemas uye dhatabhesi muOracle
Kuti ugadzire mepu yekudzokorora, unofanirwa kutanga wagadzira SQL schema muOracle. MuPro2 Administration isu tinoshandisa menyu chinhu Zvishandiso -> Gadzira kodhi -> Target Schema, ipapo mubhokisi rekutaura Sarudza Dhatabhesi sarudza imwe kana akawanda dhatabhesi uye uende navo kurudyi.
Dzvanya OK uye sarudza dhairekitori kuchengetedza SQL schemas.
Zvadaro tinogadzira nheyo. Izvi zvinogona kuitwa, semuenzaniso, kuburikidza Oracle SQL Mushambadzi. Kuti tiite izvi, tinobatana neOracle dhatabhesi uye torodha schema yekuwedzera matafura. Mushure mekushandura kuumbwa kwematafura eOracle, iwe unofanirwa kugadzirisa iyo SQL schemas muSchema Holder.
Mushure mekunge kurodha kwapera zvinobudirira, buda mubhisquitsh dhatabhesi uye vhura iyo Pro2 administrative panhi. Matafura kubva kuOracle dhatabhesi anofanirwa kuoneka paMepu tebhu kurudyi.
Tafura mapping
Kuti ugadzire mepu yekudzokorora, muPro2 administrative panhi, enda kuMepu tebhu uye sarudza iyo dhatabhesi. Dzvanya paMepu Matafura, sarudza Sarudza Shanduko kuruboshwe rwematafura anofanirwa kudzokororwa muOracle, aendese kurudyi uye simbisa iyo sarudzo. Mepu ichagadzirwa otomatiki yematafura akasarudzwa. Isu tinodzokorora kushanda kuti tigadzire mepu yekudzokorora kune mamwe dhatabhesi.
Kugadzira Pro2 Replication processor Raibhurari uye Bulk-Copy processor Zvirongwa
Iyo Replication processor Raibhurari yakagadzirirwa tsika dzekudzokorora maitiro (ETLs) iyo inogadzirisa Pro2 yekudzokorora mutsara uye kusundira shanduko kuOracle dhatabhesi. Replication processor raibhurari zvirongwa zvinochengetwa otomatiki kune dhairekitori mushure mechizvarwa bprepl/repl_proc (PROC_DIRECTORY parameter). Kuti ugadzire raibhurari yekudzokorora processor, enda ku Zvishandiso -> Gadzira kodhi -> processor Raibhurari. Mushure mekunge chizvarwa chapera, mapurogiramu achaonekwa mudhairekitori bprepl/repl_proc.
Zvirongwa zveBulk Load Processor zvinoshandiswa kuwiriranisa dura reBudiriro dhatabhesi nechinangwa cheOracle dhatabhesi zvichibva paPurogiramu ABL (4GL) mutauro wekuronga. Kuti uzvigadzire, enda kune chinhu chemenu Zvishandiso -> Gadzira Code -> Bulk-Copy processor. MuSarudzo Database dialog box, sarudza iyo dhatabhesi dhatabhesi, fambisa iwo kurudyi rwehwindo uye tinya OK. Mushure mekunge chizvarwa chapera, mapurogiramu achaonekwa mudhairekitori bpreplrepl_mproc.
Kumisikidza maitiro ekudzokorora muPro2
Kutsemura matafura mumaseti anoshandirwa netambo yekudzokorodza yakaparadzana kunovandudza mashandiro uye kugona kwePro2 Oracle. Nekusagadzika, zvese zvinongedzo zvinogadzirwa mumepu yekudzokorora yematafura matsva ekudzokorora zvinosanganiswa neshinda nhamba 1. Zvinokurudzirwa kuparadzanisa matafura mushinda dzakasiyana.
Ruzivo nezve chimiro chetambo dzekudzokorora inoratidzwa paPro2 Administration skrini muMonitor tebhu muchikamu cheReplication Status. Tsanangudzo yakadzama yemhando dzeparameter inogona kuwanikwa mune Pro2 zvinyorwa (dhairekitori C: Pro2Docs).
Gadzira uye shandisa mitemo yeCDC
Mitemo seti yemitemo yeOpenEdge CDC injini yekutarisa shanduko kumatafura. Panguva yekunyora, Pro2 inongotsigira CDC marongero ane nhanho 0, ndiko kuti, chokwadi chete chinotariswa rekodhi shanduko.
Kuti ugadzire mutemo weCDC, pane yekutonga, enda kuMepu tab, sarudza iyo dhatabhesi uye tinya bhatani reKuwedzera/Bvisa Policies. MuSarudzo Shanduko hwindo rinovhurwa, sarudza kuruboshwe uye enda kurudyi matafura aunoda kugadzira kana kudzima mutemo weCDC.
Kuti uite activate, vhura iyo Mepu tebhu zvakare, sarudza iyo dhatabhesi uye tinya bhatani (In) Activate Policies. Sarudza uye uende kudivi rekurudyi retafura iyo mitemo inoda kushandiswa, tinya OK. Mushure meizvi vanoiswa chiratidzo negirini. Nokushandisa (In) Activate Policies Iwe unogona zvakare kudzima mitemo yeCDC. Zvese zvekutengesa zvinoitwa online.
Mushure mekunge mutemo weCDC waitwa, manotsi nezve marekodhi akagadziridzwa anochengetwa kunzvimbo yekuchengetera "ReplCDCArea" zvinoenderana neiyo source database. Izvi zvinyorwa zvichagadziriswa kuburikidza nehurongwa hunokosha CDCBatch, iyo yakavakirwa pavari ichagadzira zvinyorwa muPro2 replication queue mu database cdc (repl).
Saka, tine mitsetse miviri yekudzokorora. Nhanho yekutanga ndeye CDCBatch: kubva kudhatabhesi, iyo data inotanga kuenda kune yepakati CDC dhatabhesi. Yechipiri nhanho ndipo painotamiswa data kubva kuCDC dhatabhesi kuenda kuOracle. Ichi chikamu chezvivakwa zvazvino uye chigadzirwa pachacho - kusvika parizvino vagadziri havasati vakwanisa kumisikidza kudzokorora kwakananga.
Primary synchronization
Mushure mekugonesa iyo CDC mashini uye kumisikidza iyo Pro2 replication server, isu tinofanirwa kutanga iyo yekutanga kuwiriranisa. Initial synchronization command:
/pro2/bprepl/Script/replLoad.sh bisquit tafura-zita
Mushure mekunge kuwiriranisa kwekutanga kwapera, maitiro ekudzokorora anogona kutangwa.
Kutanga kwekudzokorora maitiro
Kuti utange kudzokorora maitiro unofanirwa kumhanyisa script replbatch.sh. Usati watanga, ita shuwa kuti pane replbatch zvinyorwa zvetambo dzese - replbatch1, replbatch2, nezvimwe. Kana zvese zviri panzvimbo, vhura mutsara wekuraira (semuenzaniso, proenv), enda kudhairekitori /bprepl/scripts uye tanga script. Mudare rekutonga, tinotarisa kuti maitiro anoenderana agamuchira iyo RUNNING mamiriro.
Mhinduro
Mushure mekuita, isu takamhanyisa zvakanyanya kurodha kweruzivo kune yekambani data warehouse. Iyo data inongopinda muOracle online. Hapana chikonzero chekutambisa nguva uchimhanyisa mimwe mibvunzo yakareba-inomhanya kuunganidza data kubva kune akasiyana masisitimu. Mukuwedzera, mune iyi mhinduro nzira yekudzokorora inogona kudzvinyirira data, iyo inewo mhedzisiro yakanaka pakumhanya. Ikozvino kuyananiswa kwemazuva ese kweBISKVIT system nemamwe masisitimu kwakatanga kutora 15-20 maminetsi panzvimbo ye2-2,5 maawa, uye kuyananiswa kwakakwana kwakatora maawa akawanda pane mazuva maviri.
Source: www.habr.com