Ho tloha ka 1999, ho sebeletsa ofising e ka morao, banka ea rona e sebelisitse tsamaiso e kopanetsoeng ea libanka BISKVIT sethaleng sa Progress OpenEdge, se sebelisoang haholo lefatšeng ka bophara, ho kenyeletsoa le lefapheng la lichelete. Ts'ebetso ea DBMS ena e u fa monyetla oa ho bala lirekoto tse ka bang milione kapa ho feta motsotsoana sebakeng se le seng sa database (DB). Litšebeletso tsa rona tsa Progress OpenEdge tse ka bang limilione tse 1,5 tsa li-depositi tsa motho ka mong le likonteraka tse ka bang limilione tse 22,2 bakeng sa lihlahisoa tse sebetsang (mekoloto ea makoloi le mekoloto ea matlo), hape e ikarabella bakeng sa litokiso tsohle le molaoli (Banka e Bohareng) le SWIFT.
Re sebelisa Progress OpenEdge, re ile ra tobana le tlhoko ea ho e etsa hore e sebetse le Oracle DBMS. Qalong, sephutheloana sena e ne e le tšitiso ea lisebelisoa tsa rona tsa motheo - ho fihlela re kenya le ho lokisa Pro2 CDC - sehlahisoa sa Tsoelo pele se u lumellang hore u romele lintlha tse tsoang ho Progress DBMS ho Oracle DBMS ka ho toba, inthaneteng. Ka poso ena re tla u bolella ka botlalo, ka maraba 'ohle, mokhoa oa ho etsa metsoalle ka katleho pakeng tsa OpenEdge le Oracle.
Ho etsahetse joang: ho kenya data ho QCD ka ho arolelana lifaele
Taba ea pele, lintlha tse ling mabapi le meaho ea rona. Palo ea basebelisi ba mafolofolo ba database ke hoo e ka bang likete tse 15. Bophahamo ba li-database tsohle tse hlahisoang, ho kenyeletsoa replica le standby, ke 600 TB, database e kholo ka ho fetisisa ke 16,5 TB. Ka nako e ts'oanang, li-database li lula li tsosolosoa: selemong se fetileng feela, hoo e ka bang 120 TB ea data e hlahisang e kenyelitsoe. Sistimi e tsamaisoa ke li-server tse ka pele tse 150 sethaleng sa x86. Li-database li tsamaisoa ho li-server tse 21 tsa sethala sa IBM.
Litsamaiso tse ka pele, lits'ebetso tse fapaneng tsa mantlha tsa libanka le lits'ebeletso tsa banka li kopantsoe le OpenEdge Progress (BISCUIT IBS) ka bese ea Sonic ESB. Ho kenya data ho QCD ho etsahala ka phapanyetsano ea lifaele. Ho fihlela ka nako e itseng, tharollo ena e ne e e-na le mathata a mabeli a maholo ka nako e le 'ngoe - ts'ebetso e tlaase ea ho kenya tlhahisoleseding sebakeng sa polokelo ea boitsebiso ba khoebo (CDW) le nako e telele ea ho etsa poelano ea data (poelano) le mekhoa e meng.
Ka hona, re ile ra qala ho batla sesebelisoa se ka potlakisang lits'ebetso tsena. Tharollo ea mathata ka bobeli e ne e le sehlahisoa se secha sa Progress OpenEdge - Pro2 CDC (Change Data Capture). Kahoo, ha re qaleng.
Kenya Progress OpenEdge le Pro2Oracle
Ho tsamaisa Pro2 Oracle khomphuteng ea Windows ea motsamaisi, ho lekane ho kenya Khatiso ea Phaposi ea Phaposi ea Phatlalatso ea OpenEdge Developer Kit, e ka bang teng. bakeng sa mahala. Li-directory tsa ho kenya tsa OpenEdge:
DLC: C:ProgressOpenEdge
MOSEBETSI: C:OpenEdgeWRK
Mekhoa ea ETL e hloka mofuta oa laesense ea Progress OpenEdge 11.7+ - e leng OE DataServer bakeng sa Oracle le 4GL Development System. Lilaesense tsena li kenyelelitsoe le Pro2. Bakeng sa ts'ebetso e felletseng ea DataServer bakeng sa Oracle e nang le database ea Oracle e hole, Full Oracle Client e kentsoe.
Ho seva sa Oracle o hloka ho kenya Oracle Database 12+, theha database e se nang letho 'me u kenye mosebelisi (ha re mo letse cdc).
Ho kenya Pro2Oracle, khoasolla phepelo ea morao-rao ho tsoa setsing sa download. Tlosa polokelo ea polokelo ka har'a directory C:Pro2 (Ho lokisa Pro2 ho Unix, ho sebelisoa kabo e tšoanang 'me ho sebetsa melao-motheo e tšoanang ea tlhophiso).
Ho theha setsi sa polokelo ea boitsebiso ba cdc
Replication database cdc (repl) Pro2 e sebelisoa ho boloka tlhaiso-leseling ea tlhophiso, ho kenyeletsoa 'mapa oa phetisetso, mabitso a li-database tse phetoang le litafole tsa tsona. E boetse e na le queue ea ho pheta-pheta, e nang le lintlha tse mabapi le taba ea hore lethathamo la tafole sebakeng sa mohloli oa boitsebiso le fetohile. Lintlha tse tsoang mokolokong oa ho pheta-pheta li sebelisoa ke mekhoa ea ETL ho tseba mela e lokelang ho kopitsoa ho Oracle ho tsoa polokelong ea mohloli.
Re theha polokelo ea polokelo ea cdc e arohaneng.
Mokhoa oa ho theha database
- Ho seva sa database re theha directory bakeng sa database ea cdc - mohlala, ho seva /database/cdc/.
- Theha dummy bakeng sa database ea cdc: procopy $DLC/empty cdc
- Lumella tšehetso bakeng sa lifaele tse kholo: proutil cdc -C EnableLargeFiles
- Re lokisa script bakeng sa ho qala database ea cdc. Litekanyetso tsa ho qala li tlameha ho tšoana le li-parameter tse qalang tsa database e phetoang.
- Re qala database ea cdc.
- Hokela ho database ea cdc 'me u kenye schema ea Pro2 ho tsoa faeleng cdc.df, e kenyellelitsoeng le Pro2.
- Re theha basebelisi ba latelang polokelong ea cdc:
pro2adm - bakeng sa ho hokela ho tsoa phanele ea tsamaiso ea Pro2;
pro2etl - bakeng sa ho hokahanya mekhoa ea ETL (ReplBatch);
pro2cdc - bakeng sa ho hokahanya mekhoa ea CDC (CDCBatch);
Ho kenya tshebetsong OpenEdge Change Data Capture
Joale a re bulele mochine oa CDC ka boeona, ka thuso ea hore data e tla phetoa sebakeng sa theknoloji e eketsehileng. Ho database e 'ngoe le e' ngoe ea Progress OpenEdge, o hloka ho kenyelletsa libaka tse arohaneng tsa polokelo moo data ea mohloli e tla kopitsoa, 'me u kenye mochine ka boeona u sebelisa taelo. sehlahisoa.
Mohlala oa mokhoa oa polokelo ea li-bisquit
- E kopitsa lethathamong la libuka C: Pro2db file cdcadd.st ho bukana ea polokelo ea mohloli oa li-bisquit.
- Re hlalosa ka cdcadd.st boholo bo tsitsitseng ba libaka "ReplCDCArea" и "ReplCDCArea_IDX". U ka kenya libaka tse ncha tsa polokelo inthaneteng: prostrct addonline bisquit cdcadd.st
- Kenya OpenEdge CDC:
proutil bisquit -C enablecdc area "ReplCDCArea" indexarea "ReplCDCArea_IDX" - Basebelisi ba latelang ba tlameha ho theoa sebakeng sa polokelo ea litaba ho tseba mekhoa e sebetsang:
a. pro2adm - bakeng sa ho hokela ho tsoa ho phanele ea tsamaiso ea Pro2.
b. pro2etl - bakeng sa ho hokahanya mekhoa ea ETL (ReplBatch).
c. pro2cdc - bakeng sa ho hokahanya mekhoa ea CDC (CDCBatch).
Ho theha Schema Holder bakeng sa DataServer bakeng sa Oracle
E latelang, re hloka ho theha database ea Schema Holder ho seva moo data e tsoang ho Progress DBMS e tla phetoa ho Oracle DBMS. DataServer Schema Holder ke database ea Progress OpenEdge e se nang basebelisi kapa data ea ts'ebeliso, e nang le 'mapa oa ngollano lipakeng tsa litafole tsa mohloli le litafole tsa Oracle tsa kantle.
Setsi sa polokelo ea Schema Holder bakeng sa Progress OpenEdge DataServer bakeng sa Oracle bakeng sa Pro2 se tlameha ho behoa ho seva sa ts'ebetso sa ETL; se entsoe ka thoko bakeng sa lekala ka leng.
Mokhoa oa ho theha Schema Holder
- Hlakola kabo ea Pro2 bukeng /pro2
- Theha 'me u ee ho directory /pro2/dbsh
- Theha database ea Schema Holder u sebelisa taelo kopitsa $DLC/bisquitsh e se nang letho
- Ho etsa tshokoloho bisquits ho khouto e hlokahalang - mohlala, ho UTF-8 haeba database tsa Oracle li na le khouto ea UTF-8: proutil bisquitsh -C convchar fetola UTF-8
- Kamora ho theha database e se nang letho bisquits hokela ho eona ka mokhoa oa motho a le mong: pro bisquits
- Ha re ee ho Data Dictionary: Lisebelisoa -> Bukana ea Boitsebiso -> DataServer -> ORACLE Utilities -> Theha DataServer Schema
- Qala Schema Holder
- Ho theha broker ea Oracle DataServer:
a. Qala AdminServer.
proadsv - qala
b. Ho qala ha Broker ea Oracle DataServer
oraman -name orabroker1 -qala
Ho theha phanele ea tsamaiso le sekema sa ho ikatisa
Ho sebelisoa phanele ea tsamaiso ea Pro2, li-parameter tsa Pro2 li lokisoa, ho kenyelletsa ho theha moralo oa ho pheta-pheta le ho hlahisa mekhoa ea ETL (Processor Library), mananeo a mantlha a ho hokahanya (Bulk-Copy Processor), li-triggers le maano a OpenEdge CDC. Ho boetse ho na le lisebelisoa tsa mantlha tsa ho beha leihlo le ho tsamaisa lits'ebetso tsa ETL le CDC. Pele ho tsohle, re theha lifaele tsa parameter.
Mokhoa oa ho hlophisa lifaele tsa parameter
- E-ea lethathamong la libuka C: Pro2bpreplScripts
- Bula faele bakeng sa ho hlophisoa replProc.pf
- Eketsa liparamente tsa khokahanyo ho database ea replication ea cdc:
# Database ea Replication
-db cdc -ld repl -H <main database hostname> -S <database broker port cdc>
-U pro2admin -P <password> - kenyeletsa ho replProc.pf liparamente tsa khokahano ho database tsa mohloli le Schema Holder ka mokhoa oa lifaele tsa paramente. Lebitso la faele ea paramente le tlameha ho ts'oana le lebitso la database ea mohloli o hokahaneng.
# Hokela mehloling eohle e phetotsoeng BISQUIT
-pf bpreplscriptsbisquit.pf - kenyeletsa ho replProc.pf liparamente tsa ho hokela ho Schema Holder.
#Target Pro DB Schema Holder
-db bisquitsh -ld bisquitsh
-H <ETL sebetsa lebitso la moeti>
-S <biskuitsh broker port>
-db bisquitsql
-ld bisquitsql
-dt ORACLE
-S 5162 -H <Oracle broker hostname>
-DataService orabroker1 - Boloka faele ea parameter replProc.pf
- Ka mor'a moo, o hloka ho theha le ho bula bakeng sa ho hlophisa lifaele tsa paramethara bakeng sa database e 'ngoe le e' ngoe e hokahaneng ea mohloli bukeng C:Pro2bpreplScripts: bisquit.pf. Faele e 'ngoe le e' ngoe ea pf e na le liparamente tsa ho hokahana le database e tsamaellanang, mohlala:
-db bisquit -ld bisquit -H <lebitso la moeti> -S <broker port>
-U pro2admin -P <password>
Ho lokisa li-shortcuts tsa Windows, o hloka ho ea ho directory C: Pro2bpreplScripts 'me u hlophise "Pro2 - Tsamaiso" e khaoletsang. Ho etsa sena, bula thepa ea sekhutlo le moleng Qala ka bonts'a buka ea ho kenya Pro2. Ts'ebetso e ts'oanang e tlameha ho etsoa bakeng sa likhutšoane tsa "Pro2 - Editor" le "RunBulkLoader".
Setupo sa Tsamaiso ea Pro2: Ho Loading Sebopeho sa Pele
Ha re qaleng console.
Eya ho "DB Map".
Ho hokahanya marang-rang ho Pro2 - Tsamaiso, e ea ho tab 'Mapa oa DB. Kenya 'mapa oa mehloli ea mehloli - Schema Holder - Oracle.
Eya ho tab dimmapa. E thathamisitsoe Database ea Mohloli Ka nako e sa lekanyetsoang, sebaka sa pele sa polokelo ea mohloli se hokahaneng se khethoa. Ka ho le letona la lenane ho lokela ho ba le mongolo Li-database tsohle li hokahane — li-database tse khethiloeng li hokahane. Ka tlase ka letsohong le letšehali u lokela ho bona lethathamo la litafole tsa Tsoelo-pele ho tsoa ho bisquit. Ka ho le letona ke lethathamo la litafole tse tsoang polokelong ea Oracle.
Ho theha schemas le database tsa SQL ho Oracle
Ho theha 'mapa oa ho ikatisa, o tlameha ho etsa pele SQL schema ho Oracle. Ho Tsamaiso ea Pro2 re etsa ntho ea menu Lisebelisoa -> Hlahisa Khoutu -> Target Schema, ebe ka lebokoseng la puisano Khetha Database khetha polokelongtshedimosetso ya mohlodi o le mong kapa tse ngata mme o di ise ho le letona.
Tobetsa OK 'me u khethe lenane ho boloka schemas tsa SQL.
Ka mor'a moo re theha motheo. Sena se ka etsoa, ka mohlala, ka Moqapi oa Oracle SQL. Ho etsa sena, re hokahanya le database ea Oracle mme re kenya schema bakeng sa ho eketsa litafole. Kamora ho fetola sebopeho sa litafole tsa Oracle, o hloka ho nchafatsa li-schema tsa SQL ho Schema Holder.
Kamora hore download e phetheloe ka katleho, tsoa sebakeng sa polokelo ea li-bisquitsh 'me u bule phanele ea tsamaiso ea Pro2. Litafole tse tsoang polokelong ea Oracle li lokela ho hlaha tabeng ea 'Mapa e ka ho le letona.
'Mapa oa litafole
Ho theha 'mapa oa ho pheta-pheta, ho phanele ea tsamaiso ea Pro2, e-ea ho tab ea Mapping ebe u khetha mohloli oa boitsebiso. Tobetsa ho Litafole tsa 'Mapa, khetha Khetha Liphetoho ka letsohong le letšehali la litafole tse lokelang ho phetoa ho Oracle, li tsamaise ka ho le letona ebe u netefatsa khetho. 'Mapa o tla iketsetsa litafole tse khethiloeng. Re pheta ts'ebetso ena ho theha 'mapa oa phetisetso bakeng sa database tse ling tsa mehloli.
Ho Hlahisa Laeborari ea Pro2 Replication Processor le Mananeo a Bulk-Copy Processor
Laeborari ea Replication Processor e etselitsoe mekhoa ea ho ikatisa (ETLs) e sebetsanang le mokoloko oa phetisetso ea Pro2 le ho sutumelletsa liphetoho polokelong ea Oracle. Mananeo a laeborari ea Replication processor a bolokoa ka bo eona bukeng ka mor'a moloko bprepl/repl_proc (PROC_DIRECTORY parameter). Ho hlahisa laeborari ea processor ea replication, ea ho Lisebelisoa -> Hlahisa Khoutu -> Laeborari ea processor. Ka mor'a hore moloko o phethoe, mananeo a tla hlaha bukeng bprepl/repl_proc.
Mananeo a Bulk Load Processor a sebedisetswa ho amahanya dipolokelo tsa mohlodi wa Tsoelopele le polokelongtshedimosetso ya Oracle e ipapisitseng le puo ya lenaneo la Progress ABL (4GL). Ho li hlahisa, e ea ho ntho ea menu Lisebelisoa -> Hlahisa Khoutu -> Bulk-Copy Processor. Ka lebokoseng la puisano la Khetha Database, khetha mehloli ea mehloli ea boitsebiso, e tsamaise ka lehlakoreng le letona la fensetere ebe o tobetsa OK. Ka mor'a hore moloko o phethoe, mananeo a tla hlaha bukeng bpreplrepl_mproc.
Ho theha mekhoa ea ho pheta-pheta ho Pro2
Ho arola litafole ka lihlopha tse fanoang ke khoele e arohaneng ea ho pheta-pheta ho ntlafatsa ts'ebetso le katleho ea Pro2 Oracle. Ka linako tsohle, likhokahano tsohle tse entsoeng 'mapeng oa ho ikatisa bakeng sa litafole tse ncha tsa ho ikatisa li amahanngoa le khoele ea nomoro ea 1. Ho khothalletsoa ho arola litafole ka likhoele tse fapaneng.
Lintlha tse mabapi le boemo ba likhoele tsa ho pheta-pheta li hlahisoa skrineng sa Tsamaiso ea Pro2 ho tab ea Monitor karolong ea Replication Status. Tlhaloso e qaqileng ea litekanyetso tsa parameter e ka fumanoa litokomaneng tsa Pro2 (directory C: Pro2Docs).
Theha le ho kenya tšebetsong maano a CDC
Maano ke sehlopha sa melao bakeng sa enjene ea OpenEdge CDC ho beha leihlo liphetoho litafoleng. Ka nako ea ho ngola, Pro2 e ts'ehetsa feela maano a CDC a boemo ba 0, ke hore, ke 'nete feela e behiloeng leihlo rekota liphetoho.
Ho theha leano la CDC, phanele ea tsamaiso, e-ea ho tab ea 'Mapa, khetha pokello ea mohloli ebe o tobetsa konopo ea Eketsa/Tlosa Maano. Fesetereng ea Khetha Liphetoho e butsoeng, khetha ka lehlakoreng le letšehali ebe u ea ka ho le letona litafole tseo u lokelang ho li etsa kapa ho hlakola leano la CDC.
Ho kenya ts'ebetsong, bula tab ea 'Mapa hape, khetha mohloli oa mohloli ebe o tobetsa konopo (Ho) Kenya Maano. Khetha 'me u suthele ka lehlakoreng le letona la tafole maano a lokelang ho etsoa, tobetsa OK. Ka mor'a sena li tšoauoa ka botala. Ka ho sebelisa (Ho) Kenya Maano U ka boela ua tima maano a CDC. Litšebelisano tsohle li etsoa inthaneteng.
Kamora hore leano la CDC le kenngoe tšebetsong, lintlha tse mabapi le lirekoto tse fetotsoeng li bolokoa sebakeng sa polokelo "ReplCDCArea" ho latela mohloli oa boitsebiso. Lintlha tsena li tla sebetsoa ka mokhoa o khethehileng CDCBatch, e ipapisitseng le tsona e tla theha lintlha ka har'a tatellano ea ho ikatisa ea Pro2 polokelong ea litaba cdc (repl).
Kahoo, re na le mela e 'meli ea ho ikatisa. Mokhahlelo oa pele ke CDCBatch: ho tsoa polokelong ea mohloli, lintlha li qala ho ea polokelong ea litaba ea CDC e mahareng. Mohato oa bobeli ke ha data e fetisoa ho tloha polokelong ea CDC ho ea Oracle. Ena ke tšobotsi ea mohaho oa hona joale le sehlahisoa ka boeona - ho fihlela joale bahlahisi ha ba e-s'o khone ho theha ho ikatisa ka ho toba.
Khokahano ea mantlha
Kamora ho nolofalletsa mochini oa CDC le ho theha seva sa phetiso ea Pro2, re hloka ho qala khokahano ea pele. Taelo ea pele ea kamahanyo:
/pro2/bprepl/Script/replLoad.sh bisquit table-name
Kamora hore khokahano ea pele e phetheloe, mekhoa ea ho ikatisa e ka qalisoa.
Ho qala ts'ebetso ea ho ikatisa
Ho qala mekhoa ea ho pheta-pheta u hloka ho tsamaisa script replbatch.sh. Pele o qala, etsa bonnete ba hore ho na le mangolo a replbatch bakeng sa likhoele tsohle - replbatch1, replbatch2, joalo-joalo. Haeba tsohle li le teng, bula mola oa taelo (mohlala, proenv), e-ea ho directory /bprepl/scripts ebe o qala script. Ka phanele ea tsamaiso, re hlahloba hore ts'ebetso e ts'oanang e amohetse boemo ba RUNNING.
Liphetho
Ka mor'a ho kenya ts'ebetsong, re ile ra potlakisa haholo ho kenya tlhahisoleseding sebakeng sa polokelo ea boitsebiso ba khoebo. Lintlha li tla kena ho Oracle inthaneteng. Ha ho na tlhoko ea ho senya nako ho botsa lipotso tse nkileng nako e telele ho bokella data ho tsoa lits'ebetsong tse fapaneng. Ho phaella moo, tharollong ena ts'ebetso ea ho pheta-pheta e ka hatella data, e nang le phello e ntle ka lebelo. Hona joale poelano ea letsatsi le letsatsi ea tsamaiso ea BISKVIT le litsamaiso tse ling e ile ea qala ho nka metsotso e 15-20 ho e-na le lihora tse 2-2,5, 'me poelano e feletseng e ile ea nka lihora tse' maloa ho e-na le matsatsi a mabeli.
Source: www.habr.com