Yadda ake yin abokai tsakanin tsarin banki na ci gaba na OpenEdge da Oracle DBMS

Tun daga 1999, don hidimar ofishi na baya, bankin mu ya yi amfani da tsarin hada-hadar banki na BISKVIT akan dandamalin ci gaba na OpenEdge, wanda ake amfani da shi a ko'ina cikin duniya, gami da fannin kuɗi. Ayyukan wannan DBMS yana ba ku damar karanta bayanai har miliyan ɗaya ko fiye a cikin daƙiƙa ɗaya a cikin rumbun adana bayanai (DB). Ci gaban mu na OpenEdge sabis game da adibas guda miliyan 1,5 da kusan kwangilar miliyan 22,2 don samfuran aiki (rancen mota da jinginar gida), kuma shine ke da alhakin duk matsuguni tare da mai gudanarwa (Bankin Tsakiya) da SWIFT.

Amfani da Ci gaban OpenEdge, mun fuskanci buƙatar yin aiki tare da Oracle DBMS. Da farko, wannan kullin shine ginshiƙin kayan aikin mu - har sai mun shigar kuma muka daidaita Pro2 CDC - samfurin Ci gaba wanda ke ba ku damar aika bayanai daga Ci gaban DBMS zuwa Oracle DBMS kai tsaye, kan layi. A cikin wannan sakon za mu gaya muku daki-daki, tare da duk ramummuka, yadda ake yin abokai da kyau tsakanin OpenEdge da Oracle.

Yadda ya faru: loda bayanai zuwa QCD ta hanyar raba fayil

Na farko, wasu bayanai game da ababen more rayuwa. Adadin masu amfani da rumbun adana bayanai kusan dubu 15 ne. Girman duk bayanan bayanai masu amfani, gami da kwafi da jiran aiki, TB 600 ne, mafi girman ma'ajin bayanai shine 16,5 TB. A lokaci guda, ana ci gaba da cika ma'ajin bayanai: a cikin shekarar da ta gabata kawai, an ƙara kusan TB 120 na bayanai masu amfani. Ana amfani da tsarin ta sabobin gaba 150 akan dandalin x86. Ana gudanar da ma'ajin bayanai akan sabar dandamali 21 na IBM.

Tsarin gaba-gaba, tsarin banki daban-daban da sabis na banki an haɗa su tare da OpenEdge Progress (BISCUIT IBS) ta hanyar bas ɗin Sonic ESB. Loda bayanai zuwa QCD yana faruwa ta hanyar musayar fayil. Har zuwa wani lokaci a cikin lokaci, wannan maganin yana da manyan matsaloli guda biyu a lokaci ɗaya - ƙarancin aiki na loda bayanai a cikin ɗakin ajiyar bayanan kamfanoni (CDW) da kuma dogon lokaci don yin sulhunta bayanai (salantawa) tare da wasu tsarin.

Saboda haka, mun fara neman kayan aiki wanda zai iya hanzarta waɗannan matakai. Maganin matsalolin biyu shine sabon samfur na ci gaba na OpenEdge - Pro2 CDC (Canja Bayanan Bayani). Don haka, bari mu fara.

Shigar da Ci gaban OpenEdge da Pro2Oracle

Don gudanar da Pro2 Oracle akan kwamfutar Windows na mai gudanarwa, ya isa a shigar da Buɗewar Buɗewa Mai Haɓakawa na Ci gaba, wanda zai iya zama скачать kyauta. Tsoffin kundayen shigarwa na OpenEdge:

DLC: C: ProgressOpenEdge
WRK: C: OpenEdgeWRK

Ayyukan ETL suna buƙatar sigar lasisin OpenEdge na ci gaba 11.7+ - wato OE DataServer don Oracle da Tsarin Ci gaban 4GL. An haɗa waɗannan lasisi tare da Pro2. Don cikakken aiki na DataServer don Oracle tare da bayanan Oracle mai nisa, an shigar da Cikakken Abokin Oracle.

Akan uwar garken Oracle kuna buƙatar shigar da Oracle Database 12+, ƙirƙirar bayanan da ba komai a ciki kuma ƙara mai amfani (bari mu kira shi). cdc).

Don shigar da Pro2Oracle, zazzage sabuwar rarrabawa daga cibiyar zazzagewa Software na Ci gaba. Cire kayan tarihin cikin kundin adireshi C: Pro2 (Don saita Pro2 akan Unix, ana amfani da rarraba iri ɗaya kuma ana amfani da ƙa'idodin daidaitawa iri ɗaya).

Ƙirƙirar bayanan kwafin CDc

Maimaita bayanai cdc (repl) Ana amfani da Pro2 don adana bayanan sanyi, gami da taswirar kwafi, sunayen bayanan da aka kwafi da teburinsu. Hakanan yana ƙunshe da jerin gwano, wanda ya ƙunshi bayanin kula game da gaskiyar cewa layin tebur a cikin bayanan tushen ya canza. Ana amfani da bayanai daga layin kwafi ta hanyoyin ETL don gano layuka waɗanda ke buƙatar kwafi zuwa Oracle daga tushen bayanai.

Muna ƙirƙirar bayanan cdc daban.

Hanyar ƙirƙirar bayanai

1. A kan uwar garken bayanai mun ƙirƙiri adireshi don bayanan cdc - alal misali, akan uwar garken /database/cdc/.
2. Ƙirƙiri dummy don bayanan cdc: kwafin $DLC/cdc mara komai
3. Kunna tallafi don manyan fayiloli: proutil cdc -C EnableLargeFiles
4. Muna shirya rubutun don fara bayanan cdc. Ma'aunin farawa dole ne ya yi kama da sigogin farawa na bayanan da aka kwafi.
5. Mun fara cdc database.
6. Haɗa zuwa bayanan cdc kuma loda tsarin Pro2 daga fayil ɗin cdc.df, wanda aka haɗa tare da Pro2.
7. Muna ƙirƙirar masu amfani masu zuwa a cikin bayanan cdc:

pro2adm - don haɗawa daga kwamitin gudanarwa na Pro2;
pro2etl - don haɗa hanyoyin ETL (ReplBatch);
pro2cdc - don haɗa hanyoyin CDC (CDCBatch);

Kunna BuɗeEdge Canjin Ɗaukar Bayanai

Yanzu bari mu kunna na'urar CDC kanta, tare da taimakon abin da za a iya maimaita bayanai zuwa ƙarin yankin fasaha. Ga kowane Ci gaba na tushen tushen bayanai na OpenEdge, kuna buƙatar ƙara wuraren ajiya daban waɗanda za a kwafi bayanan tushen a ciki, sannan kunna injin kanta ta amfani da umarnin. proutil.

Misalin hanya don bayanan bisquit

1. Ana kwafi daga kasidar C: Pro2db fayil cdd.st zuwa bisquit source database directory.
2. Mun bayyana a cdd.st ƙayyadaddun ƙayyadaddun girman girman yankuna "ReplCDCArewa" и "ReplCDCArewa_IDX". Kuna iya ƙara sabbin wuraren ajiya akan layi: prostrct addonline bisquit cdcadd.st
3. Kunna OpenEdge CDC:
   proutil bisquit -C damarcdc yankin "ReplCDCArea" indexarea "ReplCDCArea_IDX"
4. Dole ne a ƙirƙiri masu amfani masu zuwa a cikin bayanan tushen don gano hanyoyin tafiyarwa:
   a. pro2adm - don haɗawa daga kwamitin gudanarwa na Pro2.
   b. pro2etl - don haɗa hanyoyin ETL (ReplBatch).
   c. pro2cdc - don haɗa hanyoyin CDC (CDCBatch).

Ƙirƙirar Riƙon Tsari don DataServer don Oracle

Na gaba, muna buƙatar ƙirƙiri bayanan Riƙen Tsari akan sabar inda bayanai daga Ci gaban DBMS za a kwafi su zuwa Oracle DBMS. DataServer Schema Holder wani fanko ne na BuɗeEdge na Ci gaba ba tare da masu amfani ko bayanan aikace-aikace ba, yana ɗauke da taswirar rubutu tsakanin teburan tushe da teburan Oracle na waje.

Ma'ajin Rikicin Schema don Ci gaban OpenEdge DataServer don Oracle don Pro2 dole ne ya kasance akan sabar tsari na ETL; an ƙirƙira shi daban don kowane reshe.

Yadda ake ƙirƙira mai riƙe da tsari

1. Cire fakitin rarrabawar Pro2 zuwa cikin kundin adireshi /pro2
2. Ƙirƙiri kuma je zuwa kundin adireshi /pro2/dbsh
3. Ƙirƙiri bayanan Riƙen Tsari ta amfani da umarnin kwafi $DLC/Bisquitsh mara kyau
4. Yin jujjuyawar biskit cikin rufaffen da ake buƙata - alal misali, a cikin UTF-8 idan bayanan bayanan Oracle suna da rufaffen UTF-8: proutil bisquitsh -C convchar canza UTF-8
5. Bayan ƙirƙirar rumbun adana bayanai mara komai biskit haɗi zuwa gare shi a yanayin mai amfani guda ɗaya: pro biskit
6. Mu je zuwa Data Dictionary: Kayan aiki -> Kamus na Bayanai -> DataServer -> Abubuwan Amfani ORACLE -> Ƙirƙiri Tsarin DataServer
7. Kaddamar da Schema Holder
8. Kafa dillalin Oracle DataServer:
   a. Fara AdminServer.
   proadsv - farawa
   b. Fara dillalin Oracle DataServer
   oraman -suna orabroker1 -fara

Kafa tsarin gudanarwa da tsarin kwafi

Yin amfani da kwamiti na gudanarwa na Pro2, ana daidaita sigogin Pro2, gami da kafa tsarin kwafi da samar da hanyoyin ETL (Labarun Mai sarrafawa), shirye-shiryen daidaitawa na farko (Mai sarrafa Bulk-Copy Processor), abubuwan da ke haifar da kwafi da manufofin OpenEdge CDC. Hakanan akwai kayan aikin farko don saka idanu da sarrafa hanyoyin ETL da CDC. Da farko, mun kafa fayilolin siga.

Yadda ake saita fayilolin siga

1. Jeka kasida C: Pro2bpreplScripts
2. Bude fayil ɗin don gyarawa replProc.pf
3. Ƙara sigogin haɗin kai zuwa bayanan kwafin cdc:
   # Maimaita Database
   -db cdc -ld repl -H <main database hostname> -S <cdc dillali tashar jiragen ruwa>
   -U pro2admin -P <password>
4. kara zuwa replProc.pf sigogin haɗin kai zuwa tushen bayanan bayanai da Mai riƙe da tsari a cikin nau'in fayilolin sigina. Dole ne sunan fayil ɗin sigogi ya dace da sunan tushen bayanan da ake haɗawa.
   # Haɗa zuwa duk hanyoyin da aka kwafi BISQUIT
   -pf bpreplscriptsbisquit.pf
5. kara zuwa replProc.pf sigogi don haɗawa zuwa Mai riƙe da Schema.
   #Target Pro DB Schema Riƙe
   -db bisquitsh -ld biskit
   -H <ETL tsari sunan mai masauki>
   -S <biskuitsh dillali tashar jiragen ruwa>
   -db bisquitsql
   -ld bisquitsql
   -dt ORACLE
   -S 5162 -H <sunan mai masaukin Oracle>
   -DataService orbroker1
6. Ajiye fayil ɗin sigogi replProc.pf
7. Na gaba, kuna buƙatar ƙirƙira da buɗewa don gyara fayilolin siga don kowane tushen bayanan da aka haɗa a cikin kundin adireshi C:Pro2bpreplScripts: bisquit.pf. Kowane fayil na pf yana ƙunshe da sigogi don haɗawa da bayanan da suka dace, misali:
   -db bisquit -ld bisquit -H <hostname> -S < tashar jiragen ruwa>
   -U pro2admin -P <password>

Don saita gajerun hanyoyin Windows, kuna buƙatar zuwa ga directory C: Pro2bpreplScripts kuma shirya gajeriyar hanyar "Pro2 - Administration". Don yin wannan, buɗe kaddarorin gajeriyar hanya kuma a cikin layi Fara a ciki nuna jagorar shigarwa na Pro2. Dole ne a yi irin wannan aiki don gajerun hanyoyin "Pro2 - Edita" da "RunBulkLoader".

Saitin Gudanarwar Pro2: Loading Kanfigareshan Farko

Bari mu kaddamar da na'ura wasan bidiyo.

Je zuwa "DB Map".

Don haɗa bayanan bayanai a cikin Pro2 – Gudanarwa, je zuwa shafin DB Map. Ƙara taswirar bayanan bayanan tushe - Mai riƙe da tsari - Oracle.

Jeka tab zana taswira. An jera Database Source Ta hanyar tsoho, an zaɓi tushen bayanan farko da aka haɗa. A hannun dama na lissafin yakamata a sami rubutu An Haɗe Duk Databases - an haɗa bayanan da aka zaɓa. A ƙasa a gefen hagu ya kamata ku ga jerin Tables na Ci gaba daga bisquit. A hannun dama akwai jerin teburi daga bayanan Oracle.

Ƙirƙirar tsarin SQL da bayanan bayanai a cikin Oracle

Don ƙirƙirar taswirar kwafi, dole ne ka fara ƙirƙira SQL tsarin in Oracle. A cikin Gudanarwar Pro2 muna aiwatar da abin menu Kayan aiki -> Ƙirƙirar lamba -> Tsarin Target, sannan a cikin akwatin maganganu Zaɓi Wurin ajiye bayanai zaɓi ɗaya ko fiye tushen bayanan bayanai kuma matsar da su zuwa dama.

Danna Ok kuma zaɓi directory don adana tsarin SQL.

Na gaba muna ƙirƙirar tushe. Ana iya yin wannan, alal misali, ta hanyar Oracle SQL Mai haɓakawa. Don yin wannan, muna haɗa zuwa bayanan Oracle kuma muna loda tsarin don ƙara tebur. Bayan canza abun da ke cikin tebur na Oracle, kuna buƙatar sabunta tsare-tsaren SQL a cikin Riƙen Tsari.

Bayan an gama zazzagewar cikin nasara, fita daga bayanan bisquitsh kuma buɗe kwamitin gudanarwa na Pro2. Tebura daga bayanan Oracle yakamata su bayyana akan shafin Taswira a dama.

Taswirar tebur

Don ƙirƙirar taswirar kwafi, a cikin kwamitin gudanarwa na Pro2, je zuwa shafin Taswira kuma zaɓi tushen bayanai. Danna kan Taswirar Taswira, zaɓi Zaɓi Canje-canje a gefen hagu na tebur waɗanda yakamata a yi su a cikin Oracle, matsa su zuwa dama kuma tabbatar da zaɓin. Za a ƙirƙiri taswira ta atomatik don tebur da aka zaɓa. Muna maimaita aikin don ƙirƙirar taswirar kwafi don sauran bayanan bayanai.

Samar da Laburaren Mai-mai-mai-yi na Pro2 da Shirye-shiryen Mai Sarrafa Kwafi

An ƙirƙira Laburaren Mai Sarrafawa Mai Sauƙi don hanyoyin kwafi na al'ada (ETLs) waɗanda ke aiwatar da layin kwafi na Pro2 da tura canje-canje zuwa bayanan Oracle. Ana adana shirye-shiryen ɗakin karatu na mai sarrafawa ta atomatik zuwa kundin adireshi bayan tsara bprepl/repl_proc (PROC_DIRECTORY siga). Don samar da ɗakin karatu na mai sarrafawa, je zuwa Kayan aiki -> Ƙirƙirar Code -> Laburaren Mai sarrafawa. Bayan tsara tsara, shirye-shiryen za su bayyana a cikin directory bprepl/repl_proc.

Ana amfani da shirye-shiryen Mai sarrafa Load ɗin girma don daidaita bayanan Ci gaban tushen tushen tare da maƙasudin bayanan Oracle dangane da yaren shirye-shirye na Ci gaba ABL (4GL). Don ƙirƙirar su, je zuwa abin menu Kayan aiki -> Ƙirƙirar lamba -> Mai sarrafa-kwafi. A cikin akwatin maganganu Zaɓi Database, zaɓi tushen bayanan bayanai, matsar da su zuwa gefen dama na taga kuma danna OK. Bayan tsara tsara, shirye-shiryen za su bayyana a cikin directory bpreplrepl_mproc.

Kafa hanyoyin kwafi a cikin Pro2

Rarraba teburi zuwa saitin da aka yi amfani da shi ta hanyar zaren kwafi daban yana inganta aiki da inganci na Pro2 Oracle. Ta hanyar tsoho, duk haɗin haɗin da aka ƙirƙira a cikin taswirar kwafi don sabbin allunan kwafi suna da alaƙa da lambar zaren 1. Ana ba da shawarar raba tebur zuwa zaren daban-daban.

Ana nuna bayani game da matsayin zaren kwafi akan allon Gudanarwa na Pro2 a cikin shafin Kulawa a sashin Matsayin Maimaitawa. Ana iya samun cikakken bayanin ma'auni a cikin takaddun Pro2 (directory C: Pro2Docs).

Ƙirƙiri kuma kunna manufofin CDC

Manufofin saitin dokoki ne don injin OpenEdge CDC don saka idanu canje-canje ga tebur. A lokacin rubuce-rubuce, Pro2 kawai yana goyan bayan manufofin CDC tare da matakin 0, wato, kawai ana lura da gaskiya rikodin canje-canje.

Don ƙirƙirar manufofin CDC, akan kwamitin gudanarwa, je zuwa shafin Taswira, zaɓi tushen bayanan bayanai kuma danna maɓallin Ƙara/Cire Manufofin. A cikin Zaɓi Canje-canje taga wanda ya buɗe, zaɓi a gefen hagu kuma matsa zuwa dama teburin waɗanda kuke buƙatar ƙirƙira ko share manufar CDC.

Don kunnawa, buɗe shafin Taswira kuma, zaɓi tushen bayanan bayanai kuma danna maɓallin (A) Kunna Manufofin. Zaɓi ka matsa zuwa gefen dama na teburin manufofin da ake buƙatar kunnawa, danna Ok. Bayan wannan an yi musu alama a kore. Ta amfani (A) Kunna Manufofin Hakanan zaka iya kashe manufofin CDC. Ana yin duk ma'amaloli akan layi.

Bayan an kunna manufar CDC, bayanin kula game da gyare-gyaren bayanan ana adana su zuwa wurin ajiya "ReplCDCArewa" bisa ga tushen bayanai. Za a sarrafa waɗannan bayanan ta hanyar tsari na musamman CDCBatch, wanda ya dogara da su zai ƙirƙiri bayanin kula a cikin layin kwafi na Pro2 a cikin bayanan cdc (repl).

Don haka, muna da layuka biyu don maimaitawa. Mataki na farko shine CDCBatch: daga tushen bayanai, bayanai na farko suna zuwa tsakiyar CDC database. Mataki na biyu shine lokacin da aka canja wurin bayanai daga bayanan CDC zuwa Oracle. Wannan siffa ce ta gine-gine na yanzu da kuma samfurin kanta - ya zuwa yanzu masu haɓakawa ba su iya kafa kwafi kai tsaye ba.

Aiki tare na farko

Bayan kunna tsarin CDC da kafa uwar garken kwafi na Pro2, muna buƙatar fara aiki tare na farko. Umarnin aiki tare na farko:

/pro2/bprepl/Script/replLoad.sh bisquit table-name

Bayan an gama aiki tare na farko, ana iya fara aiwatar da kwafi.

Fara aiwatar da kwafi

Don fara aiwatar da maimaitawa kuna buƙatar gudanar da rubutun sakewa.sh. Kafin farawa, tabbatar cewa akwai rubutun maimaitawa don duk zaren - replbatch1, replbatch2, da sauransu. Idan komai yana wurin, buɗe layin umarni (misali, proenv), Je zuwa kundin adireshi /bprepl/scripts kuma fara rubutun. A cikin kwamitin gudanarwa, muna duba cewa tsarin da ya dace ya sami matsayin RUNNING.

Результаты

Bayan aiwatarwa, mun ƙara hanzarta loda bayanai zuwa ma'ajiyar bayanan kamfanoni. Bayanan suna shiga Oracle ta atomatik akan layi. Babu buƙatar ɓata lokaci don gudanar da wasu tambayoyi masu tsawo don tattara bayanai daga tsarin daban-daban. Bugu da ƙari, a cikin wannan bayani tsarin maimaitawa zai iya damfara bayanai, wanda kuma yana da tasiri mai kyau akan gudun. Yanzu sulhun yau da kullun na tsarin BISKVIT tare da sauran tsarin ya fara ɗaukar mintuna 15-20 maimakon sa'o'i 2-2,5, kuma cikakken sulhu ya ɗauki sa'o'i da yawa maimakon kwana biyu.

source: www.habr.com