Yuav ua li cas ua phooj ywg ntawm Progress OpenEdge banking system thiab Oracle DBMS

Txij li thaum xyoo 1999, los pab rau lub chaw ua haujlwm tom qab, peb lub txhab nyiaj tau siv cov kev lag luam sib koom ua ke BISKVIT ntawm Progress OpenEdge platform, uas tau siv dav thoob ntiaj teb, suav nrog hauv kev lag luam nyiaj txiag. Qhov kev ua tau zoo ntawm DBMS no tso cai rau koj nyeem txog li ib lab lossis ntau cov ntaub ntawv ib ob hauv ib lub database (DB). Peb Cov Kev Pabcuam OpenEdge txog 1,5 lab tus neeg tso nyiaj thiab kwv yees li 22,2 lab daim ntawv cog lus rau cov khoom lag luam (tsheb qiv thiab qiv nyiaj qiv), thiab tseem yog lub luag haujlwm rau txhua qhov kev cog lus nrog tus tswj hwm (Central Bank) thiab SWIFT.

Siv Progress OpenEdge, peb tau ntsib nrog qhov xav tau ua kom nws ua haujlwm nrog Oracle DBMS. Thaum xub thawj, cov pob no yog lub hauv paus ntawm peb cov txheej txheem - txog thaum peb tau teeb tsa thiab teeb tsa Pro2 CDC - cov khoom lag luam Progress uas tso cai rau koj xa cov ntaub ntawv los ntawm Progress DBMS mus rau Oracle DBMS ncaj qha, online. Hauv cov ntawv tshaj tawm no peb yuav qhia rau koj paub meej, nrog rau txhua qhov pitfalls, yuav ua li cas ua phooj ywg zoo ntawm OpenEdge thiab Oracle.

Yuav ua li cas nws tshwm sim: uploading cov ntaub ntawv rau QCD ntawm cov ntaub ntawv sib qhia

Ua ntej, qee qhov tseeb txog peb cov infrastructure. Tus naj npawb ntawm cov neeg siv nquag ntawm cov ntaub ntawv yog kwv yees li 15 txhiab. Qhov ntim ntawm tag nrho cov ntaub ntawv tsim tau, suav nrog cov khoom siv rov ua dua thiab cov chaw tos txais, yog 600 TB, cov ntaub ntawv loj tshaj plaws yog 16,5 TB. Nyob rau tib lub sijhawm, cov ntaub ntawv khaws cia tas li tau rov ua dua: dhau xyoo tas los ib leeg, txog 120 TB ntawm cov ntaub ntawv tsim tau tau ntxiv. Lub kaw lus yog powered los ntawm 150 pem hauv ntej servers ntawm x86 platform. Cov databases tau tuav ntawm 21 IBM platform servers.

Pem hauv ntej-kawg systems, ntau yam tseem ceeb hauv tuam txhab nyiaj thiab cov kev pabcuam hauv tuam txhab nyiaj tau koom ua ke nrog OpenEdge Progress (BISCUIT IBS) ntawm lub tsheb npav Sonic ESB. Uploading cov ntaub ntawv rau QCD tshwm sim los ntawm kev sib pauv ntaub ntawv. Txog rau ib lub sijhawm, qhov kev daws teeb meem no muaj ob qhov teeb meem loj ib zaug - kev ua haujlwm qis ntawm kev xa cov ntaub ntawv mus rau hauv lub tuam txhab cov ntaub ntawv khaws cia (CDW) thiab lub sijhawm ntev rau kev ua cov ntaub ntawv sib haum xeeb (kev sib haum xeeb) nrog rau lwm lub tshuab.

Yog li ntawd, peb pib nrhiav ib lub cuab tam uas yuav ua kom cov txheej txheem no. Txoj kev daws teeb meem rau ob qho tib si yog qhov tshiab Progress OpenEdge cov khoom - Pro2 CDC (Hloov cov ntaub ntawv ntes). Yog li, cia peb pib.

Nruab Progress OpenEdge thiab Pro2Oracle

Txhawm rau khiav Pro2 Oracle ntawm tus thawj coj lub khoos phis tawj Windows, nws txaus los nruab Progress OpenEdge Developer Kit Classroom Edition, uas tuaj yeem ua tau. download dawb. Default OpenEdge installation directories:

DLC: C: ProgressOpenEdge
WRK: C: OpenEdgeWRK

Cov txheej txheem ETL yuav tsum tau Progress OpenEdge cov ntawv tso cai version 11.7+ - uas yog OE DataServer rau Oracle thiab 4GL Development System. Cov ntawv tso cai no suav nrog Pro2. Rau kev ua haujlwm tag nrho ntawm DataServer rau Oracle nrog cov chaw taws teeb Oracle database, Full Oracle Client raug teeb tsa.

Ntawm Oracle neeg rau zaub mov koj yuav tsum nruab Oracle Database 12+, tsim cov ntaub ntawv khoob thiab ntxiv tus neeg siv (cia hu nws cdc).

Txhawm rau nruab Pro2Oracle, rub tawm qhov kev faib tawm tshiab los ntawm qhov chaw rub tawm Kev Tsim Kho Software. Unpack lub archive rau hauv ib phau ntawv C: Pro 2 (Txhawm rau teeb tsa Pro2 ntawm Unix, tib qhov kev faib tawm yog siv thiab siv cov qauv kev teeb tsa tib yam).

Tsim cdc replication database

Replication database cdc (rov) Pro2 yog siv los khaws cov ntaub ntawv teeb tsa, suav nrog daim ntawv qhia kev rov ua dua, cov npe ntawm cov ntaub ntawv rov qab thiab lawv cov ntxhuav. Nws kuj muaj cov kab rov ua dua tshiab, suav nrog cov ntawv sau txog qhov tseeb tias kab lus hauv cov ntaub ntawv tau hloov pauv. Cov ntaub ntawv los ntawm cov kab rov ua dua yog siv los ntawm ETL cov txheej txheem los txheeb xyuas cov kab uas yuav tsum tau theej rau Oracle los ntawm qhov chaw khaws ntaub ntawv.

Peb tab tom tsim cov ntaub ntawv cdc cais.

Txheej txheem los tsim ib lub database

1. Ntawm tus neeg rau zaub mov database peb tsim cov npe rau cdc database - piv txwv li, ntawm lub server /database/cdc/.
2. Tsim ib tug dummy rau cdc database: luam $DLC/empty cdc
3. Pab txhawb rau cov ntaub ntawv loj: proutil cdc -C EnableLargeFiles
4. Peb npaj ib tsab ntawv rau kev pib cdc database. Qhov pib tsis yuav tsum zoo ib yam li qhov pib tsis ntawm cov ntaub ntawv rov ua dua.
5. Peb pib lub cdc database.
6. Txuas mus rau cdc database thiab thauj khoom Pro2 schema los ntawm cov ntaub ntawv cdc.df ua, uas suav nrog Pro2.
7. Peb tsim cov neeg siv hauv qab no hauv cdc database:

pro2adm - rau kev sib txuas ntawm Pro2 tswj vaj huam sib luag;
pro2etl - rau kev sib txuas cov txheej txheem ETL (ReplBatch);
pro2cdc - rau txuas CDC cov txheej txheem (CDCBatch);

Qhib OpenEdge Hloov cov ntaub ntawv ntes

Tam sim no cia peb tig rau CDC mechanism nws tus kheej, nrog kev pab los ntawm cov ntaub ntawv twg yuav rov ua dua rau thaj chaw siv tshuab ntxiv. Rau txhua qhov Progress OpenEdge qhov chaw khaws ntaub ntawv, koj yuav tsum tau ntxiv cov chaw khaws cia cais rau hauv qhov chaw cov ntaub ntawv yuav raug muab luam tawm, thiab qhib lub tshuab nws tus kheej siv cov lus txib. proutil.

Piv txwv txheej txheem rau lub bisquit database

1. Luam tawm los ntawm catalog C: Pro2db cov ntaub ntawv cdcad ua mus rau bisquit qhov chaw database directory.
2. Peb piav qhia hauv cdcad ua taag qhov loj me rau cheeb tsam "ReplCDCArea" и "ReplCDCArea_IDX". Koj tuaj yeem ntxiv qhov chaw cia tshiab hauv online: prostrct addonline bisquit cdcadd.st
3. Qhib OpenEdge CDC:
   proutil bisquit -C enablecdc cheeb tsam "ReplCDCArea" indexarea "ReplCDCArea_IDX"
4. Cov neeg siv hauv qab no yuav tsum tau tsim nyob rau hauv qhov chaw database los txheeb xyuas cov txheej txheem khiav:
   a. pro2adm - rau kev sib txuas ntawm Pro2 tswj vaj huam sib luag.
   b. pro2etl - rau txuas ETL cov txheej txheem (ReplBatch).
   c. pro2cdc - rau txuas CDC cov txheej txheem (CDCBatch).

Tsim Schema Holder rau DataServer rau Oracle

Tom ntej no, peb yuav tsum tsim Schema Holder database ntawm tus neeg rau zaub mov qhov twg cov ntaub ntawv los ntawm Progress DBMS yuav rov ua dua rau Oracle DBMS. DataServer Schema Holder yog qhov khoob Progress OpenEdge cov ntaub ntawv tsis muaj cov neeg siv lossis cov ntaub ntawv thov, uas muaj daim ntawv qhia kev sib tham ntawm cov ntxhuav thiab cov rooj sab nraud Oracle.

Lub Schema Holder database rau Progress OpenEdge DataServer rau Oracle rau Pro2 yuav tsum nyob rau ntawm ETL txheej txheem server; nws yog tsim cais rau txhua ceg.

Yuav ua li cas los tsim Schema Holder

1. Unpack Pro2 faib rau hauv ib phau ntawv /pro2
2. Tsim thiab mus rau daim ntawv teev npe /pro2/dbsh
3. Tsim Schema Holder database siv cov lus txib luam $DLC/pab bisquitsh
4. Ua kev hloov dua siab tshiab bisquitsh rau hauv qhov xav tau encoding - piv txwv li, hauv UTF-8 yog Oracle databases muaj UTF-8 encoding: Hloov ua lwm yam UTF-8 rau proutil bisquitsh
5. Tom qab tsim cov ntaub ntawv khoob bisquitsh txuas rau nws nyob rau hauv ib leeg-neeg siv hom: rau bisquitsh
6. Cia peb mus rau Data Dictionary: Cov cuab yeej -> Cov ntaub ntawv phau ntawv txhais lus -> DataServer -> ORACLE Utilities -> Tsim DataServer Schema
7. Launch Schema Holder
8. Teeb tsa Oracle DataServer broker:
   a. Pib AdminServer.
   proadsv - pib
   b. Pib ntawm Oracle DataServer broker
   oraman -name orabroker1 -start

Kev teeb tsa lub vaj huam sib luag tswj hwm thiab cov txheej txheem rov ua dua

Siv lub Pro2 tswj vaj huam sib luag, Pro2 tsis tau teeb tsa, suav nrog teeb tsa cov txheej txheem rov ua dua thiab tsim cov txheej txheem ETL (Processor Library), thawj cov kev pabcuam synchronization (Cov Txheej Txheem Ntau Lawm), rov ua dua tshiab thiab OpenEdge CDC cov cai. Kuj tseem muaj cov cuab yeej tseem ceeb rau kev saib xyuas thiab tswj cov txheej txheem ETL thiab CDC. Ua ntej tshaj plaws, peb teeb tsa cov ntaub ntawv parameter.

Yuav ua li cas configure parameter cov ntaub ntawv

1. Mus rau lub catalog C: Pro2bpreplScripts
2. Qhib cov ntaub ntawv rau kev kho replProc.pf
3. Ntxiv cov kev sib txuas tsis sib xws rau cdc replication database:
   # Replication Database
   -db cdc -ld repl -H <lub ntsiab database hostname> -S <database broker chaw nres nkoj cdc>
   -U pro2admin -P <password>
4. Ntxiv rau replProc.pf kev sib txuas tsis mus rau qhov chaw databases thiab Schema Holder nyob rau hauv daim ntawv ntawm cov ntaub ntawv parameter. Lub npe ntawm cov ntaub ntawv tsis sib xws yuav tsum phim lub npe ntawm qhov chaw database uas txuas nrog.
   # Txuas mus rau txhua qhov chaw rov ua dua BISQUIT
   -pf bpreplscriptsbisquit.pf
5. Ntxiv rau replProc.pf parameters rau txuas rau Schema Holder.
   #Target Pro DB Schema Holder
   -db bisquitsh -ld bisquitsh
   -H <ETL txheej txheem host lub npe>
   -S <biskuish broker chaw nres nkoj>
   -db bisquitsql
   -ld bisquitsql
   -dt ORACLE
   -S 5162 -H <Oracle broker hostname>
   -DataService orabroker1
6. Txuag cov ntaub ntawv parameter replProc.pf
7. Tom ntej no, koj yuav tsum tsim thiab qhib rau kev kho cov ntaub ntawv parameter rau txhua qhov chaw sib txuas hauv cov ntawv teev npe C:Pro2bpreplScripts: bisquit.pf. Txhua pf cov ntaub ntawv muaj qhov tsis zoo rau kev sib txuas rau cov ntaub ntawv sib xws, piv txwv li:
   -db bisquit -ld bisquit -H <hostname> -S <broker port>
   -U pro2admin -P <password>

Txhawm rau teeb tsa Windows shortcuts, koj yuav tsum mus rau cov npe C: Pro2bpreplScripts thiab kho qhov "Pro2 - Administration" shortcut. Txhawm rau ua qhov no, qhib cov khoom ntawm qhov shortcut thiab hauv kab Pib hauv qhia rau Pro2 installation directory. Ib qho haujlwm zoo sib xws yuav tsum tau ua rau "Pro2 - Editor" thiab "RunBulkLoader" shortcuts.

Pro2 Kev Tswj Xyuas Kev teeb tsa: Loading Initial Configuration

Cia peb pib lub console.

Mus rau "DB Map".

Txhawm rau txuas cov databases hauv Pro2 - Kev tswj hwm, mus rau tab DB Map. Ntxiv daim ntawv qhia txog qhov chaw databases - Schema Holder - Oracle.

Mus rau tab kuas. Sau npe Tau qhov twg los Database Los ntawm lub neej ntawd, thawj cov ntaub ntawv txuas nrog qhov chaw raug xaiv. Nyob rau sab xis ntawm daim ntawv yuav tsum muaj ib tug inscription Txhua Databases Txuas - cov databases xaiv tau txuas nrog. Hauv qab ntawm sab laug koj yuav tsum pom ib daim ntawv teev cov lus Progress los ntawm bisquit. Ntawm sab xis yog ib daim ntawv teev cov rooj los ntawm Oracle database.

Tsim SQL schemas thiab databases hauv Oracle

Txhawm rau tsim daim ntawv qhia replication, koj yuav tsum xub tsim SQL schema hauv Oracle. Hauv Pro2 Administration peb ua cov ntawv qhia zaub mov Cov cuab yeej -> Tsim Code -> Target Schema, ces nyob rau hauv lub dialog box Xaiv Cov Ntaub Ntawv xaiv ib lossis ntau qhov chaw databases thiab txav mus rau sab xis.

Nyem OK thiab xaiv cov ntawv qhia kom txuag tau SQL schemas.

Tom ntej no peb tsim lub hauv paus. Qhov no tuaj yeem ua tau, piv txwv li, ntawm Oracle SQL Tsim Tawm. Txhawm rau ua qhov no, peb txuas rau Oracle database thiab thauj cov schema ntxiv cov ntxhuav. Tom qab hloov qhov muaj pes tsawg leeg ntawm Oracle cov ntxhuav, koj yuav tsum hloov kho SQL schemas hauv Schema Holder.

Tom qab kev rub tawm tiav tiav, tawm hauv bisquitsh database thiab qhib Pro2 tswj vaj huam sib luag. Cov lus los ntawm Oracle database yuav tsum tshwm sim ntawm Mapping tab ntawm sab xis.

Daim duab qhia

Txhawm rau tsim daim ntawv qhia kev rov ua dua, hauv Pro2 tswj vaj huam sib luag, mus rau Mapping tab thiab xaiv qhov chaw khaws ntaub ntawv. Nyem rau ntawm Daim Ntawv Qhia Cov Ntawv, xaiv Xaiv Cov Kev Hloov ntawm sab laug ntawm cov ntxhuav uas yuav tsum tau rov ua dua hauv Oracle, txav mus rau sab xis thiab paub meej tias xaiv. Daim duab qhia chaw yuav raug tsim rau cov rooj xaiv. Peb rov ua haujlwm los tsim ib daim ntawv qhia replication rau lwm qhov chaw databases.

Generating Pro2 Replication Processor Library thiab Bulk-Copy Processor Programs

Replication Processor Library yog tsim los rau kev cai replication txheej txheem (ETLs) uas ua cov Pro2 replication queue thiab thawb hloov mus rau Oracle database. Replication processor tsev qiv ntawv cov kev pab cuam tau txais kev cawmdim rau hauv phau ntawv tom qab tiam bprepl/repl_proc (PROC_DIRECTORY parameter). Txhawm rau tsim cov tsev qiv ntawv replication processor, mus rau Cov cuab yeej -> Tsim Code -> Processor Library. Tom qab tiam ua tiav, cov kev pab cuam yuav tshwm sim hauv cov ntawv teev npe bprepl/repl_proc.

Bulk Load Processor cov kev pab cuam yog siv los synchronize qhov Progress databases nrog lub hom phiaj Oracle database raws li Progress ABL (4GL) programming lus. Txhawm rau tsim lawv, mus rau cov ntawv qhia zaub mov Cov cuab yeej -> Tsim Code -> Bulk-Copy Processor. Nyob rau hauv Xaiv Database dialog box, xaiv qhov chaw databases, txav lawv mus rau sab xis ntawm lub qhov rais thiab nias OK. Tom qab tiam ua tiav, cov kev pab cuam yuav tshwm sim hauv cov ntawv teev npe bpreplrepl_mproc.

Teeb tsa cov txheej txheem rov ua dua hauv Pro2

Kev faib cov ntxhuav rau hauv cov khoom siv los ntawm cov xov sib cais sib cais ua kom cov kev ua tau zoo thiab kev ua haujlwm ntawm Pro2 Oracle. Los ntawm lub neej ntawd, tag nrho cov kev sib txuas tsim nyob rau hauv daim ntawv qhia replication rau tshiab replication ntxhuav yog txuam nrog xov tooj 1. Nws raug nquahu kom cais cov ntxhuav rau hauv cov xov sib txawv.

Cov ntaub ntawv hais txog cov xwm txheej ntawm cov xov sib txuas yog tso tawm rau ntawm Pro2 Administration screen hauv Monitor tab hauv ntu Replication Status. Cov lus piav qhia ntxaws txog qhov muaj nuj nqis tuaj yeem pom hauv Pro2 cov ntaub ntawv (cov ntawv teev npe C: Pro2Docs).

Tsim thiab qhib CDC cov cai

Txoj cai yog cov txheej txheem rau OpenEdge CDC lub cav los saib xyuas cov kev hloov ntawm cov ntxhuav. Thaum lub sijhawm sau ntawv, Pro2 tsuas yog txhawb nqa CDC cov cai nrog qib 0, uas yog, tsuas yog saib xyuas qhov tseeb cov ntaub ntawv hloov pauv.

Txhawm rau tsim CDC txoj cai, ntawm pawg thawj coj saib xyuas, mus rau ntawm Mapping tab, xaiv qhov chaw khaws ntaub ntawv thiab nyem rau ntawm Ntxiv / Tshem Tawm Txoj Cai khawm. Hauv Xaiv Qhov Hloov Hloov Qhov rai uas qhib, xaiv ntawm sab laug thiab txav mus rau sab xis cov ntxhuav uas koj xav tau los tsim lossis tshem tawm CDC txoj cai.

Txhawm rau qhib, qhib Mapping tab dua, xaiv lub hauv paus database thiab nyem rau ntawm lub pob (Hauv) Qhib Txoj Cai. Xaiv thiab txav mus rau sab xis ntawm lub rooj cov cai uas yuav tsum tau qhib, nyem OK. Tom qab no lawv tau cim ntsuab. Los ntawm kev siv (Hauv) Qhib Txoj Cai Koj tseem tuaj yeem deactivate CDC cov cai. Txhua qhov kev hloov pauv tau ua hauv online.

Tom qab CDC txoj cai tau qhib, cov ntawv sau txog cov ntaub ntawv hloov kho tau raug khaws cia rau hauv qhov chaw cia "ReplCDCArea" raws li qhov chaw database. Cov ntawv sau no yuav raug ua tiav los ntawm cov txheej txheem tshwj xeeb CDCBatch, uas raws li lawv yuav tsim cov ntawv sau hauv Pro2 replication queue hauv cov ntaub ntawv cdc (rov).

Yog li, peb muaj ob kab rau kev rov ua dua. Thawj theem yog CDCBatch: los ntawm qhov chaw database, cov ntaub ntawv thawj zaug mus rau nruab nrab CDC database. Qhov thib ob theem yog thaum cov ntaub ntawv raug xa los ntawm CDC database rau Oracle. Qhov no yog ib tug feature ntawm lub tam sim no architecture thiab cov khoom nws tus kheej - txog tam sim no cov developers tseem tsis tau tsim kom muaj ncaj qha replication.

Thawj synchronization

Tom qab ua kom lub CDC mechanism thiab teeb tsa Pro2 replication server, peb yuav tsum pib qhov pib synchronization. Thawj synchronization hais kom ua:

/pro2/bprepl/Script/replLoad.sh bisquit table-name

Tom qab thawj zaug synchronization tiav, cov txheej txheem rov ua dua tuaj yeem pib.

Pib ntawm cov txheej txheem replication

Txhawm rau pib cov txheej txheem rov ua dua koj yuav tsum tau khiav ib tsab ntawv replbatch.sh. Ua ntej pib, xyuas kom meej tias muaj cov ntawv replbatch rau tag nrho cov xov - replbatch1, replbatch2, thiab lwm yam. Yog tias txhua yam nyob hauv qhov chaw, qhib cov kab hais kom ua (piv txwv li, proenv), mus rau lub directory /bprepl/scripts thiab pib tsab ntawv. Hauv pawg thawj coj saib xyuas, peb xyuas tias cov txheej txheem sib raug tau txais RUNNING raws li txoj cai.

Результаты

Tom qab kev siv, peb tau nrawm heev rau kev xa cov ntaub ntawv mus rau lub tuam txhab cov ntaub ntawv warehouse. Cov ntaub ntawv tau txais mus rau hauv Oracle online. Tsis tas yuav nkim sijhawm khiav qee cov lus nug ntev mus sau cov ntaub ntawv los ntawm ntau lub tshuab. Tsis tas li ntawd, nyob rau hauv cov kev daws teeb meem no cov txheej txheem replication tuaj yeem compress cov ntaub ntawv, uas kuj muaj txiaj ntsig zoo ntawm kev ceev. Tam sim no txhua hnub kev sib haum xeeb ntawm BISKVIT system nrog rau lwm lub tshuab pib siv 15-20 feeb es tsis txhob siv 2-2,5 teev, thiab kev sib haum xeeb ua tiav tau siv ob peb teev es tsis txhob siv ob hnub.

Tau qhov twg los: www.hab.com