Kugadzira nzvimbo yekuchengetera ibasa rakareba uye rakakomba.
Zvakawanda muhupenyu hwepurojekiti zvinoenderana nekuti iyo modhi yechinhu uye chigadziko chimiro zvinofungidzirwa sei pakutanga.
Nzira inowanzogamuchirwa yave uye inoramba iri misiyano yakasiyana-siyana yekubatanidza nyeredzi chirongwa nechitatu fomu yakajairika. Sezvo mutemo, maererano nemusimboti: yekutanga data - 3NF, showcases - nyeredzi. Iyi nzira, yakaedzwa-nguva uye inotsigirwa nehuwandu hwekutsvakurudza, ndiyo yekutanga (uye dzimwe nguva chete) chinhu chinouya mupfungwa dzeDWH ane ruzivo nyanzvi paanenge achifunga nezve iyo analytical repository inofanira kutaridzika.
Kune rimwe divi, bhizinesi mune zvese uye zvinodiwa nevatengi kunyanya zvinoshanduka nekukurumidza, uye data rinowanzo kukura zvese "pakadzika" uye "pahupamhi". Uye apa ndipo panowanikwa kusarudzika kukuru kwenyeredzi - kugumira kuchinjika.
Uye kana muhupenyu hwako hwakanyarara uye hwakasununguka semugadziri weDWH kamwe kamwe:
- basa rakamuka "kuita chimwe chinhu nokukurumidza, uye ipapo tichaona";
- purojekiti inokurumidza kukura yakaonekwa, nekubatana kwezvitsva zvitsva uye kugadzirisazve kwebhizimisi modhi kamwechete pavhiki;
- mutengi aonekwa asingazive kuti sisitimu yacho inofanira kutaridzika sei uye kuti ndeapi mabasa aanofanira kuita pakupedzisira, asi akagadzirira kuyedza uye kugara achinatsa mhedzisiro yaunoda achiramba achiswedera pairi;
- Maneja wepurojekiti akauya nemashoko akanaka okuti: βUye zvino takurumidza!β
Kana kana iwe uchingofarira kutsvaga kuti ungavake sei zvivakwa zvekuchengetera - kugamuchirwa kune yakachekwa!
Chii chinonzi "flexibility" chinorevei?
Chekutanga, ngatitsanangure kuti ndezvipi zvimiro zvinofanirwa kuve nazvo system kuti inzi "inochinjika".
Kusiyana, zvakakosha kutaura kuti izvo zvinotsanangurwa zvivakwa zvinofanirwa kuenderana nazvo system, kwete kuti process kukura kwayo. Naizvozvo, kana iwe waida kuverenga nezve Agile senzira yekusimudzira, zviri nani kuverenga zvimwe zvinyorwa. Semuenzaniso, ipapo, paHabrΓ©, pane zvakawanda zvinonakidza zvinhu (se ΠΈ , uye ).
Izvi hazvireve kuti nzira yekuvandudza uye chimiro chedura re data haina hukama zvachose. Pakazere, zvinofanirwa kuve zviri nyore kugadzira iyo Agile repository yeagile architecture. Nekudaro, mukuita, kazhinji pane zvingasarudzwa neAgile kuvandudza kweiyo DWH yekare maererano naKimbal uye DataVault - maererano neWaterfall, pane kufara kusangana kwekushanduka-shanduka mumhando dzayo mbiri pane imwe chirongwa.
Saka, ndeapi masimba anofanirwa kuve nekuchengetedza anochinjika? Pane zvibodzwa zvitatu pano:
- Kutumirwa kwekutanga uye kukurumidza kutendeuka - izvi zvinoreva kuti mhedzisiro yekutanga yebhizinesi (semuenzaniso, mishumo yekutanga yekushanda) inofanirwa kuwanikwa nekukurumidza sezvinobvira, ndiko kuti, kunyangwe iyo system yese isati yanyatso gadzirwa uye kuitwa. Uyezve, imwe neimwe inotevera kudzokorora kunofanirawo kutora nguva shoma sezvinobvira.
- Iterative kunatsiridza - izvi zvinoreva kuti kuvandudzwa kwega kwega kunotevera hakufanire kukanganisa kushanda kuri kutoshanda. Ndiyo nguva ino inowanzova iyo yakanyanya kutyisa pamapurojekiti makuru - munguva pfupi kana gare gare, zvinhu zvega zvinotanga kuwana zvakawanda zvinongedzo zvekuti zvinova nyore kudzokorora pfungwa mukopi iri padyo pane kuwedzera munda kune tafura iripo. Uye kana iwe uchishamisika kuti kuongorora maitiro ekuvandudzwa kwezvinhu zviripo zvinogona kutora nguva yakawanda kupfuura kuvandudzwa kwacho, iwe unogona kunge usati washanda nehombe dzekuchengetera data mumabhangi kana telecoms.
- Kugara uchienderana nekuchinja bhizinesi zvinodiwa - Chimiro chechinhu chose chinofanira kugadzirwa kwete kungofunga nezvekuwedzera, asi netarisiro yekuti gwara rekuwedzera kunotevera harigone kurotwa padanho rekugadzira.
Uye hongu, kuita zvese izvi zvinodiwa mune imwe system zvinogoneka (hongu, mune zvimwe zviitiko uye nekumwe kusarudzika).
Pazasi ini ndichatarisa maviri eanonyanyozivikanwa agile dhizaini nzira dzekuchengetera data - Anchor model ΠΈ Data Vault. Kusiiwa kunze kwemabhuraketi kune hunyanzvi hwekuita senge, semuenzaniso, EAV, 6NF (mune chimiro chayo) uye zvese zvine chekuita neNoSQL mhinduro - kwete nekuti zvakatoipa, uye kwete nekuti mune iyi nyaya chinyorwa chingatyisidzira kuwana. huwandu hweavhareji disser. Ingori kuti zvese izvi zvine chekuita nemhinduro dzekirasi yakati siyanei - kungave kune matekiniki aunogona kushandisa mune chaiwo mamiriro, zvisinei neiyo yekuvaka yeprojekiti yako (seEAV), kana kune epasi rose mamwe ekuchengetedza ruzivo paradigms (senge girafu dhatabhesi. uye dzimwe sarudzo NoSQL).
Matambudziko e "classical" maitiro uye mhinduro dzawo mune inoshanduka nzira
Ne "classical" maitiro ndinoreva nyeredzi yakanaka yekare (zvisinei nekuitwa chaiko kweiyo pasi pezvikamu, dai vateveri veKimball, Inmon neCDM vandiregererewo).
1. Rigid cardinality yezvibatanidza
Iyi modhi yakavakirwa pakupatsanurwa kwakajeka kwedata mukati Dimension ΠΈ chokwadi. Uye izvi, zvakaipa, zvine musoro - mushure mezvose, kuongororwa kwedata muhuwandu hwakawanda hwenyaya kunouya pakuongororwa kwemamwe manhamba zviratidzo (chokwadi) mune zvimwe zvikamu (zviyero).
Muchiitiko ichi, kuwirirana pakati pezvinhu kunosimbiswa nenzira yehukama pakati pematafura uchishandisa kiyi yekunze. Izvi zvinotaridzika zvakasikwa, asi zvinotungamira kune yekutanga kudzikisira kuchinjika - tsananguro yakasimba yekadhiini yekubatanidza.
Izvi zvinoreva kuti padanho rekugadzira tafura, iwe unofanirwa kunyatso tarisa kune imwe neimwe yezvinhu zvine hukama kana zvichigona kudyidzana sezvizhinji-kune-zvizhinji, kana chete 1-ku-zvizhinji, uye "munzira ipi". Izvi zvinotarisa zvakananga kuti ndeipi tafura ichava nekiyi yekutanga uye iyo ichava nekiyi yekune dzimwe nyika. Kuchinja maitiro aya kana zvitsva zvinodiwa zvinogamuchirwa zvinozotungamira mukugadziriswazve kwebhesi.
Semuenzaniso, pakugadzira chinhu che "receipt yemari", iwe, uchivimba nemhiko dzedhipatimendi rekutengesa, wakaisa mukana wekuita. kumwe kukwidziridzwa kune akati wandei cheki nzvimbo (asi kwete zvinopesana):
Uye mushure menguva yakati, vatinoshanda navo vakaunza nzira nyowani yekushambadzira mavanogona kuita pachinzvimbo chimwe chete kusimudzirwa kwakawanda panguva imwe chete. Uye ikozvino iwe unofanirwa kugadzirisa matafura nekuparadzanisa hukama kuita chinhu chakasiyana.
(Zvinhu zvese zvakatorwa umo cheki yekusimudzira yakabatanidzwa izvozvi inodawo kuvandudzwa).
Hukama muData Vault uye Anchor Model
Kudzivirira mamiriro ezvinhu aya zvakazove nyore: haufanirwe kuvimba nedhipatimendi rekutengesa kuti riite izvi. zvese zvinongedzo zvinotanga kuchengetwa mumatafura akasiyana uye igadzirise sezvizhinji-kune-zvizhinji.
Iyi nzira yakarongwa Dan Linstedt sechikamu cheparadigm Data Vault uye inotsigirwa zvizere Lars RΓΆnnbΓ€ck Π² Anchor Model.
Nekuda kweizvozvo, tinowana yekutanga yakasarudzika maitiro anochinjika nzira:
Hukama pakati pezvinhu hahuchengetwe muhunhu hwenhengo dzevabereki, asi imhando yechinhu chakasiyana.
Π Data Vault matafura ekubatanidza akadaro anonzi batanidzo, uye mukati Anchor Model - Tie. Pakutanga, vakafanana zvikuru, kunyange zvazvo kusiyana kwavo kusingagumi nezita (iro richakurukurwa pasi apa). Mune zvese zvivakwa, link matafura anogona kubatana chero nhamba yemasangano (kwete hazvo 2).
Iyi redundancy, pakutanga pakuona, inopa yakakosha shanduko yekuchinja. Chimiro chakadaro chinova chinoshivirira kwete chete kune shanduko mumakadhiinality ezvisungo zviripo, asiwo nekuwedzera kwezvitsva - kana ikozvino cheki chinzvimbo chinewo chinongedzo kune cashier akaivhura, kutaridzika kweiyo link inongove kuwedzera-pamusoro pamatafura aripo pasina kukanganisa chero zvinhu zviripo uye maitiro.
2. Kudzokorora data
Dambudziko rechipiri rinogadziriswa neanochinjika zvivakwa harina kujeka uye rinogara munzvimbo yekutanga. SCD2 mhando zviyero (zvishoma nezvishoma kushandura zviyero zverudzi rwechipiri), kunyange zvisiri ivo chete.
Mune yekare imba yekuchengetera, chiyero chinowanzo tafura ine kiyi yekutsiva (sePK) uye seti yemakiyi ebhizinesi uye hunhu mumakoramu akasiyana.
Kana chiyero chichitsigira shanduro, miganhu yechokwadi yevhezheni inowedzerwa kune yakajairwa seti yeminda, uye pamutsara mumwechete mune tsime, akawanda mavhezheni anooneka mune repository (imwe yeimwe neimwe shanduko mushanduro hunhu).
Kana dimension iine kanenge kamwechete kanowanzo chinja vhezheni hunhu, huwandu hwemavhezheni ehukuru hwakadai huchanakidza (kunyangwe kana hunhu hwasara husina kushandurwa kana kusambochinja), uye kana paine akati wandei akadai, huwandu hweshanduro hunogona. kukura zvakanyanya kubva kunhamba yavo. Iyi chiyero inogona kutora yakakura yakawanda yedhisiki nzvimbo, kunyangwe yakawanda yedata yainochengeta ingori makopi easinga shanduke hunhu hutsika kubva kune mamwe mitsara.
Panguva imwecheteyo, inoshandiswawo zvakanyanya denormalization - humwe hunhu hunochengetwa nemaune sehukoshi, uye kwete sechinongedzo kubhuku rereferenzi kana imwe chiyero. Iyi nzira inomhanyisa kuwana data, ichidzikisa huwandu hwemajoini kana uchinge wawana dimension.
Kazhinji izvi zvinotungamirira kune mashoko akafanana anochengetwa panguva imwe chete munzvimbo dzinoverengeka. Semuenzaniso, ruzivo nezve dunhu rekugara uye chikamu chemutengi chinogona kuchengetwa panguva imwe chete muzvikamu zve "Client" uye "Purchase", "Delivery" uye "Call Center Calls" chokwadi, pamwe ne "Client - Client Manager." β link table.
Kazhinji, zvinotsanangurwa pamusoro apa zvinoshanda kune zvakajairwa (zvisina-shanduro) zviyero, asi mune zvakashandurwa zvinogona kunge zvine chiyero chakasiyana: kutaridzika kweshanduro itsva yechinhu (kunyanya mukutarisa) kunotungamirira kwete kungovandudzwa kwezvose zvine hukama. matafura, asi kune chitarisiko chinoputika chezvinyorwa zvitsva zvezvinhu zvine hukama - apo Tafura 1 inoshandiswa kuvaka Tafura 2, uye Tafura 2 inoshandiswa kuvaka Tafura 3, nezvimwe. Kunyangwe pasina kana hunhu humwechete hweTafura 1 hunobatanidzwa mukuvakwa kweTafura 3 (uye humwe hunhu hweTafura 2 hunowanikwa kubva kune mamwe masosi hunobatanidzwa), kushandura chivakwa ichi kuchaita padiki kutungamira kune yakawedzera kumusoro, uye pamwero mukuru kusvika pakuwedzera. shanduro muTable 3. iyo isina chokuita nazvo zvachose, uye kuwedzera pasi chetani.
3. Nonlinear kuoma kwekugadzirisa zvakare
Panguva imwecheteyo, chitoro chitsva chega chega chakavakirwa pahwaro hwechimwe chinowedzera huwandu hwenzvimbo iyo data inogona "kutsauka" kana shanduko inoitwa kuETL. Izvi, zvakare, zvinotungamirira kukuwedzera kwekuoma (uye nguva) yekudzokorora kwega kwega kunotevera.
Kana iyo iri pamusoro ichitsanangura masisitimu ane isingawanzo kugadziridzwa ETL maitiro, unogona kugara mune yakadaro paradigm - iwe unongoda kuve nechokwadi chekuti magadzirirwo matsva akaitwa nemazvo kune ese ane hukama zvinhu. Kana zvidzokorodzo zvikaitika kakawanda, mukana we "kushaikwa" netsaona zvibatanidza zvakawanda zvinowedzera zvakanyanya.
Kana, nekuwedzera, tikafunga kuti "versioned" ETL yakanyanya kuomarara pane "isina-versioned" imwe, zvinova zvakaoma kudzivirira kukanganisa paunogara uchigadziridza iyi nzvimbo yese.
Kuchengeta zvinhu uye hunhu muData Vault uye Anchor Model
Iyo nzira yakatsanangurwa nevanyori veanochinjika zvivakwa inogona kuumbwa seinotevera:
Zvakakosha kupatsanura shanduko kubva kune inoramba yakafanana. Ndiko kuti, chengetedza makiyi akasiyana nehunhu.
Zvisinei, munhu haafaniri kuvhiringidza not versioned hunhu ne isina kushanduka: yekutanga haina kuchengetedza nhoroondo yekuchinja kwayo, asi inogona kuchinja (semuenzaniso, kana uchigadzirisa kukanganisa kwekupinza kana kugamuchira data nyowani); yechipiri haimbochinji.
Mapoinzi ekuona anosiyana pane izvo chaizvo zvinogona kutariswa zvisingachinjike muData Vault uye Anchor Model.
Kubva pakuona kwekuvaka Data Vault, inogona kunzi haina kuchinjwa yeseti yemakiyi - zvakasikwa (TIN yesangano, kodhi yechigadzirwa mune sosi system, nezvimwewo) uye surrogate. Muchiitiko ichi, hunhu hwasara hunogona kukamurwa kuita mapoka zvichienderana nekwakabva uye/kana kuwanda kwekuchinja uye Chengetedza tafura yakasiyana yeboka rega rega ine yakazvimiririra seti yeshanduro.
Mune paradigm Anchor Model inofungidzirwa isina kuchinjwa chete surrogate key essence. Zvimwe zvese (kusanganisira makiyi echisikigo) ingori nyaya yakakosha yehunhu hwayo. Wherein hunhu hwese hwakazvimiririra kune umwe neumwe nekukasira, saka pahunhu hwega hwega a tafura yakasiyana.
Π Data Vault matafura ane entity keys anonzi Hubami. Hubs inogara iine yakatarwa seti yeminda:
- Natural Entity Keys
- Surrogate key
- Link kune source
- Rekodha nguva yekuwedzera
Zvinyorwa muHubs usambochinja uye hauna shanduro. Kunze, hubs dzakanyanya kufanana neID-mepu mhando matafura anoshandiswa mune mamwe masisitimu kugadzira surrogates, zvisinei, zvinokurudzirwa kushandisa hashi kubva kune seti yemakiyi ebhizinesi sevanotsiva muData Vault. Iyi nzira inorerutsa kurodha hukama uye hunhu kubva kune masosi (haufanire kujoinha hub kuti uwane surrogate, iwe unongoda kuverenga hashi yekiyi yechisikigo), asi inogona kukonzera mamwe matambudziko (ane chekuita, semuenzaniso, kudhumhana. , makesi uye mavara asingadhindike mumakiyi etambo, nezvimwewo .p.), saka haigamuchirwi.
Humwe humwe hunhu hwenhengo hunochengetwa mumatafura akakosha anonzi Satellites. Imwe hub inogona kuva nemasatelliti akati wandei anochengeta akasiyana seti ehunhu.
Kugoverwa kwehunhu pakati pemasatellite kunoitika zvinoenderana nemusimboti kuchinja kwemajoini - mune imwe setiraiti isina-shanduro hunhu inogona kuchengetwa (somuenzaniso, zuva rekuzvarwa uye SNILS kune mumwe munhu), mune imwe - isingawanzo kushandura shanduro (semuenzaniso, zita rekupedzisira uye nhamba yepasipoti), mune yechitatu - inochinja kazhinji. (semuenzaniso, kero yekutumira, chikamu, zuva rekupedzisira odha, nezvimwewo). Muchiitiko ichi, kushandura kunoitwa pamwero wemasatelliti ega, uye kwete sangano rose, saka zvinokurudzirwa kugovera hunhu kuitira kuti mharadzano yeshanduro mukati mesetiraiti imwe ishoma (izvo zvinoderedza huwandu hwese hweshanduro dzakachengetwa. )
Zvakare, kukwirisa maitiro ekurodha data, hunhu hunowanikwa kubva kwakasiyana masosi hunowanzo kuverengerwa mune yega satellite.
Satellite anotaurirana neHub kuburikidza kiyi yekune dzimwe nyika (iyo inoenderana ne1-kune-yakawanda kadhinari). Izvi zvinoreva kuti hunhu hwakawanda hunokosha (semuenzaniso, nhamba dzenhare dzakawanda dzemutengi mumwe) dzinotsigirwa neiyi "default" yekuvaka.
Π Anchor Model matafura anochengeta makiyi anonzi Anchors. Uye vanochengeta:
- Makiyi ekutsiva chete
- Link kune source
- Rekodha nguva yekuwedzera
Makiyi echisikigo kubva pakuona kweAnchor Model anoonekwa zvakajairika. Iyi sarudzo inogona kuita seyakaoma kunzwisisa, asi inopa mukana wakawanda wekuziva chinhu.
Semuenzaniso, kana data pamusoro pechinhu chimwe chete ichigona kubva kune akasiyana masisitimu, imwe neimwe inoshandisa yayo chaiyo kiyi. MuData Vault, izvi zvinogona kutungamira kune zvinoremedza zvimiro zveakati wandei hubs (imwe yega sosi + inobatanidza master vhezheni), nepo muAnchor modhi, kiyi yechisikigo chega yega sosi inowira mune yayo hunhu uye inogona kushandiswa pakurodha yakazvimiririra. vamwe vose.
Asi pane zvakare imwe pfungwa inonyengera pano: kana hunhu kubva kune akasiyana masisitimu akasanganiswa mune chimwe chinhu, kazhinji pane zvimwe. mitemo ye "gluing", iyo sisitimu inofanirwa kunzwisisa kuti marekodhi kubva kwakasiyana-siyana zvinoenderana nechiitiko chimwe chesangano.
Π Data Vault mitemo iyi inogona kunyanya kusarudza kuumbwa "Surrogate hub" yechinhu chikuru uye kwete nenzira ipi neipi inopesvedzera maHub anochengeta echisikigo makiyi uye hunhu hwavo hwepakutanga. Kana pane imwe nguva mitemo yekubatanidza ichichinja (kana hunhu hwainoitwa nahwo hwakagadziridzwa), zvichave zvakakwana kugadziridza nzvimbo dzekugara.
Π Anchor model chinhu chakadaro chingango chengetedzwa mukati chibatiso chega. Izvi zvinoreva kuti hunhu hwese, zvisinei kuti hunobva kupi, hunosungwa kune mumwe chete surrogate. Kupatsanura marekodhi akabatanidzwa zvisiri izvo uye, kazhinji, kutarisa kukosha kwekubatanidza muhurongwa hwakadaro kunogona kuve kwakaoma zvakanyanya, kunyanya kana iyo mitemo yakaoma uye ichichinja kazhinji, uye hunhu humwe chete hunogona kuwanikwa kubva kwakasiyana (kunyangwe iri chokwadi. zvinogoneka, sezvo imwe neimwe vhezheni inochengeta chinongedzo kune kwainotangira).
Chero zvazvingaitika, kana system yako ichifanirwa kuita basa racho deduplication, kubatanidza marekodhi uye zvimwe zvinhu zveMDM, zvakakodzera kutarisisa kune zvikamu zvekuchengeta makiyi echisikigo mune agile nzira. Zvingangoita kuti iyo bulkier Data Vault dhizaini ichaerekana yakachengeteka maererano nekusanganisa zvikanganiso.
Anchor model inopawo imwe mhando yechinhu chinonzi Knot zvakakosha chaizvo degenerate type of anchor, iyo inogona kuva nechinhu chimwe chete. Manodhi anofanirwa kushandiswa kuchengetedza akafuratira madhairekitori (semuenzaniso, murume kana mukadzi, chimiro chewanano, chikamu chebasa revatengi, nezvimwewo). Kusiyana neAnchor, iyo Knot haina mubatanidzwa matafura, uye hunhu hwayo chete (zita) rinogara rakachengetwa mutafura imwechete nekiyi. Node dzakabatana neAnchors nematafura machira (Tie) nenzira imwecheteyo seAnchors yakabatana kune imwe neimwe.
Iko hakuna maonero akajeka maererano nekushandiswa kweNodes. Semuyenzaniso, , uyo anoshingairira kukurudzira kushandiswa kweAnchor Model muRussia, anotenda (kwete zvisina musoro) kuti hapana kana bhuku rimwe chete rereferensi rinogona kutaurwa nechokwadi kuti. nguva dzose ichave yakamira uye imwechete-level, saka zviri nani kukurumidza kushandisa yakazara-yakazara Anchor yezvinhu zvese.
Mumwe mutsauko wakakosha pakati peData Vault uye Anchor modhi ndiko kuwanikwa hunhu hwekubatanidza:
Π Data Vault Manongedzo ndiwo akafanana akazara-akazara zvinhu seHubs, uye anogona kuva hunhu hwako. The Anchor model Zvisungo zvinoshandiswa chete kubatanidza Anchors uye havakwanise kuva nehunhu hwavo. Musiyano uyu unoguma nenzira dzakasiyana siyana dzekuenzanisira chokwadi, iyo ichakurukurwa mberi.
Chokwadi kuchengetedza
Izvi zvisati zvaitika, taitaura zvakanyanya nezve kuyerwa kwekuenzanisa. Chokwadi hachina kujeka zvishoma.
Π Data Vault chinhu chakajairika chekuchengetedza chokwadi ndeye Link, mune masatellite chaiwo zviratidzo zvinowedzerwa.
Iyi nzira inoita seinonzwisisika. Inopa nyore kuwana kune yakaongororwa zviratidzo uye inowanzofanana neyakajairwa tafura yechokwadi (chete zviratidzo zvinochengetwa kwete mutafura pachayo, asi mu "muvakidzani" tafura). Asi kune zvakare misungo: imwe yeyakajairwa shanduko yemuenzaniso - kuwedzera kwekiyi kiyi - inoda. kuwedzera kiyi itsva yekunze kune Link. Uye izvi, zvakare, "zvinotyora" modularity uye zvinogona kukonzera kudiwa kwekugadziriswa kune zvimwe zvinhu.
Π Anchor model Kubatana hakugone kuve neayo hunhu, saka nzira iyi haishande - zvachose hunhu uye zviratidzo zvinofanirwa kubatanidzwa kune imwe chaiyo anchor. Mhedziso kubva pane iyi iri nyore - Chokwadi chimwe nechimwe chinodawo simbisiro yacho. Kune zvimwe zvatakajaira kuona sechokwadi, izvi zvinogona kutaridzika sechisikigo - semuenzaniso, chokwadi chekutenga chinogona kudzikiswa zvakakwana kune chinhu "kuraira" kana "risiti", kushanyira saiti kuchikamu, nezvimwe. Asi panewo chokwadi chekuti hazvisi nyore kuwana "chinhu chinotakura" chakadaro - semuenzaniso, zvakasara zvezvinhu mumatura pakutanga kwezuva rega rega.
Saizvozvo, matambudziko nemodularity kana uchiwedzera kiyi yechokwadi muiyo Anchor modhi haimuke (zvakakwana kungo wedzera Hukama hutsva kune inoenderana Anchor), asi kugadzira modhi yekuratidzira chokwadi haina kujeka; "artificial" Anchors anogona kuoneka. iyo inoratidza chimiro chechinhu chebhizinesi nenzira isina kujeka.
Kuti kuchinjika kunowanikwa sei
Kuvakwa kunoguma mune zvese zviri zviviri zvine zvikuru mamwe matafurakupfuura kuyera kwechinyakare. Asi zvinogona kutora zvakanyanya zvishoma disk nzvimbo neseti imwechete yezvimiro zvakashandurwa seyechinyakare dimension. Sezvingatarisirwa, hapana mashiripiti pano - zvese nezve normalization. Nekugovera hunhu mukati meSatellite (muData Vault) kana matafura ega (Anchor Model), isu tinoderedza (kana kubvisa zvachose) kudzokorora kwehunhu hwehumwe hunhu kana uchichinja humwe.
nokuti Data Vault iyo winnings ichaenderana nekugoverwa kwehunhu pakati peSatellite, uye ye Anchor model β inoda kuenzana neavhareji yenhamba yeshanduro pachiyero chechinhu.
Nekudaro, kuchengetedza nzvimbo kwakakosha, asi kwete iyo huru, mukana wekuchengeta hunhu zvakasiyana. Pamwe chete nekuchengetedza kwakasiyana kwehukama, nzira iyi inoita chitoro modular design. Izvi zvinoreva kuti kuwedzera ese ari maviri hunhu uye yakazara nzvimbo dzezvidzidzo zvitsva mumuenzaniso wakadaro zvinotaridzika superstructure pamusoro pegadziriro iripo yezvinhu pasina kuzvishandura. Uye izvi ndizvo chaizvo zvinoita kuti nzira dzakatsanangurwa dzishanduke.
Izvi zvakafanana neshanduko kubva pakugadzirwa kwechidimbu kuenda kukugadzirwa kwakawanda - kana nenzira yechinyakare tafura yega yega yemuenzaniso yakasarudzika uye inoda kutariswa kwakanyanya, saka nenzira dzinochinjika yatove seti yeyakajairwa "zvikamu". Kune rimwe divi, kune mamwe matafura, uye maitiro ekurodha nekutora data anofanirwa kutaridzika zvakanyanya kuoma. Kune rumwe rutivi, vanova normal. Zvinoreva kuti panogona kunge paine otomatiki uye metadata inofambiswa. Mubvunzo wekuti "tichaisa sei?", Mhinduro iyo inogona kutora chikamu chakakosha chebasa rekugadzira kuvandudza, ikozvino haina kufanira (pamwe nemubvunzo nezve kukanganisa kwekuchinja modhi pamaitiro ekushanda. )
Izvi hazvireve kuti vaongorori havadiwe muhurongwa hwakadaro zvachose - mumwe munhu achiri kuda kushanda kuburikidza neseti yezvinhu zvine hunhu uye kuona kuti kupi uye sei kurodha zvese. Asi huwandu hwebasa, pamwe chete nemukana uye mutengo wekukanganisa, zvakaderedzwa zvakanyanya. Zvese zviri padanho rekuongorora uye panguva yekuvandudzwa kweETL, iyo muchikamu chakakosha inogona kudzikiswa kuita metadata yekugadzirisa.
Divi rakasviba
Zvese zviri pamusoro zvinoita kuti nzira dzese dzinyatso shanduka, tekinoroji yepamusoro uye yakakodzera kuvandudzwa kwekudzokorora. Ehezve, pane zvakare "dhiramu mune yekuzora", iyo yandinofunga iwe unogona kutofungidzira nezvayo.
Kuparara kwedata, uko kuri pasi peiyo modularity yeanochinjika zvivakwa, zvinotungamira mukuwedzera kwenhamba yematafura uye, saizvozvo, pamusoro kujoina pakuita sampling. Kuti ungowana hunhu hwese hwechiyero, muchitoro chekare imwe sarudzo inokwana, asi inochinjika dhizaini inoda nhevedzano yekubatanidza. Uyezve, kana zvese izvi zvakabatana zvemishumo zvinogona kunyorwa pachine nguva, saka vaongorori vakajaira kunyora SQL nemaoko vachatambura zvakapetwa kaviri.
Pane zvinhu zvakawanda zvinoita kuti mamiriro aya ave nyore:
Paunenge uchishanda nehukuru hukuru, hunhu hwayo hwese hunenge husina kumboshandiswa panguva imwe chete. Izvi zvinoreva kuti panogona kunge paine mashoma ekujoinha pane zvinoita sekutanga kutarisa pamuenzaniso. Data Vault inogona zvakare kufunga nezve inotarisirwa frequency yekugovana kana uchigovera hunhu kune satellite. Panguva imwecheteyo, maHub kana Anchors pachawo anodiwa zvakanyanya kugadzira uye mepu vanomirira padanho rekurodha uye haawanzo kushandiswa mumibvunzo (izvi ndezvechokwadi kune Anchors).
Majoini ese ari nekiyi. Uye zvakare, imwe nzira "yakadzvanywa" yekuchengetedza data inoderedza kumusoro kwekutarisa matafura painodiwa (semuenzaniso, kana kusefa nehunhu kukosha). Izvi zvinogona kutungamira kune chokwadi chekuti sampling kubva kune yakajairwa dhatabhesi ine boka remajoini ichave yakakurumidza kupfuura kuongorora imwe inorema dimension ine akawanda mavhezheni pamutsetse.
Somuenzaniso, pano Chinyorwa chacho chine yakadzama yekuenzanisa bvunzo yekuita kweiyo Anchor modhi ine sampuli kubva pane imwe tafura.
Zvakawanda zvinoenderana neinjini. Mapuratifomu mazhinji emazuva ano ane majoin optimization emukati. Semuenzaniso, MS SQL neOracle vanogona "kusvetuka" kujoinha kumatafura kana data ravo risingashandiswe chero kupi kunze kwekunge kune mamwe majoini uye isingakanganise sarudzo yekupedzisira (tafura/join kubviswa), uye MPP Vertica. , yakaratidza kuva injini yakanaka kwazvo yeAnchor Model, yakapihwa mamwe manyorero ekugadzirisa chirongwa chemubvunzo. Kune rimwe divi, kuchengetedza iyo Anchor Model, semuenzaniso, pa Click House, iyo ine shoma yekujoinha rutsigiro, haisati yaita senge zano rakanaka kwazvo.
Mukuwedzera, kune zvose zvivakwa zviripo special moves, kuita kuti kuwanikwa kwedata kuve nyore (zvese kubva pamubvunzo wekuita maonero uye kune vashandisi vekupedzisira). Semuyenzaniso, Point-In-Nguva matafura muData Vault kana yakakosha tafura mabasa muAnchor modhi.
Total
Chinhu chikuru cheanoonekwa anochinjika zvivakwa ndeye modularity ye "dhizaini" yavo.
Iyi ndiyo pfuma inobvumira:
- Mushure meimwe yekutanga gadziriro ine chekuita nemetadata kuendesa uye kunyora ekutanga ETL algorithms, kurumidza kupa mutengi mhedzisiro yekutanga muchimiro chemishumo yakati wandei ine data kubva kune mashoma sosi zvinhu. Hazvina kudikanwa kunyatso funga kuburikidza (kunyangwe padanho repamusoro) iyo yose yechinhu modhi.
- A data model inogona kutanga kushanda (uye inobatsira) ne 2-3 chete zvinhu, uye ipapo kukura zvishoma nezvishoma (nezve Anchor modhi Nikolai kuenzanisa kwakanaka ne mycelium).
- Kwakawanda kuvandudzwa, kusanganisira kuwedzera nzvimbo yezvidzidzo uye kuwedzera zvinyorwa zvitsva haikanganisi mashandiro aripo uye haikonzeri njodzi yekupwanya chimwe chinhu chiri kutoshanda.
- Nekuda kwekuora kuita zvinhu zvakajairwa, ETL maitiro mune akadaro masisitimu anotaridzika zvakafanana, kunyora kwavo kunokweretesa kune algorithmization uye, pakupedzisira, otomatiki.
Mutengo wekushanduka uku kuita. Izvi hazvirevi kuti hazvibviri kuwana kushanda kunogamuchirwa pamhando dzakadaro. Kazhinji pane kwete, iwe ungangoda kushanda nesimba uye kutarisisa kune zvakadzama kuti uwane metrics yaunoda.
Apps
Entity marudzi Data Vault
Mamwe ruzivo nezve Data Vault:
Entity marudzi Anchor Model
Mamwe ruzivo nezve Anchor Model:
Pfupiso tafura ine zvakajairika maficha uye misiyano yemaitiro akatariswa:
Source: www.habr.com