Hei Habr!
Data chinhu chakakosha chekambani. Inenge kambani yese ine dhijitari yakatarisa inozivisa izvi. Zvakaoma kupokana neizvi: hapana kana musangano mukuru weIT unoitwa pasina kukurukura nzira dzekutonga, kuchengetedza uye kugadzirisa data.
Dhata inouya kwatiri kubva kunze, inogadzirwawo mukati mekambani, uye kana tikataura nezve data kubva kukambani yenharembozha, saka kune vashandi vemukati iyi idura reruzivo nezve mutengi, zvaanofarira, maitiro, uye nzvimbo. Nekurongeka kwakakodzera uye kupatsanura, zvinopihwa zvekushambadzira zvinonyanya kushanda. Nekudaro, mukuita, hazvisi zvese zvine rosy. Iyo data inochengetwa nemakambani inogona kunge isiri yekare, isina basa, kudzokorora, kana kuvepo kwayo hazvizivikanwe kune chero munhu kunze kwedenderedzwa diki revashandisi. ¯_(ツ)_/¯
Mushoko, data inofanirwa kutungamirwa zvinobudirira - chete ipapo ichave chinhu chinounza mabhenefiti chaiwo uye purofiti kubhizinesi. Nehurombo, kugadzirisa nyaya dzekutarisira data kunoda kukunda zvakanyanya kuomarara. Izvo zvinonyanya kukonzerwa nezvose zviri zviviri nhaka yenhoroondo muchimiro che "zoo" yehurongwa uye kushomeka kwemaitiro akabatana uye nzira dzekutungamira kwavo. Asi zvinorevei kuva "data driven"?
Izvi ndizvo chaizvo zvatichataura nezvazvo pasi pekuchekwa, pamwe nekuti iyo opensource stack yakatibatsira sei.
Iyo pfungwa yehurongwa hwekutarisira data Data Governance (DG) yatove yakanyatsozivikanwa mumusika weRussia, uye zvinangwa zvakawanikwa nebhizinesi semugumisiro wekuita kwayo zvakajeka uye zvakajeka kuziviswa. Kambani yedu yakanga isiri iyo uye yakazvigadzirira iyo basa rekuunza iyo pfungwa yekutarisira data.
Saka takatangira papi? Kutanga, takazvigadzirira zvinangwa zvakakosha:
- Chengetedza data redu richiwanikwa.
- Ita shuwa kuve pachena kweiyo data lifecycle.
- Ipa vashandisi vekambani data inowirirana, inowirirana.
- Ipa vashandisi vekambani data yakasimbiswa.
Nhasi, kune gumi nemaviri Data Governance kirasi maturusi pamusika wesoftware.
Asi mushure mekuongorora kwakadzama uye kudzidza kwemhinduro, isu takanyora akati wandei akakosha mhinduro isu pachedu:
- Vazhinji vagadziri vanopa iyo yakazara seti yemhinduro, iyo kwatiri isingadhuri uye inodzokorora mashandiro aripo. Uyezve, inodhura maererano nezviwanikwa, kubatanidzwa mune yazvino IT landscape.
- Iko kushanda uye interface zvakagadzirirwa tekinoroji, kwete bhizinesi rekupedzisira vashandisi.
- Kuderera kwekupona kwezvigadzirwa uye kushomeka kwekubudirira kwekuita pamusika weRussia.
- Kudhura kwakanyanya kwesoftware uye kumwe kutsigirwa.
Maitiro uye kurudziro zvakataurwa pamusoro apa maererano nekumisikidzwa kwesoftware kumakambani eRussia zvakatikurudzira kuti tiende kubudiriro yedu pachedu yakavhurika stack. Nzvimbo yatakasarudza yaive Django, yemahara uye yakavhurika sosi sisitimu yakanyorwa muPython. Uye nekudaro takaona mamodule akakosha anozobatsira kune zvinangwa zvataurwa pamusoro apa:
- Rejista yemishumo.
- Bhizinesi dudziro.
- Module yekutsanangura shanduko yehunyanzvi.
- Module yekutsanangura kutenderera kwehupenyu hwedata kubva kutsime kuenda kuBI chishandiso.
- Data quality control module.
Rejista yemishumo
Zvinoenderana nemhedzisiro yezvidzidzo zvemukati mumakambani makuru, pakugadzirisa matambudziko ane chekuita nedata, vashandi vanopedza 40-80% yenguva yavo vachivatsvaga. Naizvozvo, takazvigadzirira basa rekugadzira ruzivo rwakavhurika nezvemishumo yaivepo yaimbowanikwa kune vatengi chete. Nekudaro, isu tinodzikisira nguva yekugadzira mishumo mitsva uye nekuona democratization yedata.
Rejista yekuzivisa yave hwindo rimwe rekuzivisa kune vashandisi vemukati kubva kumatunhu akasiyana, madhipatimendi, uye zvikamu. Iyo inosanganisa ruzivo rwemashoko masevhisi akagadzirwa mune akati wandei emakambani repositori yekambani, uye kune akawanda awo muRostelecom.
Asi iyo registry haingori rondedzero yakaoma yemashumo akagadzirwa. Pamushumo wega wega, tinopa ruzivo rwunodiwa kuti mushandisi ajairane nayo:
- tsananguro pfupi yemushumo;
- kudzika kwekuwanikwa kwedata;
- chikamu chevatengi;
- visualization tool;
- zita rekuchengetedza kwekambani;
- bhizinesi rinoshanda zvinodiwa;
- link kune report;
- link kune application yekuwana;
- kuita chimiro.
Yekushandisa level analytics iripo kune mishumo, uye mishumo inoiswa pamusoro peiyo rondedzero zvichibva pane log analytics zvichienderana nehuwandu hwevashandisi vakasiyana. Uye handizvo. Pamusoro pezvakajairwa hunhu, isu takapawo tsananguro yakadzama yekuumbwa kwehunhu hwemishumo nemienzaniso yehunhu uye nzira dzekuverenga. Kudonongodza kwakadaro kunongopa mushandisi mhinduro kuti shumo yacho inomubatsira here kana kuti kwete.
Kuvandudzwa kweiyi module yaive nhanho yakakosha mudemocratization yedata uye zvakanyanya kuderedza nguva yainotora kuwana ruzivo rwunodiwa. Pamusoro pekudzikisa nguva yekutsvaga, huwandu hwezvikumbiro kuchikwata chekutsigira kuti vape mazano hwakadzikirawo. Hazvigoneke kuti tisacherechedze mumwe mubairo unobatsira watakawana nekugadzira rejista yakabatana yemishumo - kudzivirira kugadzirwa kweduplicate mishumo yezvikamu zvakasiyana zvezvimiro.
Bhizinesi dudziro
Imi mose munoziva kuti kunyange mukati mekambani imwe chete, mabhizinesi anotaura mitauro yakasiyana. Hongu, vanoshandisa mazwi akafanana, asi zvinoreva zvakasiyana zvachose. Bhizinesi glossary rakagadzirirwa kugadzirisa dambudziko iri.
Kwatiri, glossary yebhizinesi harisi bhuku rereferenzi rine tsananguro yematemu uye maitiro ekuverenga. Iyi inzvimbo yakazara-yakazara yekugadzira, kubvumirana uye kubvumidza mazwi, kuvaka hukama pakati pematemu uye mamwe ruzivo ruzivo rwekambani. Usati wapinda mubhizinesi glossary, izwi rinofanirwa kupfuura nematanho ese ekubvumidzwa nevatengi vebhizinesi uye data data data. Chete mushure meizvi ndipo inowanikwa kuti ishandiswe.
Sezvandanyora pamusoro, kusarudzika kwechishandiso ichi ndechekuti inobvumira kubatanidza kubva padanho rebhizinesi rekuti kune chaiyo mushandisi mishumo yaanoshandiswa mairi, pamwe neiyo nhanho yezvinhu zve database.
Izvi zvinogoneka kuburikidza nekushandiswa kweglossary term identifiers mune yakadzama tsananguro ye registry mishumo uye tsananguro yezvinhu zvepa database.
Parizvino, mazwi anopfuura 4000 akatsanangurwa uye akabvumiranwa muGlossary. Kushandiswa kwayo kunorerutsa uye nekumhanyisa kugadziridzwa kwezvikumbiro zvinouya zvekuchinja mumasisitimu eruzivo ekambani. Kana chiratidzo chinodiwa chatoitwa mune chero mushumo, ipapo mushandisi anobva aona seti yeakagadzirira-yakagadzirwa mishumo panoshandiswa chiratidzo ichi, uye achakwanisa kusarudza pamusoro pekushandiswa zvakare kunoshanda kwekuita kwagara kana kugadziridzwa kwayo kushoma, pasina kutanga. zvikumbiro zvitsva zvekugadzirwa kwechinyorwa chitsva.
Module yekutsanangura shanduko yehunyanzvi uye DataLineage
Ndeapi aya mamodule, iwe unobvunza? Izvo hazvina kukwana kungo shandisa iyo Report Rejista uye Glossary; zvinodikanwa zvakare kudzika mazwi ese ebhizinesi pane iyo chaiyo dhatabhesi modhi. Nekudaro, isu takakwanisa kupedzisa maitiro ekugadzira iyo data data kutenderera kubva kusource masisitimu kuenda kuBI kuona kuburikidza nezvikamu zvese zvedura re data. Mune mamwe mazwi, gadzira DataLineage.
Isu takagadzira chinongedzo chinoenderana nefomati yakashandiswa kare mukambani kutsanangura mitemo uye pfungwa yekushandurwa kwedata. Ruzivo rumwe cheterwo rwunopinzwa kuburikidza neiyo interface sepakutanga, asi tsananguro yeizwi identifier kubva kubhizinesi glossary yave chinhu chinodiwa. Aya ndiwo magadzirirwo atinoita hukama pakati pebhizinesi uye zvidimbu zvemuviri.
Ndiani anoida? Chii chakanga chakaipa nefomati yekare yawakashanda nayo kwemakore akati wandei? Mari yevashandi yekugadzira zvinodiwa yakakwira yakawanda sei? Taifanira kutarisana nemibvunzo yakadai panguva yekushandiswa kwechishandiso. Mhinduro dziri pano dzakareruka - tese tinoda izvi, hofisi yedata yekambani yedu nevashandisi vedu.
Chokwadi, vashandi vaifanira kuchinjika; pakutanga, izvi zvakakonzera kuwedzera kudiki kwemitengo yevashandi pakugadzira zvinyorwa, asi isu takagadzirisa nyaya iyi. Dzidzira, kuona uye optimize dambudziko nzvimbo vakaita basa ravo. Isu takawana chinhu chikuru - isu takavandudza hutano hwezvakagadzirwa zvinodiwa. Minda inosungirwa, mabhuku akabatana ereferenzi, masks ekuisa, akavakirwa-mukati macheki - zvese izvi zvakaita kuti zvikwanise kuvandudza zvakanyanya kunaka kwekutsanangurwa kweshanduko. Takasiya tsika yekupa zvinyorwa mumhando yezvinodiwa zvekusimudzira uye kugovana ruzivo rwaingowanikwa kune boka rekuvandudza. Iyo yakagadzirwa metadata dhatabhesi inoderedza zvakanyanya nguva inodiwa kuita regression ongororo uye inopa kugona kukurumidza kuongorora kukanganisa kwekuchinja pane chero layer yeIT landscape (showcase report, aggregates, sources).
Izvi zvine chekuita nei nevashandisiwo zvavo mishumo, ndezvipi zvakanakira ivo? Nekuda kwekugona kuvaka DataLineage, vashandisi vedu, kunyangwe avo vari kure neSQL uye mimwe mitauro yekuronga, vanokurumidza kugamuchira ruzivo nezvezvinobva uye zvinhu pahwaro hwekuti imwe rondedzero inogadzirwa.
Data Quality Control Module
Zvese zvatakataura pamusoro pazvo maererano nekuona kuti data iri pachena hazvina kukosha pasina kunzwisisa kuti data ratinopa vashandisi ndeyechokwadi. Imwe yemamodule akakosha eiyo Dhata Governance pfungwa ndeye data quality control module.
Padanho razvino, iyi ikhathalogi yemacheki emasangano akasarudzwa. Chinangwa chekukurumidza chekugadzirwa kwechigadzirwa ndechekuwedzera rondedzero yecheki uye kubatanidza neregistry yekubika.
Ichapei uye kunaani? Mushandisi wekupedzisira we registry achawana ruzivo rwemazuva akarongwa uye chaiwo ekugadzirira mushumo, mhedzisiro yekupedzwa kwecheki ine dynamics, uye ruzivo rwezvakaiswa mushumo.
Kwatiri, iyo data yemhando module yakabatanidzwa mumaitiro edu ebasa ndeiyi:
- Kukurumidza kuumbwa kwezvinotarisirwa nevatengi.
- Kuita sarudzo pakuwedzera kushandiswa kwedata.
- Kuwana preliminary set of problem point at the first stages of work for development of agara ari quality controls.
Ehe, aya ndiwo matanho ekutanga mukuvaka yakazara-yakazara data manejimendi maitiro. Asi isu tine chivimbo chekuti chete nekuita basa iri nemaune, nekushingairira kusuma maturusi eDhita rekutonga mukuita basa, isu tinopa vatengi vedu ruzivo rwemukati, danho repamusoro rekuvimba nedata, pachena mukugamuchira kwavo uye kuwedzera kukurumidza kwekutanga. kushanda kutsva.
DataOffice Team
Source: www.habr.com