{"id":77180,"date":"2020-04-08T13:42:32","date_gmt":"2020-04-08T11:42:32","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya"},"modified":"2020-04-08T13:42:32","modified_gmt":"2020-04-08T11:42:32","slug":"testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya","title":{"rendered":"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>P\u00ebrsh\u00ebndetje t\u00eb gjith\u00ebve, un\u00eb quhem Aleksand\u00ebr dhe jam inxhinier i Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave, i cili merret me kontrollin e cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave. N\u00eb k\u00ebt\u00eb artikull do t\u00eb flas p\u00ebr m\u00ebnyr\u00ebn si e arrita k\u00ebt\u00eb nivel dhe pse n\u00eb vitin 2020 kjo drejtim testimi u b\u00eb i njohur. <\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/0691d5a78bcbc8f4f102eafbbef20c0a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Tendenca globale <\/h2>\n<p>\nBota e sotme po kalon nj\u00eb revolucion tjet\u00ebr teknologjik, nj\u00eb nga aspektet e t\u00eb cilit \u00ebsht\u00eb p\u00ebrdorimi nga kompanit\u00eb e t\u00eb dh\u00ebnave t\u00eb grumbulluara p\u00ebr t\u00eb rritur shitjet, fitimet dhe marketingun e tyre. Dukej se pik\u00ebrisht disponimi i t\u00eb dh\u00ebnave t\u00eb mira (cil\u00ebsore), si dhe idet\u00eb e men\u00e7ura q\u00eb mund t\u00eb prodhojn\u00eb para prej tyre (t\u00eb p\u00ebrpunojn\u00eb, t\u00eb vizualizojn\u00eb, t\u00eb nd\u00ebrtojn\u00eb modele t\u00eb m\u00ebsimit t\u00eb makinerive etj.), jan\u00eb \u00e7el\u00ebsi i suksesit p\u00ebr shum\u00eb kompani sot. N\u00ebse 15-20 vjet m\u00eb par\u00eb, puna me grumbullimin dhe monetizimin e t\u00eb dh\u00ebnave ishin kryesisht n\u00eb duar t\u00eb kompanive t\u00eb m\u00ebdha, sot kjo \u00ebsht\u00eb nj\u00eb domosdoshm\u00ebri p\u00ebr \u00e7do kompani q\u00eb mendon p\u00ebr t\u00eb ardhmen. <\/p>\n<p>P\u00ebr k\u00ebt\u00eb arsye, disa vite m\u00eb par\u00eb, t\u00eb gjitha portalet q\u00eb merren me k\u00ebrkimin e pun\u00ebs n\u00eb t\u00eb gjith\u00eb bot\u00ebn filluan t\u00eb mbushen me pozita p\u00ebr Data Scientists, pasi t\u00eb gjith\u00eb ishin t\u00eb sigurt se duke pun\u00ebsuar nj\u00eb specialist t\u00eb till\u00eb mund t\u00eb nd\u00ebrtojn\u00eb nj\u00eb supermodel t\u00eb m\u00ebsimit t\u00eb makinerive, t\u00eb parashikojn\u00eb t\u00eb ardhmen dhe t\u00eb b\u00ebjn\u00eb nj\u00eb \u00abk\u00ebrkes\u00eb kuantike\u00bb p\u00ebr kompanin\u00eb. Me kalimin e koh\u00ebs, njer\u00ebzit kuptuan se ky qasje nuk funksiononte n\u00eb shumic\u00ebn e rasteve, pasi jo t\u00eb gjith\u00eb t\u00eb dh\u00ebnat q\u00eb vijn\u00eb n\u00eb duar t\u00eb specialist\u00ebve t\u00eb till\u00eb jan\u00eb t\u00eb p\u00ebrshtatshme p\u00ebr trajnim modelesh. <\/p>\n<p>Dhe filluan k\u00ebrkesat nga Data Scientists: \u00abLe t\u00eb blejm\u00eb m\u00eb shum\u00eb t\u00eb dh\u00ebna nga k\u00ebta dhe ata.\u2026\u00bb, \u00abNa mungojn\u00eb t\u00eb dh\u00ebnat\u2026\u00bb, \u00abNa nevojiten edhe pak t\u00eb dh\u00ebna dhe preferohet q\u00eb t\u00eb jen\u00eb t\u00eb cil\u00ebsis\u00eb s\u00eb lart\u00eb\u2026\u00bb. Bazuar n\u00eb k\u00ebto k\u00ebrkesa, filluan t\u00eb nd\u00ebrtohen nd\u00ebrveprime t\u00eb shumta mes kompanive q\u00eb zot\u00ebronin nj\u00eb grup t\u00eb caktuar t\u00eb dh\u00ebnash. Natyrisht, kjo k\u00ebrkonte organizimin teknik t\u00eb k\u00ebtij procesi \u2014 lidhja me burimin e t\u00eb dh\u00ebnave, shkarkimi i tyre, kontrollimi q\u00eb ato jan\u00eb ngarkuar n\u00eb mas\u00eb t\u00eb plot\u00eb, etj. Numri i k\u00ebtyre proceseve filloi t\u00eb rritet, dhe deri m\u00eb sot kemi arritur n\u00eb nj\u00eb nevoj\u00eb t\u00eb madhe p\u00ebr nj\u00eb lloj tjet\u00ebr specialist\u00ebsh \u2014 inxhinier\u00ebve t\u00eb Cil\u00ebsis\u00eb s\u00eb T\u00eb Dh\u00ebnave \u2014 ata q\u00eb do t\u00eb monitoronin fluksin e t\u00eb dh\u00ebnave n\u00eb sistem (data pipelines), p\u00ebr cil\u00ebsin\u00eb e t\u00eb dh\u00ebnave n\u00eb hyrje dhe dalje, duke nxjerr\u00eb p\u00ebrfundime mbi mjaftueshm\u00ebrin\u00eb, integritetin dhe karakteristika t\u00eb tjera t\u00eb tyre. <\/p>\n<p>Trendi i inxhinier\u00ebve t\u00eb Data Quality erdhi nga SHBA, ku n\u00eb mesin e epok\u00ebs q\u00eb ka trazuar kapitalizmin, askush nuk \u00ebsht\u00eb i gatsh\u00ebm t\u00eb humbas\u00eb betej\u00ebn p\u00ebr t\u00eb dh\u00ebnat. M\u00eb posht\u00eb kam paraqitur screenshot-e nga dy nga faqet m\u00eb t\u00eb njohura t\u00eb k\u00ebrkimit t\u00eb pun\u00ebs n\u00eb SHBA: <noindex><a rel=\"nofollow\" href=\"https:\/\/www.monster.com\">www.monster.com<\/a><\/noindex> dhe <noindex><a rel=\"nofollow\" href=\"https:\/\/www.dice.com\">www.dice.com<\/a><\/noindex> \u2014 ku jan\u00eb shfaqur t\u00eb dh\u00ebnat e dat\u00ebs 17 mars 2020 n\u00eb lidhje me numrin e vendeve t\u00eb pun\u00ebs t\u00eb postuara, sipas fjal\u00ebve ky\u00e7e: Data Quality dhe Data Scientist. <\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/www.monster.com\">www.monster.com<\/a><\/noindex><\/p>\n<p>Data Scientists \u2013 21416 vende pune<br \/>\nData Quality \u2013 41104 vende pune<\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/8578f2d1e600791e166ed6e8a7068015.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/3ff69acc3c7e0d649eae0808b1b535be.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/www.dice.com\">www.dice.com<\/a><\/noindex> <\/p>\n<p>Data Scientists \u2013 404 vende pune<br \/>\nData Quality \u2013 2020 vende pune<\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/b35a9b423ba4bef8736659580e3c281e.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/2c29b83bbe2361089de5a0ab65017300.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\n\u00cbsht\u00eb e qart\u00eb se k\u00ebto profesione n\u00eb asnj\u00eb m\u00ebnyr\u00eb nuk konkurrojn\u00eb mes tyre. Me screenshot-et thjesht kam dashur t\u00eb ilustroj situat\u00ebn aktuale n\u00eb tregun e pun\u00ebs n\u00eb lidhje me k\u00ebrkesat p\u00ebr inxhinier\u00eb t\u00eb Data Quality, t\u00eb cil\u00ebt tani k\u00ebrkohen shum\u00eb m\u00eb tep\u00ebr sesa Data Scientists. <\/p>\n<p>N\u00eb qershor 2019, EPAM, duke u p\u00ebrgjigjur nevojave t\u00eb tregut modern t\u00eb IT-s\u00eb, ndau fush\u00ebn e Data Quality n\u00eb nj\u00eb praktik\u00eb t\u00eb ve\u00e7ant\u00eb. Inxhinier\u00ebt e Data Quality gjat\u00eb pun\u00ebs s\u00eb tyre t\u00eb p\u00ebrditshme menaxhojn\u00eb t\u00eb dh\u00ebnat, kontrollojn\u00eb sjelljen e tyre n\u00eb kushte dhe sisteme t\u00eb reja, monitorojn\u00eb relevanc\u00ebn e t\u00eb dh\u00ebnave, mjaftueshm\u00ebrin\u00eb dhe aktualitetin e tyre. Megjithat\u00eb, n\u00eb praktik\u00eb, inxhinier\u00ebt e Data Quality n\u00eb t\u00eb v\u00ebrtet\u00eb i kushtojn\u00eb pak koh\u00eb testimit funksional klasik, <u>POR<\/u> kjo varet shum\u00eb nga projekti (shembuj do t\u00eb jap m\u00eb pas). <\/p>\n<p>Detyrat e inxhinierit t\u00eb Data Quality nuk kufizohen vet\u00ebm n\u00eb kontrollet manuale\/automatik\u00eb rutin\u00eb p\u00ebr \"nulls, count dhe sums\" n\u00eb tabelat e DB, por k\u00ebrkojn\u00eb nj\u00eb kuptim t\u00eb thell\u00eb t\u00eb nevojave t\u00eb biznesit t\u00eb klientit dhe, p\u00ebr pasoj\u00eb, aft\u00ebsin\u00eb p\u00ebr t\u00eb transformuar t\u00eb dh\u00ebnat ekzistuese n\u00eb informacion t\u00eb dobish\u00ebm p\u00ebr biznesin. <\/p>\n<h2>Teoria e Data Quality<\/h2>\n<p>\n<img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/d920a7a0435d6f3a1eb84fc16221c742.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>P\u00ebr t\u00eb kuptuar m\u00eb s\u00eb miri rolin e nj\u00eb inxhinieri t\u00eb till\u00eb, le t\u00eb shqyrtojm\u00eb se \u00e7far\u00eb \u00ebsht\u00eb Data Quality n\u00eb teori. <\/p>\n<p><b>Data Quality<\/b> \u2014 \u00ebsht\u00eb nj\u00eb nga fazat e Data Management (nj\u00eb bot\u00eb e t\u00ebr\u00eb, t\u00eb cil\u00ebn do ta l\u00ebm\u00eb p\u00ebr studim t\u00eb pavarur) dhe \u00ebsht\u00eb p\u00ebrgjegj\u00ebse p\u00ebr analizimin e t\u00eb dh\u00ebnave sipas kritereve t\u00eb m\u00ebposhtme: <\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/8f5e75bad69e3cb589cc0e1104ad946f.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nMendoj se nuk ka nevoj\u00eb t\u00eb shpjegohet \u00e7do pik\u00eb (teorikisht quhen \u00abdimensionet e t\u00eb dh\u00ebnave\u00bb), ato jan\u00eb p\u00ebrshkruar mir\u00eb n\u00eb imazh. Por vet\u00eb procesi i testimit nuk n\u00ebnkupton kopjimin e rrept\u00eb t\u00eb k\u00ebtyre ve\u00e7orive n\u00eb rastet e testimit dhe verifikimin e tyre. N\u00eb Cil\u00ebsin\u00eb e t\u00eb Dh\u00ebnave, ashtu si n\u00eb \u00e7do form\u00eb tjet\u00ebr testimi, \u00ebsht\u00eb thelb\u00ebsore t\u00eb merret parasysh k\u00ebrkesat p\u00ebr cil\u00ebsin\u00eb e t\u00eb dh\u00ebnave, t\u00eb dakorduar me pjes\u00ebmarr\u00ebsit e projektit q\u00eb marrin vendime biznesi. <\/p>\n<p><\/p>\n<blockquote><p>N\u00eb var\u00ebsi t\u00eb projektit, inxhinieri i Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave mund t\u00eb realizoj\u00eb funksione t\u00eb ndryshme: nga testuesi i zakonsh\u00ebm i automatizuar me nj\u00eb vler\u00ebsim t\u00eb sip\u00ebrfaqsh\u00ebm t\u00eb cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave, deri te personi q\u00eb kryen profilizimin e thell\u00eb n\u00eb lidhje me ve\u00e7orit\u00eb e p\u00ebrmendura m\u00eb sip\u00ebr. <\/p><\/blockquote>\n<p>Nj\u00eb p\u00ebrshkrim shum\u00eb i detajuar i proceseve t\u00eb Menaxhimit t\u00eb t\u00eb Dh\u00ebnave, Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave dhe t\u00eb af\u00ebrmve \u00ebsht\u00eb p\u00ebrshkruar n\u00eb nj\u00eb lib\u00ebr me titullin <i>\u00abDAMA-DMBOK: Trupi i Njohurive p\u00ebr Menaxhimin e t\u00eb Dh\u00ebnave: Botimi i 2-t\u00eb\u00bb<\/i>. E rekomandoj k\u00ebt\u00eb lib\u00ebr si nj\u00eb hyrje n\u00eb k\u00ebt\u00eb tem\u00eb (m\u00ebnyr\u00ebn p\u00ebr ta gjetur do ta b\u00ebni n\u00eb fund t\u00eb artikullit).<\/p>\n<h2>Historia ime<\/h2>\n<p>\nN\u00eb industrin\u00eb IT kam kaluar nga Junior testues n\u00eb kompanit\u00eb produktive deri n\u00eb Lead Data Quality Engineer n\u00eb kompanin\u00eb EPAM. Pas rreth dy vitesh pune si testues, kisha nj\u00eb bindje t\u00eb fort\u00eb se kisha kryer t\u00eb gjitha llojet e testimeve: regresion, funksional, stres, stabiliteti, siguria, UI, etj. \u2014 dhe kisha provuar shum\u00eb mjete testimi, duke punuar n\u00eb t\u00eb nj\u00ebjtat koh\u00eb me tri gjuh\u00eb programimi: Java, Scala, Python. <\/p>\n<p><\/p>\n<blockquote><p>Duke u kthyer pas, kuptoj pse grupi im i aft\u00ebsive profesionale doli kaq i larmish\u00ebm \u2014 morem pjes\u00eb n\u00eb projekte t\u00eb lidhura me pun\u00ebn me t\u00eb dh\u00ebna, t\u00eb m\u00ebdha dhe t\u00eb vogla. Kjo m\u00eb \u00e7oi n\u00eb bot\u00ebn e shum\u00eb mjeteve dhe mund\u00ebsive p\u00ebr rritje.<\/p><\/blockquote>\n<p>P\u00ebr t\u00eb vler\u00ebsuar shum\u00ebllojshm\u00ebrin\u00eb e mjeteve dhe mund\u00ebsive p\u00ebr t\u00eb marr\u00eb njohuri dhe aft\u00ebsi t\u00eb reja, mjafton t\u00eb shikoni imazhin m\u00eb posht\u00eb, n\u00eb t\u00eb cilin jan\u00eb p\u00ebrshkruar ato m\u00eb popullore n\u00eb bot\u00ebn e \u00abData &amp; AI\u00bb.<\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/e3278694195bf59b44055e1eaafd62b1.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nT\u00eb tilla ilustime p\u00ebrgatitet \u00e7do vit nga nj\u00eb nga kapitalist\u00ebt e njohur t\u00eb riskut, Matt Turck, t\u00eb cilit ka dal\u00eb nga zhvillimi i softuerit. K\u00ebtu <noindex><a rel=\"nofollow\" href=\"https:\/\/mattturck.com\">link<\/a><\/noindex> \u00ebsht\u00eb blogu i tij dhe <noindex><a rel=\"nofollow\" href=\"https:\/\/firstmarkcap.com\/team\/matt-turck\">firma e kapitalit t\u00eb riskut<\/a><\/noindex>, ku ai punon si partner. <\/p>\n<p>Ve\u00e7an\u00ebrisht kam arritur t\u00eb rritem profesionalisht shum\u00eb shpejt, kur isha testuesi i vet\u00ebm n\u00eb projekt, ose, t\u00eb pakt\u00ebn, n\u00eb fillim t\u00eb projektit. N\u00eb at\u00eb moment duhet t\u00eb marr p\u00ebrgjegj\u00ebsin\u00eb p\u00ebr t\u00eb gjith\u00eb procesin e testimit, dhe nuk ke mund\u00ebsin\u00eb t\u00eb t\u00ebrhiqesh, vet\u00ebm p\u00ebrpara. N\u00eb fillim kjo m\u00eb frik\u00ebsonte, megjithat\u00eb tani jan\u00eb t\u00eb qarta t\u00eb gjitha p\u00ebrfitimet e nj\u00eb sfide t\u00eb till\u00eb: <\/p>\n<ul>\n<li>Fillon t\u00eb komunikosh me t\u00eb gjith\u00eb ekipin si kurr\u00eb m\u00eb par\u00eb, sepse nuk ka asnj\u00eb nd\u00ebrmjet\u00ebs p\u00ebr komunikimin: as menaxher testi, as koleg\u00eb testuesish. <\/li>\n<li>Zhytesh n\u00eb projekt n\u00eb m\u00ebnyr\u00eb t\u00eb jasht\u00ebzakonshme t\u00eb thell\u00eb, dhe ke informacion mbi t\u00eb gjitha komponent\u00ebt si n\u00eb p\u00ebrgjith\u00ebsi ashtu edhe n\u00eb detaje. <\/li>\n<li>Zhvilluesit nuk t\u00eb shohin si \"at\u00eb djalin nga testi q\u00eb nuk dihet \u00e7far\u00eb b\u00ebn\", por m\u00eb tep\u00ebr si nj\u00eb t\u00eb barabart\u00eb, i cili sjell nj\u00eb p\u00ebrfitim t\u00eb jasht\u00ebzakonsh\u00ebm p\u00ebr ekipin me testet e tij t\u00eb automatizuara dhe parashikimin e shfaqjes s\u00eb defekteve n\u00eb nj\u00eb nyje specifike t\u00eb produktit. <\/li>\n<li>Si rezultat \u2014 je m\u00eb i efektsh\u00ebm, m\u00eb i kualifikuar, m\u00eb i k\u00ebrkuar. <\/li>\n<\/ul>\n<p>\nMe rritjen e projektit, n\u00eb 100% t\u00eb rasteve un\u00eb b\u00ebhesha mentor p\u00ebr testuesit e rinj q\u00eb vinin n\u00eb t\u00eb, i trajnoja ata dhe i kaloja njohurit\u00eb q\u00eb kisha m\u00ebsuar vet\u00eb. P\u00ebr m\u00eb tep\u00ebr, var\u00ebsisht nga projekti, nuk kam marr\u00eb gjithmon\u00eb nga menaxhmenti specialist\u00eb t\u00eb testimit t\u00eb automatizuar me nivel t\u00eb lart\u00eb dhe ka qen\u00eb e nevojshme ose t'i trajnoj ata p\u00ebr automatizimin (p\u00ebr ata q\u00eb d\u00ebshirojn\u00eb), ose t\u00eb krijoj mjete p\u00ebr t'i p\u00ebrdorur ata n\u00eb aktivitete t\u00eb p\u00ebrditshme (mjete p\u00ebr gjenerimin e t\u00eb dh\u00ebnave dhe ngarkimin e tyre n\u00eb sistem, mjet p\u00ebr kryerjen e testimeve t\u00eb ngarkes\u00ebs \/ testimeve t\u00eb stabilitetit \"me shpejt\u00ebsi\" etj.). <\/p>\n<h3>Shembulli i nj\u00eb projekti konkret<\/h3>\n<p>\nFatkeq\u00ebsisht, p\u00ebr shkak t\u00eb angazhimeve p\u00ebr moszbulim, nuk mund t\u00eb flas n\u00eb detaje p\u00ebr projektet n\u00eb t\u00eb cilat kam punuar, megjithat\u00eb do t\u00eb jap shembuj t\u00eb detyrave tipike p\u00ebr Inxhinierin e Cil\u00ebsis\u00eb s\u00eb T\u00eb Dh\u00ebnave n\u00eb nj\u00eb nga projektet. <\/p>\n<p>Thelbi i projektit \u2014 realizimi i nj\u00eb platforme p\u00ebr p\u00ebrgatitjen e t\u00eb dh\u00ebnave p\u00ebr trajnimin e modeleve t\u00eb m\u00ebsimit t\u00eb makinerive mbi baz\u00ebn e atyre t\u00eb dh\u00ebnave. Klienti ishte nj\u00eb kompani e madhe farmaceutike nga Shtetet e Bashkuara. Teknikisht ishte nj\u00eb klaster <noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes.io\">Kubernetes<\/a><\/noindex>, q\u00eb ngritej n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/ec2\">AWS EC2<\/a><\/noindex> instancave, me disa mikrosh\u00ebrbime dhe me nj\u00eb projekt Open Source n\u00eb thelb nga kompania EPAM \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/legion-platform\/legion\">Legion<\/a><\/noindex>, i adaptuar p\u00ebr nevojat e klientit t\u00eb caktuar ( aktualisht projekti \u00ebsht\u00eb transformuar n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/odahu\/odahu-flow\">odahu<\/a><\/noindex>). Proceset ETL u organizuan p\u00ebrmes <noindex><a rel=\"nofollow\" href=\"https:\/\/airflow.apache.org\">Apache Airflow<\/a><\/noindex> dhe transferuan t\u00eb dh\u00ebnat nga <noindex><a rel=\"nofollow\" href=\"https:\/\/www.salesforce.com\">SalesForce<\/a><\/noindex> sistemi i klientit n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/s3\/?nc1=h_ls\">AWS S3<\/a><\/noindex> Buckets. Pastaj, n\u00eb platform\u00eb u dep\u00ebrtua nj\u00eb imazh Docker i modelit t\u00eb m\u00ebsimit t\u00eb makinerive, i cili u st\u00ebrvit me t\u00eb dh\u00ebna t\u00eb reja dhe p\u00ebrmes nd\u00ebrfaqes REST API ofronte parashikime q\u00eb ishin t\u00eb r\u00ebnd\u00ebsishme p\u00ebr biznesin dhe zgjidhnin probleme t\u00eb ve\u00e7anta. <\/p>\n<p><b>Vizualisht, gjith\u00e7ka dukej m\u00eb pak k\u00ebshtu: <\/b><\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/c2de3c1919f3f9ff29147fce09eb14c4.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nTestimi funksional n\u00eb k\u00ebt\u00eb projekt ishte m\u00eb than\u00eb mjaft i mjaftuesh\u00ebm, dhe duke marr\u00eb parasysh shpejt\u00ebsin\u00eb e zhvillimit t\u00eb tipareve dhe nevoj\u00ebn p\u00ebr t\u00eb ruajtur ritmin e ciklit t\u00eb l\u00ebshimit (sprint-e dyjavore), ishte e nevojshme t\u00eb mendoheshim menj\u00ebher\u00eb p\u00ebr automatizimin e testimit t\u00eb nyjave m\u00eb kritike t\u00eb sistemit. Pjesa m\u00eb e madhe e platform\u00ebs, e cila kishte baz\u00ebn e Kubernetes, ishte e mbuluar me teste automatike t\u00eb realizuara n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/robotframework.org\/\">Robot Framework<\/a><\/noindex> + Python, por ishte gjithashtu e nevojshme t\u00eb ruhej dhe zgjerohej. P\u00ebr m\u00eb tep\u00ebr, p\u00ebr leht\u00ebsin\u00eb e klientit, u krijua nj\u00eb GUI p\u00ebr menaxhimin e modeleve t\u00eb m\u00ebsimit t\u00eb makinerive t\u00eb deponuara n\u00eb klaster, si dhe mund\u00ebsia p\u00ebr t\u00eb specifikuar se nga ku dhe ku duheshin transferuar t\u00eb dh\u00ebnat p\u00ebr st\u00ebrvitjen e modeleve. Ky zgjerim i gjer\u00eb solli me vete zgjerimin e kontrollave funksionale automatike, t\u00eb cilat kryesisht jan\u00eb realizuar p\u00ebrmes thirrjeve REST API dhe nj\u00eb numri t\u00eb vog\u00ebl testesh end-2-end UI. Rreth ekuatorit t\u00eb gjith\u00eb k\u00ebtij procesi, na u bashkua nj\u00eb tester manual, i cili e menaxhoi mjaft mir\u00eb testimin e pranimit t\u00eb versioneve t\u00eb produktit dhe bised\u00ebn me klientin n\u00eb lidhje me pranimin e l\u00ebshimit t\u00eb radh\u00ebs. P\u00ebr m\u00eb tep\u00ebr, me shfaqjen e specialistit t\u00eb ri, mund\u00ebm t\u00eb dokumentojm\u00eb pun\u00ebn ton\u00eb dhe t\u00eb shtojm\u00eb disa kontrolle manuale shum\u00eb t\u00eb r\u00ebnd\u00ebsishme, t\u00eb cilat ishte e v\u00ebshtir\u00eb t'i automatizojm\u00eb menj\u00ebher\u00eb. <\/p>\n<p>Dhe fund, pasi arrit\u00ebm stabilitetin nga platforma dhe nd\u00ebrfaqja grafike q\u00eb e mb\u00ebshtet at\u00eb, ne filluam nd\u00ebrtimin e pipeline-ve ETL duke p\u00ebrdorur DAG-t\u00eb e Apache Airflow. Kontrolli automatizuar i cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave u realizua duke shkruar DAG t\u00eb ve\u00e7ant\u00eb Airflow q\u00eb kontrollonin t\u00eb dh\u00ebnat sipas rezultateve t\u00eb procesit ETL. N\u00eb kuad\u00ebr t\u00eb k\u00ebtij projekti, pata fatin, dhe klienti na ofroi akses n\u00eb grupe t\u00eb dh\u00ebnash t\u00eb anonomizuara, n\u00eb t\u00eb cilat ne testoheshim. K\u00ebto t\u00eb dh\u00ebna u kontrolluan rresht p\u00ebr rresht p\u00ebr p\u00ebrputhshm\u00ebrin\u00eb me llojet, pranin\u00eb e t\u00eb dh\u00ebnave t\u00eb prishura, numrin total t\u00eb regjistrimeve para dhe pas, krahasimin e transformimeve t\u00eb kryera nga procesi ETL p\u00ebr agregimin, ndryshimin e emrave t\u00eb kolonave dhe t\u00eb tjera. P\u00ebr m\u00eb tep\u00ebr, k\u00ebto kontrollime u zgjeruan n\u00eb burime t\u00eb ndryshme t\u00eb t\u00eb dh\u00ebnave, p\u00ebr shembull p\u00ebrve\u00e7 SalesForce, gjithashtu edhe n\u00eb MySQL. <\/p>\n<p>Kontrollimet e cil\u00ebsis\u00eb s\u00eb fundit t\u00eb t\u00eb dh\u00ebnave u realizuan tashm\u00eb n\u00eb nivelin S3, ku ato ruheshin dhe ishin n\u00eb gjendje ready-to-use p\u00ebr trajnimin e modeleve t\u00eb m\u00ebsimit t\u00eb makin\u00ebs. P\u00ebr t\u00eb marr\u00eb t\u00eb dh\u00ebnat nga skedari p\u00ebrfundimtar CSV, q\u00eb ndodhej n\u00eb S3 Bucket dhe p\u00ebr validimin e tyre, u shkrua nj\u00eb kod duke p\u00ebrdorur <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/sdk-for-python\">klientin boto3<\/a><\/noindex>. <\/p>\n<p>Gjithashtu, nga ana e klientit kishte nj\u00eb k\u00ebrkes\u00eb p\u00ebr ruajtjen e nj\u00eb pjese t\u00eb t\u00eb dh\u00ebnave n\u00eb nj\u00eb S3 Bucket, nj\u00eb pjese n\u00eb nj\u00eb tjet\u00ebr. P\u00ebr k\u00ebt\u00eb gjithashtu ishte e nevojshme t\u00eb shkruheshin kontrollime shtes\u00eb, q\u00eb kontrollojn\u00eb besueshm\u00ebrin\u00eb e k\u00ebsaj renditjeje. <\/p>\n<h3>P\u00ebrvoja e p\u00ebrgjithshme nga projekte t\u00eb tjera<\/h3>\n<p>\nShembulli i list\u00ebs m\u00eb t\u00eb gjer\u00eb t\u00eb aktiviteteve t\u00eb inxhinierit t\u00eb Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave: <\/p>\n<ul>\n<li>T\u00eb p\u00ebrgatis\u00eb t\u00eb dh\u00ebna testuese (t\u00eb vlefshme, t\u00eb pavlefshme, t\u00eb m\u00ebdha, t\u00eb vogla) p\u00ebrmes nj\u00eb instrumenti automatizuar. <\/li>\n<li>T\u00eb ngarkoj\u00eb grupin e p\u00ebrgatitur t\u00eb t\u00eb dh\u00ebnave n\u00eb burimin origjinal dhe t\u00eb verifikoj\u00eb gatishm\u00ebrin\u00eb e tij p\u00ebr p\u00ebrdorim. <\/li>\n<li>T\u00eb aktivizoj\u00eb proceset ETL p\u00ebr trajtimin e grupit t\u00eb t\u00eb dh\u00ebnave nga depoja origjinale n\u00eb at\u00eb p\u00ebrfundimtare apo nd\u00ebrmjet nj\u00eb seti t\u00eb caktuar t\u00eb parametrave (n\u00eb rast se \u00ebsht\u00eb e mundur t\u00eb jepen parametra konfigurohesh\u00ebm p\u00ebr detyr\u00ebn ETL). <\/li>\n<li>T\u00eb verifikoj\u00eb t\u00eb dh\u00ebnat e p\u00ebrpunuara nga procesi ETL p\u00ebr cil\u00ebsin\u00eb e tyre dhe p\u00ebrputhshm\u00ebrin\u00eb me k\u00ebrkesat e biznesit. <\/li>\n<\/ul>\n<p>\nMegjithat\u00eb, theksi kryesor i kontrolleve duhet t\u00eb bie jo vet\u00ebm n\u00eb at\u00eb q\u00eb fluksi i t\u00eb dh\u00ebnave n\u00eb sistem ka funksionuar n\u00eb p\u00ebrgjith\u00ebsi dhe ka arritur n\u00eb fund (\u00e7ka \u00ebsht\u00eb pjes\u00eb e testimit funksional), por p\u00ebr m\u00eb shum\u00eb n\u00eb kontrollin dhe validimin e t\u00eb dh\u00ebnave p\u00ebr t'iu p\u00ebrmbajtur k\u00ebrkesave t\u00eb pritur, identifikimin e anomalive dhe t\u00eb tjera. <\/p>\n<h3>Mjetet<\/h3>\n<p>\nNj\u00eb nga teknikat e tilla t\u00eb kontrollit t\u00eb t\u00eb dh\u00ebnave mund t\u00eb jet\u00eb organizimi i kontrollit t\u00eb zinxhir\u00ebve n\u00eb \u00e7do faz\u00eb t\u00eb p\u00ebrpunimit t\u00eb t\u00eb dh\u00ebnave, i njohur n\u00eb literatur\u00eb si 'data chain' \u2014 kontrolli i t\u00eb dh\u00ebnave nga burimi deri n\u00eb pik\u00ebn e p\u00ebrdorimit p\u00ebrfundimtar. Kontrolli i k\u00ebtij lloji realizohet m\u00eb s\u00eb shumti p\u00ebrmes shkruarjes s\u00eb pyetjeve SQL kontrolluese. E kuptueshme, k\u00ebto pyetje duhet t\u00eb jen\u00eb sa m\u00eb t\u00eb lehta dhe kontrollojn\u00eb pjes\u00ebt e ve\u00e7anta t\u00eb cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave (metadata e tabelave, rreshta bosh, NULL, gabime n\u00eb sintaks\u00eb \u2014 atributet e tjera t\u00eb k\u00ebrkuara p\u00ebr kontroll). <\/p>\n<p>N\u00eb rastin e testimit regresiv, n\u00eb t\u00eb cilin p\u00ebrdoren tashm\u00eb grupe t\u00eb dh\u00ebnash t\u00eb gatshme (t\u00eb pal\u00ebvizshme ose t\u00eb ndryshueshme pak), n\u00eb kodin e testave automatike mund t\u00eb ruhen tashm\u00eb modele t\u00eb gatshme t\u00eb kontrollit t\u00eb t\u00eb dh\u00ebnave p\u00ebr p\u00ebrputhshm\u00ebri me cil\u00ebsin\u00eb (p\u00ebrshkrimet e metadatave t\u00eb pritura t\u00eb tabelave; objektet p\u00ebrzgjedh\u00ebse t\u00eb rast\u00ebsishme q\u00eb mund t\u00eb p\u00ebrzgjidhen gjat\u00eb testit dhe t\u00eb tjera). <\/p>\n<p>Po ashtu, gjat\u00eb testimit \u00ebsht\u00eb e nevojshme t\u00eb shkruhen procese testuese ETL, p\u00ebrmes strukturave t\u00eb tilla si Apache Airflow, <noindex><a rel=\"nofollow\" href=\"https:\/\/spark.apache.org\/\">Apache Spark<\/a><\/noindex> ose plot\u00ebsisht mjete cloud black-box si <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/dataprep\">GCP Dataprep<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/dataflow\">GCP Dataflow<\/a><\/noindex> dhe t\u00eb tjera. Ky rrethan\u00eb e detyron inxhinierin e testimit t\u00eb thellohet n\u00eb parimet e funksionimit t\u00eb mjeteve t\u00eb p\u00ebrmendura dhe t\u00eb realizoj\u00eb m\u00eb efektivisht si testimin funksional (p\u00ebr shembull, t\u00eb proceseve ETL ekzistuese n\u00eb projekt), ashtu edhe t'i p\u00ebrdor\u00eb ato p\u00ebr kontrollin e t\u00eb dh\u00ebnave. N\u00eb ve\u00e7anti, p\u00ebr Apache Airflow ekzistojn\u00eb tashm\u00eb operator\u00eb t\u00eb gatsh\u00ebm p\u00ebr pun\u00eb me bazat e t\u00eb dh\u00ebnave analitike popullore si\u00e7 \u00ebsht\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery\">GCP BigQuery<\/a><\/noindex>. Nj\u00eb shembull themelor i p\u00ebrdorimit t\u00eb tij tashm\u00eb \u00ebsht\u00eb paraqitur, <noindex><a rel=\"nofollow\" href=\"https:\/\/medium.com\/@yuu.ishikawa\/apache-airflow-as-a-data-quality-checker-416ca7f5a3ad\">k\u00ebtu<\/a><\/noindex>prandaj nuk do t\u00eb p\u00ebrs\u00ebris. <\/p>\n<p>P\u00ebrve\u00e7 zgjidhjeve t\u00eb gatshme, askush nuk e ndalon q\u00eb t\u00eb realizoni teknikat dhe mjetet tuaja. Kjo do t\u00eb jet\u00eb e dobishme p\u00ebr projektin, por edhe p\u00ebr vet\u00eb Inxhinierin e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave, i cili k\u00ebshtu do t\u00eb p\u00ebrmir\u00ebsoj\u00eb horizontin e tij teknik dhe aft\u00ebsit\u00eb e kodimit. <\/p>\n<h3>Si funksionon kjo n\u00eb nj\u00eb projekt real<\/h3>\n<p>\nNj\u00eb ilust\u00ebrim i mir\u00eb i paragraf\u00ebve t\u00eb fundit rreth 'data chain', ETL dhe kontrollit t\u00eb gjithansh\u00ebm \u00ebsht\u00eb procesi i m\u00ebposht\u00ebm nga nj\u00eb prej projekteve reale: <\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/4a4cd3043b584bb3fbce4b9c3021e9f6.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nK\u00ebtu n\u00eb 'funnel'-in e sistemit ton\u00eb hyjn\u00eb t\u00eb dh\u00ebna t\u00eb ndryshme (natyrisht, t\u00eb p\u00ebrgatitura nga ne): t\u00eb sakta, t\u00eb pasakta, t\u00eb kombinuara etj., pastaj ato filtrohen dhe kalojn\u00eb n\u00eb nj\u00eb depo nd\u00ebrmjet\u00ebse, m\u00eb pas i pret nj\u00eb seri e re transformimesh dhe vendosja n\u00eb depozita p\u00ebrfundimtare, nga e cila, p\u00ebr pasoj\u00eb, do t\u00eb kryhet analiza, nd\u00ebrtimi i paneleve t\u00eb dh\u00ebnash dhe k\u00ebrkimi i njohurive t\u00eb biznesit. N\u00eb nj\u00eb sistem t\u00eb till\u00eb, pa kontrolluar funksionalitetin e proceseve ETL, ne fokusohemi n\u00eb cil\u00ebsin\u00eb e t\u00eb dh\u00ebnave para dhe pas transformimeve, si dhe n\u00eb daljen n\u00eb analiz\u00eb. <\/p>\n<p>P\u00ebrmbledhja e asaj q\u00eb u tha m\u00eb sip\u00ebr, pavar\u00ebsisht nga vendet ku kam punuar, kam qen\u00eb gjithmon\u00eb i p\u00ebrfshir\u00eb n\u00eb projekte Data, t\u00eb cilat kishin k\u00ebto karakteristika: <\/p>\n<ul>\n<li>Vet\u00ebm p\u00ebrmes automatizimit mund t\u00eb kontrollohen disa raste dhe t\u00eb arrihet nj\u00eb cik\u00ebl l\u00ebshimi i pranuesh\u00ebm p\u00ebr biznesin. <\/li>\n<li>Testuesi n\u00eb nj\u00eb projekt t\u00eb till\u00eb \u00ebsht\u00eb nj\u00eb nga an\u00ebtar\u00ebt m\u00eb t\u00eb respektuar t\u00eb ekipit, pasi sjell nj\u00eb dobi t\u00eb madhe p\u00ebr t\u00eb gjith\u00eb pjes\u00ebtar\u00ebt (shpejtimi i testimeve, t\u00eb dh\u00ebna t\u00eb mira p\u00ebr Data Scientist, identifikimi i defekteve n\u00eb fazat e hershme). <\/li>\n<li>Nuk ka r\u00ebnd\u00ebsi n\u00eb pajisjet tuaja punoni apo n\u00eb re \u2014 t\u00eb gjitha burimet jan\u00eb t\u00eb abstractuara n\u00eb nj\u00eb klas\u00eb si Hortonworks, Cloudera, Mesos, Kubernetes etj. <\/li>\n<li>Projektet nd\u00ebrtohen mbi nj\u00eb qasje mikrosh\u00ebrbimi, dominuese jan\u00eb llogaritjet e shp\u00ebrndara dhe paralel. <\/li>\n<\/ul>\n<p>\nDua t\u00eb theksoj se, duke u marr\u00eb me testimin n\u00eb fush\u00ebn e Cil\u00ebsis\u00eb s\u00eb T\u00eb Dh\u00ebnave, specialisti i testimit e zhvendos profesionin e tij n\u00eb kodin e produktit dhe mjetet e p\u00ebrdorura. <\/p>\n<h3>Karakteristikat dalluese t\u00eb testimit t\u00eb Cil\u00ebsis\u00eb s\u00eb T\u00eb Dh\u00ebnave<\/h3>\n<p>\nP\u00ebrve\u00e7 k\u00ebsaj, p\u00ebr veten time, kam identifikuar karakateristika t\u00eb tjera (dhe dua t\u00eb theksoj SHUM\u00cb t\u00eb p\u00ebrgjith\u00ebsuara dhe ekskluzivisht subjektive) dalluese t\u00eb testimit n\u00eb projektet Data (Big Data) dhe drejtime t\u00eb tjera: <\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/8df52fbae2998e70755bf84e254db83e.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<h2>Useful links<\/h2>\n<p><\/p>\n<ol>\n<li>Teoria: <noindex><a rel=\"nofollow\" href=\"https:\/\/www.amazon.com\/DAMA-DMBOK-Data-Management-Body-Knowledge\/dp\/1634622340\">DAMA-DMBOK: Data Management Body of Knowledge: Botimi i 2-t\u00eb<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/training.ru\/#!\/Home?lang=ru\">Qendra e Trajnimit<\/a><\/noindex>&nbsp;EPAM&nbsp;<\/li>\n<li>Materialet e rekomanduara p\u00ebr inxhinierin e ri t\u00eb Cil\u00ebsis\u00eb s\u00eb T\u00eb Dh\u00ebnave:\n<ol>\n<li>Kursi falas n\u00eb Stepik:&nbsp;<noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/551\/syllabus\">Hyrje n\u00eb bazat e t\u00eb dh\u00ebnave<\/a><\/noindex>.&nbsp;<\/li>\n<li>Kursi n\u00eb LinkedIn Learning:&nbsp;<noindex><a rel=\"nofollow\" href=\"https:\/\/www.linkedin.com\/learning\/data-science-foundations-data-engineering\/welcome\">Themela t\u00eb Shkenc\u00ebs s\u00eb T\u00eb Dh\u00ebnave: Inxhinieria e T\u00eb Dh\u00ebnave<\/a><\/noindex>.<\/li>\n<li>Artikuj:\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.stickyminds.com\/article\/business-intelligence-and-data-quality\">Inteligjenca e Biznesit dhe Cil\u00ebsia e T\u00eb Dh\u00ebnave<\/a><\/noindex>;&nbsp;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.datagaps.com\/concepts\/etl-testing\">Testimi ETL<\/a><\/noindex>;&nbsp;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/panoply.io\/analytics-stack-guide\/data-profiling-best-practices\">\u00c7far\u00eb \u00ebsht\u00eb Profilimi i T\u00eb Dh\u00ebnave? Procesi, Praktikat M\u00eb t\u00eb Mira dhe Mjetet<\/a><\/noindex>;&nbsp;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.guru99.com\/software-testing-test-data.html\">Generimi i T\u00eb Dh\u00ebnave t\u00eb Testimit: \u00c7far\u00eb \u00ebsht\u00eb, Si t\u00eb, Shembuj, Mjete;&nbsp;<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.testbytes.net\/blog\/5-test-data-generation-techniques-to-know\">5 Teknikat e Generimit t\u00eb T\u00eb Dh\u00ebnave t\u00eb Testimit q\u00eb Duhet t\u00eb Dini.<\/a><\/noindex>&nbsp;<\/li>\n<\/ul>\n<\/li>\n<li>Video:\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=HWaBdqmTqEA\">Koncepat e Cil\u00ebsis\u00eb s\u00eb T\u00eb Dh\u00ebnave | Tutorial p\u00ebr Cil\u00ebsin\u00eb e T\u00eb Dh\u00ebnave | Tutorial p\u00ebr Depozita t\u00eb Dh\u00ebnash<\/a><\/noindex>;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=avDqzhuHFkw\">Profilimi i T\u00eb Dh\u00ebnave duke p\u00ebrdorur SSIS<\/a><\/noindex>;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=r7SF5WldITk\">Profilimi i T\u00eb Dh\u00ebnave dhe Procesimi i Pipeline me Spark.<\/a><\/noindex><\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p><\/p>\n<h2>P\u00ebrfundim<\/h2>\n<p>\n<b>Data Quality<\/b> \u2014 kjo \u00ebsht\u00eb nj\u00eb drejtim shum\u00eb i ri dhe perspektiv\u00eb, t\u00eb jesh pjes\u00eb e t\u00eb cilit do t\u00eb thot\u00eb t\u00eb jesh pjes\u00eb e nj\u00eb startup-i. Duke u futur n\u00eb Data Quality, do t\u00eb zhyt\u00ebsh n\u00eb nj\u00eb s\u00ebr\u00eb t\u00eb madhe teknologjish moderne dhe t\u00eb k\u00ebrkuara, por m\u00eb e r\u00ebnd\u00ebsishmja \u2014 do t\u00eb hapen para teje mund\u00ebsi t\u00eb m\u00ebdha p\u00ebr t\u00eb gjeneruar dhe realizuar idet\u00eb tuaja. Do t\u00eb mund t\u00eb aplikosh qasjen e p\u00ebrmir\u00ebsimit t\u00eb vazhduesh\u00ebm jo vet\u00ebm n\u00eb projekt, por edhe p\u00ebr veten, duke u zhvilluar vazhdimisht si specialist.<br \/>\n<br \/>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/epam_systems\/blog\/495478\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u0438 \u044f Data Quality \u0438\u043d\u0436\u0435\u043d\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 \u043d\u0430 \u043f\u0440\u0435\u0434\u043c\u0435\u0442 \u0438\u0445 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0430. \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u0440\u0435\u0447\u044c \u043f\u043e\u0439\u0434\u0451\u0442 \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u044f \u043a \u044d\u0442\u043e\u043c\u0443 \u043f\u0440\u0438\u0448\u0451\u043b \u0438 \u043f\u043e\u0447\u0435\u043c\u0443 \u0432 2020 \u0433\u043e\u0434\u0443 \u044d\u0442\u043e \u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043e\u043a\u0430\u0437\u0430\u043b\u043e\u0441\u044c \u043d\u0430 \u0433\u0440\u0435\u0431\u043d\u0435 \u0432\u043e\u043b\u043d\u044b. \u041c\u0438\u0440\u043e\u0432\u043e\u0439 \u0442\u0440\u0435\u043d\u0434 \u0421\u0435\u0433\u043e\u0434\u043d\u044f\u0448\u043d\u0438\u0439 \u043c\u0438\u0440 \u043f\u0435\u0440\u0435\u0436\u0438\u0432\u0430\u0435\u0442 \u043e\u0447\u0435\u0440\u0435\u0434\u043d\u0443\u044e \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0443\u044e \u0440\u0435\u0432\u043e\u043b\u044e\u0446\u0438\u044e, \u043e\u0434\u043d\u0438\u043c \u0438\u0437 \u0430\u0441\u043f\u0435\u043a\u0442\u043e\u0432 \u043a\u043e\u0442\u043e\u0440\u043e\u0439 \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":77181,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-77180","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u0438 \u044f Data Quality \u0438\u043d\u0436\u0435\u043d\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 \u043d\u0430 \u043f\u0440\u0435\u0434\u043c\u0435\u0442 \u0438\u0445 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0430.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0422\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0449\u0438\u043a \u0431\u043e\u043b\u044c\u0448\u0438\u0445 \u0438 \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u0445 \u0434\u0430\u043d\u043d\u044b\u0445: \u0442\u0440\u0435\u043d\u0434\u044b, \u0442\u0435\u043e\u0440\u0438\u044f, \u043c\u043e\u044f \u0438\u0441\u0442\u043e\u0440\u0438\u044f | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u0438 \u044f Data Quality \u0438\u043d\u0436\u0435\u043d\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 \u043d\u0430 \u043f\u0440\u0435\u0434\u043c\u0435\u0442 \u0438\u0445 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0430.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-04-08T11:42:32+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-04-08T11:42:32+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trende, teori, historia ime | ProHoster","description":"P\u00ebrsh\u00ebndetje t\u00eb gjith\u00ebve, emri im \u00ebsht\u00eb Aleksand\u00ebr, dhe un\u00eb jam inxhinier i Data Quality, i cili merret me verifikimin e t\u00eb dh\u00ebnave p\u00ebr cil\u00ebsin\u00eb e tyre.","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0422\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0449\u0438\u043a \u0431\u043e\u043b\u044c\u0448\u0438\u0445 \u0438 \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u0445 \u0434\u0430\u043d\u043d\u044b\u0445: \u0442\u0440\u0435\u043d\u0434\u044b, \u0442\u0435\u043e\u0440\u0438\u044f, \u043c\u043e\u044f \u0438\u0441\u0442\u043e\u0440\u0438\u044f | ProHoster","og:description":"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u0438 \u044f Data Quality \u0438\u043d\u0436\u0435\u043d\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 \u043d\u0430 \u043f\u0440\u0435\u0434\u043c\u0435\u0442 \u0438\u0445 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0430.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-04-08T11:42:32+00:00","article:modified_time":"2020-04-08T11:42:32+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"77180","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 17:22:32","updated":"2022-10-03 00:36:00","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/77180","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=77180"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/77180\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/77181"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=77180"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=77180"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=77180"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}