{"id":77180,"date":"2020-04-08T13:42:32","date_gmt":"2020-04-08T11:42:32","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya"},"modified":"2020-04-08T13:42:32","modified_gmt":"2020-04-08T11:42:32","slug":"testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya","title":{"rendered":"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>P\u00ebrsh\u00ebndetje t\u00eb gjith\u00ebve, un\u00eb quhem Aleksand\u00ebr dhe jam inxhinier i Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave, i angazhuar n\u00eb verifikimin e cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave. Ky artikull do t\u00eb trajtoj\u00eb se si kam arritur n\u00eb k\u00ebt\u00eb pozit\u00eb dhe pse n\u00eb vitin 2020 ky drejtim i testimit u b\u00eb shum\u00eb i r\u00ebnd\u00ebsish\u00ebm. <\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/0691d5a78bcbc8f4f102eafbbef20c0a.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Tendenca Bot\u00ebrore <\/h2>\n<p>\nBota e sotme po p\u00ebrjeton nj\u00eb revolucion teknologjik, nj\u00eb nga aspektet e t\u00eb cilit \u00ebsht\u00eb p\u00ebrdorimi i t\u00eb dh\u00ebnave t\u00eb grumbulluara nga kompani t\u00eb ndryshme p\u00ebr t\u00eb rritur volumet e shitjeve, fitimet dhe marketingun. K\u00ebtu vjen n\u00eb pah r\u00ebnd\u00ebsia e t\u00eb dh\u00ebnave t\u00eb mira (cil\u00ebsore) dhe e mendjeve t\u00eb zgjidhura q\u00eb mund t\u00eb nxjerrin para nga ato (t\u00eb p\u00ebrpunojn\u00eb, vizualizojn\u00eb, ndjekin modele t\u00eb m\u00ebsimit t\u00eb makineris\u00eb etj.), q\u00eb sot jan\u00eb \u00e7el\u00ebsi i suksesit p\u00ebr shum\u00eb kompani. Nd\u00ebrsa 15-20 vjet m\u00eb par\u00eb, vet\u00ebm kompanit\u00eb e m\u00ebdha merreshin me grumbullimin dhe shp\u00ebrndarjen e t\u00eb dh\u00ebnave, tani ky detyr\u00eb \u00ebsht\u00eb p\u00ebr praktikisht \u00e7do kompani q\u00eb mendon me arsyen. <\/p>\n<p>P\u00ebr k\u00ebt\u00eb arsye, disa vjet m\u00eb par\u00eb, t\u00eb gjitha portalet globale t\u00eb pun\u00ebs u mbush\u00ebn me oferta p\u00ebr Data Scientists, pasi t\u00eb gjith\u00eb ishin t\u00eb bindur se duke marr\u00eb nj\u00eb specialist t\u00eb till\u00eb n\u00eb ekip, mund t\u00eb nd\u00ebrtohej nj\u00eb supermodel m\u00ebsimi t\u00eb makineris\u00eb, t\u00eb parashikohej e ardhmja dhe t\u00eb b\u00ebhej nj\u00eb \"k\u00ebrcim kuantik\" p\u00ebr kompanin\u00eb. Me kalimin e koh\u00ebs, njer\u00ebzit kuptuan se ky qasje nuk funksiononte asnj\u00ebher\u00eb, pasi jo t\u00eb gjith\u00eb t\u00eb dh\u00ebnat q\u00eb p\u00ebrfundonin n\u00eb duar t\u00eb k\u00ebtyre specialist\u00ebve ishin t\u00eb dobishme p\u00ebr m\u00ebsimin e modeleve. <\/p>\n<p>Dhe filluan k\u00ebrkesat nga Data Scientists: \"Le t\u00eb blejm\u00eb m\u00eb shum\u00eb t\u00eb dh\u00ebna nga k\u00ebta dhe ata...\", \"Na mungojn\u00eb t\u00eb dh\u00ebnat...\", \"Na nevojiten edhe disa t\u00eb dh\u00ebna dhe t\u00eb jen\u00eb t\u00eb mira...\". Duke u bazuar n\u00eb k\u00ebto k\u00ebrkesa, filluan t\u00eb krijoheshin shum\u00eb nd\u00ebrveprime mes kompanive q\u00eb zot\u00ebronin nj\u00eb set t\u00eb dh\u00ebnash. Sigurisht, kjo k\u00ebrkonte nj\u00eb organizim teknik t\u00eb k\u00ebtij procesi - t\u00eb lidhej me burimin e t\u00eb dh\u00ebnave, t\u00eb shkarkoheshin ato, t\u00eb kontrollohej q\u00eb ishin ngarkuar n\u00eb mas\u00ebn e plot\u00eb etj. Numri i k\u00ebtyre proceseve filloi t\u00eb rritet, dhe n\u00eb dit\u00ebt e sotme kemi nj\u00eb nevoj\u00eb t\u00eb madhe p\u00ebr nj\u00eb lloj tjet\u00ebr specialist\u00ebsh - inxhinier\u00ebt e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave - ata q\u00eb do t\u00eb mbik\u00ebqyrnin rrjedh\u00ebn e t\u00eb dh\u00ebnave n\u00eb sistem (data pipelines), p\u00ebr cil\u00ebsin\u00eb e t\u00eb dh\u00ebnave n\u00eb hyrje dhe dalje, do t\u00eb nxirrnin p\u00ebrfundime mbi mjaftueshm\u00ebrin\u00eb, integritetin dhe karakteristika t\u00eb tjera t\u00eb tyre. <\/p>\n<p>Tendenca p\u00ebr inxhinier\u00eb t\u00eb Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave erdhi te ne nga SHBA, ku n\u00eb mes t\u00eb er\u00ebs kapitaliste askush nuk \u00ebsht\u00eb i gatsh\u00ebm t\u00eb humbas\u00eb betej\u00ebn p\u00ebr t\u00eb dh\u00ebnat. M\u00eb posht\u00eb kam paraqitur kapje ekranesh nga dy nga faqet e pun\u00ebs m\u00eb t\u00eb njohura n\u00eb SHBA: <noindex><a rel=\"nofollow\" href=\"https:\/\/www.monster.com\">www.monster.com<\/a><\/noindex> dhe <noindex><a rel=\"nofollow\" href=\"https:\/\/www.dice.com\">www.dice.com<\/a><\/noindex> \u2014 n\u00eb t\u00eb cilat paraqiten t\u00eb dh\u00ebnat p\u00ebr dat\u00ebn 17 mars 2020 p\u00ebr numrin e ofertave t\u00eb publikuara, t\u00eb marra p\u00ebr fjal\u00eb ky\u00e7e: Data Quality dhe Data Scientist. <\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/www.monster.com\">www.monster.com<\/a><\/noindex><\/p>\n<p>Data Scientists \u2013 21416 oferta<br \/>\nData Quality \u2013 41104 oferta<\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/8578f2d1e600791e166ed6e8a7068015.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/3ff69acc3c7e0d649eae0808b1b535be.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\n<noindex><a rel=\"nofollow\" href=\"https:\/\/www.dice.com\">www.dice.com<\/a><\/noindex> <\/p>\n<p>Data Scientists \u2013 404 oferta<br \/>\nData Quality \u2013 2020 oferta<\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/b35a9b423ba4bef8736659580e3c281e.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/2c29b83bbe2361089de5a0ab65017300.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>\nPadyshim se k\u00ebto profesione nuk konkurrojn\u00eb n\u00eb asnj\u00eb m\u00ebnyr\u00eb me nj\u00ebra-tjetr\u00ebn. Me k\u00ebto kapje ekrani, thjesht doja t\u00eb ilustroja situat\u00ebn aktuale n\u00eb tregun e pun\u00ebs sa i p\u00ebrket k\u00ebrkesave p\u00ebr inxhinier\u00eb t\u00eb Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave, t\u00eb cil\u00ebt tani k\u00ebrkohen shum\u00eb m\u00eb tep\u00ebr se Data Scientists. <\/p>\n<p>N\u00eb qershor 2019, EPAM, duke u p\u00ebrgjigjur k\u00ebrkesave t\u00eb tregut modern t\u00eb IT-s\u00eb, e ndau drejtimin e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave n\u00eb nj\u00eb praktik\u00eb t\u00eb ve\u00e7ant\u00eb. Inxhinier\u00ebt e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave gjat\u00eb pun\u00ebs s\u00eb tyre t\u00eb p\u00ebrditshme menaxhojn\u00eb t\u00eb dh\u00ebnat, kontrollojn\u00eb sjelljen e tyre n\u00eb kushte dhe sisteme t\u00eb reja, monitorojn\u00eb relevanc\u00ebn, mjaftueshm\u00ebrin\u00eb dhe aktualitetin e t\u00eb dh\u00ebnave. Nd\u00ebrkoh\u00eb, n\u00eb kuptimin praktik, inxhinier\u00ebt e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave v\u00ebrtet i kushtojn\u00eb pak koh\u00eb testimit tradicional funksional, <u>POR<\/u> kjo varet shum\u00eb nga projekti (nj\u00eb shembuj do t\u00eb jap m\u00eb pas). <\/p>\n<p>Detyrat e inxhinierit t\u00eb Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave nuk p\u00ebrfshijn\u00eb vet\u00ebm verifikimet rutin\u00eb manuale\/automatik\u00eb p\u00ebr \"nulls, count dhe sums\" n\u00eb tabelat e bazave t\u00eb t\u00eb dh\u00ebnave, por k\u00ebrkojn\u00eb nj\u00eb kuptim t\u00eb thell\u00eb t\u00eb nevojave t\u00eb biznesit t\u00eb klient\u00ebve dhe, p\u00ebr rrjedhoj\u00eb, aft\u00ebsin\u00eb p\u00ebr t\u00eb transformuar t\u00eb dh\u00ebnat e disponueshme n\u00eb informacion t\u00eb dobish\u00ebm p\u00ebr biznesin. <\/p>\n<h2>Teoria e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave<\/h2>\n<p>\n<img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/d920a7a0435d6f3a1eb84fc16221c742.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p>P\u00ebr t\u00eb p\u00ebrshkruar m\u00eb mir\u00eb rolin e k\u00ebtij inxhinieri, le t\u00eb kuptojm\u00eb se \u00e7far\u00eb \u00ebsht\u00eb Cil\u00ebsia e t\u00eb Dh\u00ebnave n\u00eb teori. <\/p>\n<p><b>Cil\u00ebsia e t\u00eb Dh\u00ebnave<\/b> \u2014 nj\u00eb nga fazat e Menaxhimit t\u00eb t\u00eb Dh\u00ebnave (nj\u00eb bot\u00eb e t\u00ebr\u00eb, t\u00eb cil\u00ebn do ta l\u00ebm\u00eb p\u00ebr studim t\u00eb pavarur) dhe p\u00ebrgjigjet p\u00ebr analiz\u00ebn e t\u00eb dh\u00ebnave sipas kritereve t\u00eb m\u00ebposhtme: <\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/8f5e75bad69e3cb589cc0e1104ad946f.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nMendoj se nuk \u00ebsht\u00eb e nevojshme t\u00eb shpjegojm\u00eb \u00e7do pik\u00eb (n\u00eb teori quhen \"dimensionet e t\u00eb dh\u00ebnave\"), ato jan\u00eb t\u00eb p\u00ebrshkruara shum\u00eb mir\u00eb n\u00eb imazh. Por procesi i testimit nuk n\u00ebnkupton kopjimin e rrept\u00eb t\u00eb k\u00ebtyre karakteristikave n\u00eb rastet e testeve dhe kontrollin e tyre. N\u00eb Cil\u00ebsin\u00eb e t\u00eb Dh\u00ebnave, si n\u00eb \u00e7do form\u00eb tjet\u00ebr testimi, \u00ebsht\u00eb e nevojshme s\u00eb pari t\u00eb bazohemi n\u00eb k\u00ebrkesat p\u00ebr cil\u00ebsin\u00eb e t\u00eb dh\u00ebnave, t\u00eb miratuara me pjes\u00ebmarr\u00ebsit e projektit, t\u00eb cil\u00ebt marrin vendime biznesi. <\/p>\n<p><\/p>\n<blockquote><p>N\u00eb var\u00ebsi t\u00eb projektit, inxhinieri i Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave mund t\u00eb kryej\u00eb funksione t\u00eb ndryshme: nga nj\u00eb tester automatizues i zakonsh\u00ebm me nj\u00eb vler\u00ebsim t\u00eb sip\u00ebrfaqsh\u00ebm t\u00eb cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave, deri te nj\u00eb person q\u00eb kryen profilizimin e thell\u00eb t\u00eb tyre sipas karakteristikave t\u00eb m\u00ebsip\u00ebrme. <\/p><\/blockquote>\n<p>Nj\u00eb p\u00ebrshkrim shum\u00eb i detajuar i proceseve t\u00eb Menaxhimit t\u00eb t\u00eb Dh\u00ebnave, Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave dhe fushave t\u00eb lidhura \u00ebsht\u00eb p\u00ebrshkruar shk\u00eblqyer n\u00eb lib\u00ebr me titullin <i>\u00abDAMA-DMBOK: Data Management Body of Knowledge: 2nd Edition\u00bb<\/i>. E rekomandoj shum\u00eb k\u00ebt\u00eb lib\u00ebr si nj\u00eb hyrje n\u00eb k\u00ebt\u00eb fush\u00eb (linkun p\u00ebr t\u00eb do ta gjeni n\u00eb fund t\u00eb artikullit).<\/p>\n<h2>Historia ime<\/h2>\n<p>\nN\u00eb industrin\u00eb IT kam kaluar nga Junior tester n\u00eb kompanit\u00eb produktive deri n\u00eb Lead Data Quality Engineer n\u00eb kompanin\u00eb EPAM. Rreth dy vjet pas fillimit t\u00eb pun\u00ebs si tester, pata nj\u00eb bindje t\u00eb fort\u00eb se kisha b\u00ebr\u00eb t\u00eb gjitha llojet e testimeve: regresive, funksionale, stresit, stabilitetit, siguris\u00eb, UI etj. \u2014 dhe kam provuar nj\u00eb s\u00ebr\u00eb tools testimi, duke punuar n\u00eb tre gjuh\u00eb programimi: Java, Scala, Python. <\/p>\n<p><\/p>\n<blockquote><p>Duke u kthyer pas, kuptoj se p\u00ebrse grupi im i aft\u00ebsive profesionale u b\u00eb kaq i ndrysh\u00ebm \u2014 kam marr\u00eb pjes\u00eb n\u00eb projekte q\u00eb lidhen me pun\u00ebn me t\u00eb dh\u00ebna, t\u00eb m\u00ebdha dhe t\u00eb vogla. Ky ishte shkaku q\u00eb m\u00eb \u00e7oi n\u00eb bot\u00ebn e shum\u00eb mjeteve dhe mund\u00ebsive p\u00ebr rritje.<\/p><\/blockquote>\n<p>P\u00ebr t\u00eb vler\u00ebsuar larmishm\u00ebrin\u00eb e mjeteve dhe mund\u00ebsive p\u00ebr t\u00eb fituar njohuri dhe aft\u00ebsi t\u00eb reja, mjafton t\u00eb shikoni imazhin m\u00eb posht\u00eb, ku jan\u00eb paraqitur m\u00eb t\u00eb njohurit n\u00eb bot\u00ebn e \u00abData &amp; AI\u00bb.<\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/e3278694195bf59b44055e1eaafd62b1.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nK\u00ebto ilustime p\u00ebrgatiten \u00e7do vit nga nj\u00eb nga kapitalist\u00ebt m\u00eb t\u00eb njohur t\u00eb riskut, Matt Turck, nj\u00eb ish zhvillues softuer\u00ebsh. Ja <noindex><a rel=\"nofollow\" href=\"https:\/\/mattturck.com\">link<\/a><\/noindex> blogu i tij dhe <noindex><a rel=\"nofollow\" href=\"https:\/\/firstmarkcap.com\/team\/matt-turck\">firma e kapitalit t\u00eb rrezikut<\/a><\/noindex>, ku ai punon si partner. <\/p>\n<p>Ve\u00e7an\u00ebrisht kam p\u00ebrparuar profesionalisht shpejt kur isha tester i vet\u00ebm n\u00eb projekt, ose t\u00eb pakt\u00ebn n\u00eb fillim t\u00eb projektit. N\u00eb ato momente je p\u00ebrgjegj\u00ebs p\u00ebr t\u00eb gjith\u00eb procesin e testimit, dhe nuk ke mund\u00ebsi t\u00eb t\u00ebrheq\u00ebsh prapa, vet\u00ebm p\u00ebrpara. N\u00eb fillim m\u00eb frik\u00ebsonte kjo, por tani m\u00eb jan\u00eb t\u00eb qarta t\u00eb gjitha avantazhet e nj\u00eb sfide t\u00eb till\u00eb: <\/p>\n<ul>\n<li>Fillon t\u00eb komunikosh me gjith\u00eb ekipin si kurr\u00eb m\u00eb par\u00eb, pasi nuk ka asnj\u00eb nd\u00ebrmjet\u00ebs p\u00ebr komunikim: as menaxher testi, as koleg\u00eb testers. <\/li>\n<li>P\u00ebrfshirja n\u00eb projekt b\u00ebhet jasht\u00ebzakonisht e thell\u00eb, dhe ti disponon informacion mbi t\u00eb gjitha komponent\u00ebt si n\u00eb p\u00ebrgjith\u00ebsi, ashtu edhe n\u00eb detaje. <\/li>\n<li>Zhvilluesit nuk t\u00eb shohin si \u00abat\u00eb djalin nga testimi q\u00eb merret me di\u00e7ka q\u00eb nuk \u00ebsht\u00eb e qart\u00eb\u00bb, por m\u00eb tep\u00ebr si nj\u00eb t\u00eb barabart\u00eb, q\u00eb sjell nj\u00eb dobi t\u00eb jasht\u00ebzakonshme p\u00ebr ekipin me testet e tij automatike dhe parashikimin e shfaqjes s\u00eb gabimeve n\u00eb nj\u00eb bllok specifik t\u00eb produktit. <\/li>\n<li>Si rezultat \u2014 je m\u00eb efikas, m\u00eb i kualifikuar, m\u00eb i k\u00ebrkuar. <\/li>\n<\/ul>\n<p>\nMe rritjen e projektit, n\u00eb 100% t\u00eb rasteve kam qen\u00eb mentor p\u00ebr tester\u00ebt e rinj q\u00eb kan\u00eb ardhur n\u00eb t\u00eb, kam m\u00ebsuar ata dhe kam ndar\u00eb njohurit\u00eb q\u00eb kam fituar. N\u00eb k\u00ebt\u00eb rast, var\u00ebsisht nga projekti, nuk kam marr\u00eb gjithmon\u00eb nga drejtor\u00ebt specialist\u00eb t\u00eb testimit automatizues me nivel t\u00eb lart\u00eb dhe ka pasur nevoj\u00eb p\u00ebr t\u00eb m\u00ebsuar ata n\u00eb automatizim (p\u00ebr ata q\u00eb d\u00ebshironin), ose p\u00ebr t\u00eb krijuar mjete p\u00ebr t'i p\u00ebrdorur ata n\u00eb aktivitete t\u00eb p\u00ebrditshme (mjete gjenerimi t\u00eb t\u00eb dh\u00ebnave dhe ngarkimi i tyre n\u00eb sistem, mjet p\u00ebr kryerjen e testimeve t\u00eb ngarkes\u00ebs\/testimeve t\u00eb stabilitetit \u00abshpejt\u00bb etj.). <\/p>\n<h3>Nj\u00eb shembull i projektit konkret<\/h3>\n<p>\nFatkeq\u00ebsisht, p\u00ebr shkak t\u00eb angazhimeve p\u00ebr moszbulim, nuk mund t\u00eb flas n\u00eb detaje p\u00ebr projektet n\u00eb t\u00eb cilat kam punuar, megjithat\u00eb do t\u00eb jap shembuj t\u00eb detyrave tipike t\u00eb Inxhinierit t\u00eb Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave n\u00eb nj\u00eb nga projektet. <\/p>\n<p>Q\u00ebllimi i projektit \u2014 t\u00eb realizohet nj\u00eb platform\u00eb p\u00ebr p\u00ebrgatitjen e t\u00eb dh\u00ebnave p\u00ebr trajnim mbi baz\u00ebn e modeleve t\u00eb m\u00ebsimit t\u00eb makineris\u00eb. Klienti ishte nj\u00eb kompani farmaceutike e madhe nga SHBA. Teknikisht, ky ishte nj\u00eb grup <noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes.io\">Kubernetes<\/a><\/noindex>, q\u00eb ngrihej n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/ec2\">AWS EC2<\/a><\/noindex> instance, me disa mikrosh\u00ebrbime dhe nj\u00eb projekt Open Source n\u00eb thelb t\u00eb kompanis\u00eb EPAM \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/legion-platform\/legion\">Legion<\/a><\/noindex>, i adaptuar p\u00ebr nevojat e klientit t\u00eb ve\u00e7ant\u00eb (tani projekti \u00ebsht\u00eb transformuar n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/odahu\/odahu-flow\">odahu<\/a><\/noindex>). Proceset ETL u organizuan p\u00ebrmes <noindex><a rel=\"nofollow\" href=\"https:\/\/airflow.apache.org\">Apache Airflow<\/a><\/noindex> dhe transferuan t\u00eb dh\u00ebnat nga <noindex><a rel=\"nofollow\" href=\"https:\/\/www.salesforce.com\">sistemi SalesForce<\/a><\/noindex> i klientit n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/s3\/?nc1=h_ls\">AWS S3<\/a><\/noindex> Buckets. M\u00eb pas, nj\u00eb imazh docker i modelit t\u00eb m\u00ebsimit t\u00eb makineris\u00eb u vendos n\u00eb platform\u00eb, e cila m\u00ebsonte mbi t\u00eb dh\u00ebna t\u00eb reja dhe p\u00ebrmes nd\u00ebrfaqes REST API jepte parashikime q\u00eb interesonin biznesin dhe zgjidhnin probleme specifike. <\/p>\n<p><b>Pamja vizuale ishte p\u00ebraf\u00ebrsisht k\u00ebshtu: <\/b><\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/c2de3c1919f3f9ff29147fce09eb14c4.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nKa p\u00ebrfundimeve t\u00eb testimit funksional n\u00eb k\u00ebt\u00eb projekt kishte me boll\u00ebk, dhe duke marr\u00eb parasysh shpejt\u00ebsin\u00eb e zhvillimit t\u00eb funksioneve dhe nevoj\u00ebn p\u00ebr t\u00eb mbajtur ritmin e ciklit t\u00eb l\u00ebshimit (sprintet dyjavore), ishte e domosdoshme t\u00eb mendonim menj\u00ebher\u00eb p\u00ebr automatizimin e testimit t\u00eb nyjave m\u00eb kritike t\u00eb sistemit. Pjesa m\u00eb e madhe e platform\u00ebs me baz\u00eb Kubernetes ishte e mbuluar nga testet automatike, t\u00eb realizuara n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/robotframework.org\/\">Robot Framework<\/a><\/noindex> + Python, por ishte e nevojshme gjithashtu t\u00eb mbaheshin dhe zgjerohej ato. Gjithashtu, p\u00ebr leht\u00ebsin\u00eb e klientit, u krijua nj\u00eb GUI p\u00ebr menaxhimin e modeleve t\u00eb m\u00ebsimit t\u00eb makinerive, t\u00eb deponuar n\u00eb grumbull, si dhe mund\u00ebsia p\u00ebr t\u00eb treguar nga dhe n\u00eb cilin vend duhej t\u00eb transferoheshin t\u00eb dh\u00ebnat p\u00ebr trajnimin e modeleve. Ky shtes\u00eb i gjer\u00eb solli zgjerimin e kontrolleve automatizuese funksionale, q\u00eb kryesisht u realizuan p\u00ebrmes thirrjeve REST API dhe nj\u00eb numri t\u00eb vog\u00ebl t\u00eb testeve UI end-2-end. Rreth ekuatorit t\u00eb gjith\u00eb k\u00ebtij procesi, u bashkua nj\u00eb testues manual, i cili e menaxhoi shum\u00eb mir\u00eb testimin e pranuar t\u00eb versioneve t\u00eb produktit dhe komunikimin me klientin p\u00ebr miratimin e l\u00ebshimit t\u00eb ardhsh\u00ebm. P\u00ebr m\u00eb tep\u00ebr, me ardhjen e nj\u00eb specialisti t\u00eb ri, ne arrit\u00ebm t\u00eb dokumentonim pun\u00ebn ton\u00eb dhe t\u00eb shtonin disa kontrolle manuale shum\u00eb t\u00eb r\u00ebnd\u00ebsishme, t\u00eb cilat ishte e v\u00ebshtir\u00eb t'i automatizonim menj\u00ebher\u00eb. <\/p>\n<p>Dhe p\u00ebrfundimisht, pasi arrit\u00ebm stabilitetin nga platforma dhe shtresa GUI mbi t\u00eb, filluam t\u00eb nd\u00ebrtojm\u00eb ETL pipelines duke p\u00ebrdorur Apache Airflow DAGs. Kontrolli automatizuar i cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave u realizua duke shkruar DAG t\u00eb ve\u00e7ant\u00eb Airflow q\u00eb kontrollonin t\u00eb dh\u00ebnat n\u00eb p\u00ebrputhje me rezultatet e procesit ETL. N\u00eb k\u00ebt\u00eb projekt na u afrua fat, dhe klienti na dha akses n\u00eb grupe t\u00eb dh\u00ebnash t\u00eb paidentifikuara, mbi t\u00eb cilat ne testoheshim. Kontrollonim t\u00eb dh\u00ebnat rresht pas rreshti p\u00ebr p\u00ebrputhshm\u00ebrin\u00eb me tiparet, pranin\u00eb e t\u00eb dh\u00ebnave t\u00eb prishura, shifr\u00ebn totale t\u00eb regjistrimeve para dhe pas, krahasimin e transformimeve t\u00eb kryera nga procesi ETL n\u00eb agregimin, ndryshimin e emrave t\u00eb kolonave dhe m\u00eb shum\u00eb. P\u00ebr m\u00eb tep\u00ebr, k\u00ebto kontrolle u skaluan n\u00eb burime t\u00eb ndryshme t\u00eb t\u00eb dh\u00ebnave, p\u00ebr shembull p\u00ebrve\u00e7 SalesForce edhe n\u00eb MySQL. <\/p>\n<p>Kontrollimi i cil\u00ebsis\u00eb s\u00eb fundit t\u00eb t\u00eb dh\u00ebnave u realizua tashm\u00eb n\u00eb nivelin S3, ku ato ishin ruajtur dhe ishin n\u00eb gjendje ready-to-use p\u00ebr trajnimin e modeleve t\u00eb m\u00ebsimit t\u00eb makinerive. P\u00ebr t\u00eb marr\u00eb t\u00eb dh\u00ebnat nga skedari p\u00ebrfundimtar CSV, q\u00eb ndodhej n\u00eb S3 Bucket, dhe p\u00ebr validimin e tyre, u shkrua nj\u00eb kod duke p\u00ebrdorur <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/sdk-for-python\">boto3 klientin<\/a><\/noindex>. <\/p>\n<p>Gjithashtu, nga ana e klientit kishte k\u00ebrkes\u00ebn p\u00ebr ruajtjen e nj\u00eb pjese t\u00eb t\u00eb dh\u00ebnave n\u00eb nj\u00eb S3 Bucket dhe nj\u00eb pjese n\u00eb nj\u00eb tjet\u00ebr. P\u00ebr k\u00ebt\u00eb u nevojit\u00ebn t\u00eb shkruheshin kontrolle shtes\u00eb, q\u00eb kontrollonin sakt\u00ebsin\u00eb e nj\u00eb ndarjeje t\u00eb till\u00eb. <\/p>\n<h3>Eksperienca e p\u00ebrmbledhur nga projekte t\u00eb tjera<\/h3>\n<p>\nShembulli i list\u00ebs m\u00eb t\u00eb p\u00ebrgjithshme t\u00eb aktiviteteve t\u00eb Data Quality inxhinierit: <\/p>\n<ul>\n<li>P\u00ebrgatitni t\u00eb dh\u00ebna testuese (valide, t\u00eb pavlefshme, t\u00eb m\u00ebdha, t\u00eb vogla) p\u00ebrmes nj\u00eb mjeti automatizuar. <\/li>\n<li>Ngarko setin e p\u00ebrgatitur t\u00eb t\u00eb dh\u00ebnave n\u00eb burimin origjinal dhe kontrolloni gatishm\u00ebrin\u00eb e tij p\u00ebr p\u00ebrdorim. <\/li>\n<li>Aktivizoni proceset ETL p\u00ebr p\u00ebrpunimin e setit t\u00eb t\u00eb dh\u00ebnave nga depoja origjinale n\u00eb p\u00ebrfundimtare ose t\u00eb nd\u00ebrmjetme duke p\u00ebrdorur nj\u00eb grup t\u00eb caktuar konfigurimesh (n\u00ebse \u00ebsht\u00eb e mundur t\u00eb caktohen parametra konfigurues p\u00ebr detyr\u00ebn ETL). <\/li>\n<li>Verifikoni t\u00eb dh\u00ebnat e p\u00ebrpunuara nga procesi ETL p\u00ebr cil\u00ebsin\u00eb e tyre dhe p\u00ebrputhshm\u00ebrin\u00eb me k\u00ebrkesat e biznesit. <\/li>\n<\/ul>\n<p>\nDhe n\u00eb k\u00ebt\u00eb pik\u00eb, theksi kryesor i kontrollit duhet t\u00eb jet\u00eb jo vet\u00ebm n\u00eb at\u00eb se si rrjedha e t\u00eb dh\u00ebnave n\u00eb sistem ka kaluar n\u00eb p\u00ebrgjith\u00ebsi, por kryesisht n\u00eb verifikimin dhe validimin e t\u00eb dh\u00ebnave n\u00eb p\u00ebrputhje me k\u00ebrkesat e pritura, identifikimin e anomalive dhe m\u00eb shum\u00eb. <\/p>\n<h3>Mjetet<\/h3>\n<p>\nNj\u00eb nga teknik\u00ebt e till\u00eb t\u00eb kontrollit t\u00eb t\u00eb dh\u00ebnave mund t\u00eb jet\u00eb organizimi i kontrollove t\u00eb zinxhir\u00ebve n\u00eb \u00e7do faz\u00eb t\u00eb p\u00ebrpunimit t\u00eb t\u00eb dh\u00ebnave, t\u00eb njohura n\u00eb literatur\u00eb si 'data chain' \u2014 kontrolli i t\u00eb dh\u00ebnave nga burimi deri n\u00eb pik\u00ebn e p\u00ebrdorimit p\u00ebrfundimtar. Kontroll\u00ebt e tilla zakonisht realizohen p\u00ebrmes shkrimit t\u00eb pyetjeve SQL t\u00eb verifikimit. Natyrisht, q\u00eb k\u00ebto pyetje duhet t\u00eb jen\u00eb sa m\u00eb t\u00eb lehta dhe t\u00eb kontrollojn\u00eb cop\u00ebza t\u00eb ve\u00e7anta t\u00eb cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave (metadata e tabelave, rreshtat bosh, NULL, Gabimet n\u00eb sintaks\u00eb \u2014 atributet e tjera t\u00eb k\u00ebrkuara p\u00ebr kontroll). <\/p>\n<p>N\u00eb rastin e testimit regresiv, n\u00eb t\u00eb cilin p\u00ebrdoren sete t\u00eb dh\u00ebnash tashm\u00eb t\u00eb gatshme (t\u00eb pandryshueshme ose pak t\u00eb ndryshueshme), n\u00eb kodin e testeve automatike mund t\u00eb ruhen t\u00eb gatshme shabllone verifikimi p\u00ebr cil\u00ebsin\u00eb e t\u00eb dh\u00ebnave (p\u00ebrshkrime t\u00eb metadatas s\u00eb pritura t\u00eb tabelave; objekte t\u00eb p\u00ebrzgjedhura n\u00eb rresht, t\u00eb cilat mund t\u00eb zgjidhen rast\u00ebsisht gjat\u00eb testit, etj.). <\/p>\n<p>Gjithashtu, gjat\u00eb testimit \u00ebsht\u00eb e nevojshme t\u00eb shkruhen procese testuese ETL, duke p\u00ebrdorur framework-e si Apache Airflow, <noindex><a rel=\"nofollow\" href=\"https:\/\/spark.apache.org\/\">Apache Spark<\/a><\/noindex> ose fare mjete cloud black-box si <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/dataprep\">GCP Dataprep<\/a><\/noindex>, <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/dataflow\">GCP Dataflow<\/a><\/noindex> etj. Ky faktor b\u00ebn q\u00eb inxhinieri i testimit t\u00eb thellohet n\u00eb parimet e funksionimit t\u00eb mjeteve t\u00eb p\u00ebrmendura dhe t\u00eb kryej\u00eb m\u00eb efektivisht si testimin funksional (p\u00ebr shembull, t\u00eb proceseve ETL ekzistuese n\u00eb projekt), ashtu edhe t'i p\u00ebrdor\u00eb ato p\u00ebr t\u00eb verifikuar t\u00eb dh\u00ebnat. N\u00eb ve\u00e7anti, p\u00ebr Apache Airflow, ekzistojn\u00eb tashm\u00eb operator\u00eb t\u00eb gatsh\u00ebm p\u00ebr pun\u00eb me bazat e t\u00eb dh\u00ebnave analitike popullore, si\u00e7 \u00ebsht\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery\">GCP BigQuery<\/a><\/noindex>. Shembulli m\u00eb bazik i p\u00ebrdorimit t\u00eb tij \u00ebsht\u00eb p\u00ebrshkruar tashm\u00eb, <noindex><a rel=\"nofollow\" href=\"https:\/\/medium.com\/@yuu.ishikawa\/apache-airflow-as-a-data-quality-checker-416ca7f5a3ad\">k\u00ebtu<\/a><\/noindex>, prandaj nuk do t\u00eb p\u00ebrs\u00ebritem. <\/p>\n<p>P\u00ebrve\u00e7 zgjidhjeve t\u00eb gatshme, askush nuk e ndalon t\u00eb realizoni teknikat dhe mjetet tuaja. Kjo do t\u00eb jet\u00eb e dobishme jo vet\u00ebm p\u00ebr projektin, por edhe p\u00ebr vet\u00eb Inxhinierin e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave, i cili k\u00ebshtu do t\u00eb zhvilloj\u00eb horizontin e tij teknik dhe aft\u00ebsit\u00eb e kodimit. <\/p>\n<h3>Si funksionon kjo n\u00eb nj\u00eb projekt real<\/h3>\n<p>\nNj\u00eb ilustim i mir\u00eb i parashtrimeve t\u00eb fundit p\u00ebr \"zinxhirin e t\u00eb dh\u00ebnave\", ETL dhe kontrollin e gjeneruar kudo \u00ebsht\u00eb procesi n\u00eb vijim nga nj\u00eb nga projektet reale: <\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/4a4cd3043b584bb3fbce4b9c3021e9f6.jpg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nK\u00ebtu, n\u00eb \"vrullin\" e sistemit ton\u00eb hyjn\u00eb t\u00eb dh\u00ebna t\u00eb ndryshme (natyrisht, t\u00eb p\u00ebrgatitura nga ne): t\u00eb vlefshme, t\u00eb pavlefshme, t\u00eb p\u00ebrziera, etj., pastaj ato filtrohen dhe kalojn\u00eb n\u00eb nj\u00eb depo t\u00eb p\u00ebrkohshme, m\u00eb pas p\u00ebsojn\u00eb nj\u00eb s\u00ebr\u00eb p\u00ebrpunimesh dhe vendosen n\u00eb nj\u00eb depo p\u00ebrfundimtare, nga e cila, p\u00ebr rrjedhoj\u00eb, do t\u00eb kryhet analiza, nd\u00ebrtimi i pamjeve t\u00eb t\u00eb dh\u00ebnave dhe gjetja e njohurive t\u00eb biznesit. N\u00eb nj\u00eb sistem t\u00eb till\u00eb, pa verifikuar funksionalisht pun\u00ebn e proceseve ETL, p\u00ebrqendrohemi n\u00eb cil\u00ebsin\u00eb e t\u00eb dh\u00ebnave para dhe pas p\u00ebrpunimeve, si dhe n\u00eb daljen n\u00eb analiz\u00eb. <\/p>\n<p>Duke p\u00ebrmbledhur at\u00eb q\u00eb u tha m\u00eb sip\u00ebr, pa marr\u00eb parasysh vendet ku kam punuar, gjithmon\u00eb kam qen\u00eb i p\u00ebrfshir\u00eb n\u00eb projekte t\u00eb t\u00eb dh\u00ebnave, t\u00eb cilat p\u00ebrfshinin karakteristika t\u00eb m\u00ebposhtme: <\/p>\n<ul>\n<li>Vet\u00ebm p\u00ebrmes automatizimit mund t\u00eb kontrollohen disa raste dhe t\u00eb arrihet nj\u00eb cik\u00ebl publikimi i pranuesh\u00ebm p\u00ebr biznesin. <\/li>\n<li>Testuesi n\u00eb nj\u00eb projekt t\u00eb till\u00eb \u00ebsht\u00eb nj\u00eb nga an\u00ebtar\u00ebt m\u00eb t\u00eb respektuar t\u00eb ekipit, pasi sjell nj\u00eb p\u00ebrfitim t\u00eb madh p\u00ebr secilin prej an\u00ebtar\u00ebve (shpejtimi i testimit, t\u00eb dh\u00ebna t\u00eb mira p\u00ebr Data Scientist, identifikimi i defekteve n\u00eb fazat e hershme). <\/li>\n<li>Nuk ka r\u00ebnd\u00ebsi n\u00ebse punoni n\u00eb pajisjet tuaja apo n\u00eb cloud \u2014 t\u00eb gjith\u00eb burimet jan\u00eb t\u00eb abstaruara n\u00eb nj\u00eb klas\u00eb si Hortonworks, Cloudera, Mesos, Kubernetes etj. <\/li>\n<li>Projektet nd\u00ebrtohen me nj\u00eb qasje mikrosh\u00ebrbimesh, dhe dominon llogaritja e shp\u00ebrndar\u00eb dhe e paralel. <\/li>\n<\/ul>\n<p>\nDua t\u00eb theksoj se, duke u marr\u00eb me testimin n\u00eb fush\u00ebn e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave, specialisti i testimit zhvendos fokusin e tij profesional n\u00eb kodin e produktit dhe mjetet e p\u00ebrdorura. <\/p>\n<h3>Karakteristikat dalluese t\u00eb testimit t\u00eb Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave<\/h3>\n<p>\nP\u00ebrve\u00e7 k\u00ebsaj, p\u00ebr veten time kam identifikuar karakteristika t\u00eb tjera (t\u00eb theksuara shum\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb p\u00ebrgjith\u00ebsuar dhe n\u00eb m\u00ebnyr\u00eb subjektive) t\u00eb testimit n\u00eb projekte (sistemat) Data (Big Data) dhe drejtime t\u00eb tjera: <\/p>\n<p><img decoding=\"async\" alt=\"Testues i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: trendet, teoria, historia ime\" src=\"\/wp-content\/uploads\/2020\/04\/8df52fbae2998e70755bf84e254db83e.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<h2>Linqe t\u00eb dobishme<\/h2>\n<p><\/p>\n<ol>\n<li>Teoria: <noindex><a rel=\"nofollow\" href=\"https:\/\/www.amazon.com\/DAMA-DMBOK-Data-Management-Body-Knowledge\/dp\/1634622340\">DAMA-DMBOK: Data Management Body of Knowledge: Botimi i 2-t\u00eb<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/training.ru\/#!\/Home?lang=ru\">Qendra e Trajnimeve<\/a><\/noindex>&nbsp;EPAM&nbsp;<\/li>\n<li>Materialet e rekomanduara p\u00ebr nj\u00eb inxhinier t\u00eb ri t\u00eb Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave:\n<ol>\n<li>Kurs falas n\u00eb Stepik:&nbsp;<noindex><a rel=\"nofollow\" href=\"https:\/\/stepik.org\/course\/551\/syllabus\">Hyrje n\u00eb bazat e t\u00eb dh\u00ebnave<\/a><\/noindex>.&nbsp;<\/li>\n<li>Kurs n\u00eb LinkedIn Learning:&nbsp;<noindex><a rel=\"nofollow\" href=\"https:\/\/www.linkedin.com\/learning\/data-science-foundations-data-engineering\/welcome\">Data Science Foundations: Data Engineering<\/a><\/noindex>.<\/li>\n<li>Artikuj:\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.stickyminds.com\/article\/business-intelligence-and-data-quality\">Inteligjenca Biznesore dhe Cil\u00ebsia e t\u00eb Dh\u00ebnave<\/a><\/noindex>;&nbsp;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.datagaps.com\/concepts\/etl-testing\">Testimi ETL<\/a><\/noindex>;&nbsp;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/panoply.io\/analytics-stack-guide\/data-profiling-best-practices\">\u00c7far\u00eb \u00ebsht\u00eb Profilimi i t\u00eb Dh\u00ebnave? Procesi, Praktikat m\u00eb t\u00eb Mir\u00eb dhe Mjetet<\/a><\/noindex>;&nbsp;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.guru99.com\/software-testing-test-data.html\">Krijimi i t\u00eb Dh\u00ebnave p\u00ebr Testim: \u00c7far\u00eb \u00ebsht\u00eb, Si t\u00eb, Shembuj, Mjete;&nbsp;<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.testbytes.net\/blog\/5-test-data-generation-techniques-to-know\">5 Teknikat e Krijimit t\u00eb t\u00eb Dh\u00ebnave p\u00ebr Testim q\u00eb Duhet t\u00eb Dini.<\/a><\/noindex>&nbsp;<\/li>\n<\/ul>\n<\/li>\n<li>Video:\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=HWaBdqmTqEA\">Koncepat e Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave | Tutorial p\u00ebr Cil\u00ebsin\u00eb e t\u00eb Dh\u00ebnave | Tutorial p\u00ebr Depo t\u00eb Dh\u00ebnash<\/a><\/noindex>;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=avDqzhuHFkw\">Profilimi i t\u00eb Dh\u00ebnave duke p\u00ebrdorur SSIS<\/a><\/noindex>;<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=r7SF5WldITk\">Profilimi i t\u00eb Dh\u00ebnave dhe Procesimi i Pipeline me Spark.<\/a><\/noindex><\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p><\/p>\n<h2>P\u00ebrfundimi<\/h2>\n<p>\n<b>Cil\u00ebsia e t\u00eb Dh\u00ebnave<\/b> \u2014 \u00ebsht\u00eb nj\u00eb fush\u00eb shum\u00eb e re dhe premtuese, t\u00eb jesh pjes\u00eb e s\u00eb cil\u00ebs do t\u00eb thot\u00eb t\u00eb jesh pjes\u00eb e nj\u00eb startup-i. Duke hyr\u00eb n\u00eb Cil\u00ebsin\u00eb e t\u00eb Dh\u00ebnave, do t\u00eb zhytet n\u00eb nj\u00eb mori teknologjish t\u00eb k\u00ebrkuara moderne, por m\u00eb e r\u00ebnd\u00ebsishmja \u2014 do t\u00eb hapen mund\u00ebsi t\u00eb m\u00ebdha p\u00ebr t\u00eb gjeneruar dhe realizuar idet\u00eb tuaja. Do t\u00eb keni mund\u00ebsin\u00eb t\u00eb p\u00ebrdorni qasjen e p\u00ebrmir\u00ebsimit t\u00eb vazhduesh\u00ebm jo vet\u00ebm n\u00eb projekt, por edhe p\u00ebr veten, duke u zhvilluar vazhdimisht si profesionist.<br \/>\n<br \/>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/epam_systems\/blog\/495478\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u0438 \u044f Data Quality \u0438\u043d\u0436\u0435\u043d\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 \u043d\u0430 \u043f\u0440\u0435\u0434\u043c\u0435\u0442 \u0438\u0445 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0430. \u0412 \u044d\u0442\u043e\u0439 \u0441\u0442\u0430\u0442\u044c\u0435 \u0440\u0435\u0447\u044c \u043f\u043e\u0439\u0434\u0451\u0442 \u043e \u0442\u043e\u043c, \u043a\u0430\u043a \u044f \u043a \u044d\u0442\u043e\u043c\u0443 \u043f\u0440\u0438\u0448\u0451\u043b \u0438 \u043f\u043e\u0447\u0435\u043c\u0443 \u0432 2020 \u0433\u043e\u0434\u0443 \u044d\u0442\u043e \u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u0442\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f \u043e\u043a\u0430\u0437\u0430\u043b\u043e\u0441\u044c \u043d\u0430 \u0433\u0440\u0435\u0431\u043d\u0435 \u0432\u043e\u043b\u043d\u044b. \u041c\u0438\u0440\u043e\u0432\u043e\u0439 \u0442\u0440\u0435\u043d\u0434 \u0421\u0435\u0433\u043e\u0434\u043d\u044f\u0448\u043d\u0438\u0439 \u043c\u0438\u0440 \u043f\u0435\u0440\u0435\u0436\u0438\u0432\u0430\u0435\u0442 \u043e\u0447\u0435\u0440\u0435\u0434\u043d\u0443\u044e \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0447\u0435\u0441\u043a\u0443\u044e \u0440\u0435\u0432\u043e\u043b\u044e\u0446\u0438\u044e, \u043e\u0434\u043d\u0438\u043c \u0438\u0437 \u0430\u0441\u043f\u0435\u043a\u0442\u043e\u0432 \u043a\u043e\u0442\u043e\u0440\u043e\u0439 \u044f\u0432\u043b\u044f\u0435\u0442\u0441\u044f [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":77181,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-77180","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.0.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u0438 \u044f Data Quality \u0438\u043d\u0436\u0435\u043d\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 \u043d\u0430 \u043f\u0440\u0435\u0434\u043c\u0435\u0442 \u0438\u0445 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0430.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.0.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0422\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0449\u0438\u043a \u0431\u043e\u043b\u044c\u0448\u0438\u0445 \u0438 \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u0445 \u0434\u0430\u043d\u043d\u044b\u0445: \u0442\u0440\u0435\u043d\u0434\u044b, \u0442\u0435\u043e\u0440\u0438\u044f, \u043c\u043e\u044f \u0438\u0441\u0442\u043e\u0440\u0438\u044f | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u0438 \u044f Data Quality \u0438\u043d\u0436\u0435\u043d\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 \u043d\u0430 \u043f\u0440\u0435\u0434\u043c\u0435\u0442 \u0438\u0445 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0430.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-04-08T11:42:32+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-04-08T11:42:32+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Testuesi i t\u00eb dh\u00ebnave t\u00eb m\u00ebdha dhe t\u00eb vogla: tendencat, teoria, historia ime | ProHoster","description":"P\u00ebrsh\u00ebndetje t\u00eb gjith\u00ebve, emri im \u00ebsht\u00eb Aleksand\u00ebr, dhe un\u00eb jam inxhinier i Cil\u00ebsis\u00eb s\u00eb t\u00eb Dh\u00ebnave q\u00eb merret me kontrollin e cil\u00ebsis\u00eb s\u00eb t\u00eb dh\u00ebnave.","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0422\u0435\u0441\u0442\u0438\u0440\u043e\u0432\u0449\u0438\u043a \u0431\u043e\u043b\u044c\u0448\u0438\u0445 \u0438 \u043c\u0430\u043b\u0435\u043d\u044c\u043a\u0438\u0445 \u0434\u0430\u043d\u043d\u044b\u0445: \u0442\u0440\u0435\u043d\u0434\u044b, \u0442\u0435\u043e\u0440\u0438\u044f, \u043c\u043e\u044f \u0438\u0441\u0442\u043e\u0440\u0438\u044f | ProHoster","og:description":"\u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442, \u043c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043b\u0435\u043a\u0441\u0430\u043d\u0434\u0440, \u0438 \u044f Data Quality \u0438\u043d\u0436\u0435\u043d\u0435\u0440, \u043a\u043e\u0442\u043e\u0440\u044b\u0439 \u0437\u0430\u043d\u0438\u043c\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0432\u0435\u0440\u043a\u043e\u0439 \u0434\u0430\u043d\u043d\u044b\u0445 \u043d\u0430 \u043f\u0440\u0435\u0434\u043c\u0435\u0442 \u0438\u0445 \u043a\u0430\u0447\u0435\u0441\u0442\u0432\u0430.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/testirovshhik-bolshih-i-malenkih-dannyh-trendy-teoriya-moya-istoriya","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-04-08T11:42:32+00:00","article:modified_time":"2020-04-08T11:42:32+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"77180","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 17:22:32","updated":"2022-10-03 00:36:00","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/77180","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=77180"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/77180\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/77181"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=77180"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=77180"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=77180"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}