{"id":92508,"date":"2020-08-28T07:42:10","date_gmt":"2020-08-28T05:42:10","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak"},"modified":"2020-08-28T07:42:10","modified_gmt":"2020-08-28T05:42:10","slug":"kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak","title":{"rendered":"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>Ne jemi n\u00eb nj\u00eb koh\u00eb t\u00eb jasht\u00ebzakonshme, kur \u00ebsht\u00eb e mundur t\u00eb lidhen shpejt dhe leht\u00eb disa mjete t\u00eb gatshme open-source, t'i konfigurojm\u00eb ato me 'nd\u00ebrgjegje t\u00eb fikur' sipas k\u00ebshillave t\u00eb stackoverflow, pa u p\u00ebrfshir\u00eb n\u00eb 'shkronja t\u00eb shumta', dhe t\u00eb nisemi p\u00ebr shfryt\u00ebzim komercial. Dhe kur t\u00eb vij\u00eb koha p\u00ebr t'u p\u00ebrdit\u00ebsuar\/zgjeruar ose kur dikush t\u00eb rinis\u00eb disa makina \u2014 do t\u00eb kuptojm\u00eb se ka filluar nj\u00eb \u00ebnd\u00ebrr e keqe q\u00eb morton, gjith\u00e7ka u komplikuar papritur deri n\u00eb njohje, nuk ka rrug\u00eb prapa, e ardhmja \u00ebsht\u00eb e paqart\u00eb dhe m\u00eb e sigurt, n\u00eb vend t\u00eb programimit, do t\u00eb kujdesim p\u00ebr blet\u00ebt dhe do t\u00eb b\u00ebjm\u00eb djath\u00eb.<\/p>\n<p>Nuk \u00ebsht\u00eb rast\u00ebsi q\u00eb koleg\u00ebt m\u00eb t\u00eb p\u00ebrvojsh\u00ebm, me kok\u00eb t\u00eb bardh\u00eb nga gabimet dhe shpejt\u00ebsin\u00eb e jasht\u00ebzakonshme t\u00eb shp\u00ebrndarjes s\u00eb dyzimeve n\u00eb 'kubik\u00ebt' e server\u00ebve t\u00eb shumt\u00eb n\u00eb 'gjuh\u00ebt moderne' me mb\u00ebshtetje t\u00eb integruar p\u00ebr hyrje-dalje asinkrone dhe jo bllokuese \u2014 qeshin me modestin\u00eb e tyre. Dhe vazhdojn\u00eb n\u00eb heshtje t\u00eb rishikojn\u00eb 'man ps', ngulmojn\u00eb deri n\u00eb pikim n\u00eb burimet e 'nginx' dhe shkruajn\u00eb-testojn\u00eb pa fund. Kolektet e din\u00eb q\u00eb gj\u00ebrat m\u00eb interesante do t\u00eb ndodhin m\u00eb von\u00eb, kur 'gjith\u00eb kjo' nj\u00eb nat\u00eb do t\u00eb b\u00ebhet nj\u00eb ngarkes\u00eb n\u00ebn Nat\u00ebn e Re. Dhe do t'u ndihmoj\u00eb vet\u00ebm nj\u00eb kuptim i thell\u00eb i natyr\u00ebs s\u00eb unix, nj\u00eb tabel\u00eb e m\u00ebsuar gjithashtu p\u00ebr gjendjet e TCP\/IP dhe algoritmet e bazuar n\u00eb renditje-k\u00ebrkim.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><br \/>\nAh po, pak u shk\u00ebputa, por shpresoj se arrita t\u00eb transmetoj gjendjen e pritjes.<br \/>\nSot dua t\u00eb ndaj p\u00ebrvoj\u00ebn ton\u00eb n\u00eb vendosjen e nj\u00eb staku t\u00eb p\u00ebrshtatsh\u00ebm dhe t\u00eb lir\u00eb p\u00ebr DataLake, i cili zgjidh shumic\u00ebn e problemeve analitike n\u00eb kompani p\u00ebr struktura t\u00eb ndryshme.<\/p>\n<p>Pak koh\u00eb m\u00eb par\u00eb arrit\u00ebm n\u00eb p\u00ebrfundimin se kompanit\u00eb kan\u00eb nevoj\u00eb gjithnj\u00eb e m\u00eb shum\u00eb p\u00ebr informacionin nga analizat produktore dhe teknike (pa p\u00ebrmendur kuror\u00ebn n\u00eb tort\u00eb si machine learning) dhe p\u00ebr t\u00eb kuptuar trendet dhe rreziqet \u2014 \u00ebsht\u00eb e nevojshme t\u00eb mbledhim dhe analizojm\u00eb gjithnj\u00eb e m\u00eb shum\u00eb metrika.<\/p>\n<h3>Analiza e baz\u00ebs teknike n\u00eb 'Bitrix24'<\/h3>\n<p>\nDisa vite m\u00eb par\u00eb, n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb me nisjen e sh\u00ebrbimit 'Bitrix24', ne investuam aktivisht koh\u00eb dhe burime n\u00eb krijimin e nj\u00eb platforme analitike t\u00eb thjesht\u00eb dhe t\u00eb besueshme, e cila ndihmon p\u00ebr t\u00eb par\u00eb shpejt problemet n\u00eb infrastruktur\u00eb dhe p\u00ebr t\u00eb planifikuar hapat e af\u00ebrt. Sigurisht, ishte e preferueshme t\u00eb merrnim mjete t\u00eb gatshme dhe sa m\u00eb t\u00eb thjeshta dhe t\u00eb kuptueshme. Si rezultat, u zgjodh\u00ebn nagios p\u00ebr monitorim dhe munin p\u00ebr analitik\u00eb dhe vizualizim. Tani kemi mij\u00ebra kontrollime n\u00eb nagios, qindra grafika n\u00eb munin dhe koleg\u00ebt tan\u00eb i p\u00ebrdorin ato \u00e7do dit\u00eb me sukses. Metrikat jan\u00eb t\u00eb kuptueshme, grafikat jan\u00eb t\u00eb qarta, sistemi funksionon besuesh\u00ebm p\u00ebr disa vite dhe vazhdimisht shtohen teste dhe grafe t\u00eb reja: kur fusim nj\u00eb sh\u00ebrbim t\u00eb ri n\u00eb p\u00ebrdorim \u2014 shtojm\u00eb disa teste dhe grafika. N\u00eb rruge t\u00eb mbar\u00eb.<\/p>\n<h3>Dora n\u00eb puls \u2014 analitika e avancuar teknike<\/h3>\n<p>\nDeshira p\u00ebr t\u00eb marr\u00eb informacionin p\u00ebr problemet 'sa m\u00eb shpejt' na \u00e7oi n\u00eb eksperimente aktive me mjete t\u00eb thjeshta dhe t\u00eb qarta \u2014 pinba dhe xhprof.<\/p>\n<p>Pinba na d\u00ebrgonte statistika p\u00ebr shpejt\u00ebsin\u00eb e funksionimit t\u00eb pjes\u00ebve t\u00eb faqeve web n\u00eb PHP n\u00eb paketat UDP dhe mund\u00ebm t\u00eb shihnim n\u00eb m\u00ebnyr\u00eb live n\u00eb depozitat MySQL (me pinba ka motorin e vet MySQL p\u00ebr analitik\u00eb t\u00eb shpejt\u00eb t\u00eb ngjarjeve) nj\u00eb list\u00eb t\u00eb shkurt\u00ebr problemesh dhe t\u00eb reagojm\u00eb ndaj tyre. Xhprof n\u00eb m\u00ebnyr\u00eb automatike lejonte t\u00eb grumbullonim grafik\u00ebt e ekzekutimit t\u00eb faqeve PHP m\u00eb t\u00eb ngadalta te klient\u00ebt dhe t\u00eb analizojm\u00eb se \u00e7far\u00eb mund t\u00eb kishte shkaktuar k\u00ebt\u00eb \u2014 qet\u00ebsisht, duke pir\u00eb \u00e7aj ose ndonj\u00eb gj\u00eb m\u00eb t\u00eb fort\u00eb.<\/p>\n<p>Pak koh\u00eb m\u00eb par\u00eb, arsenali u pasurua me nj\u00eb motor tjet\u00ebr relativisht t\u00eb thjesht\u00eb dhe t\u00eb kuptuesh\u00ebm t\u00eb bazuar n\u00eb algoritmin e indeksimit t\u00eb invers dhe t\u00eb zbatuar mir\u00eb n\u00eb bibliotek\u00ebn legjendare Lucene \u2014 Elastic\/Kibana. Ideja e thjesht\u00eb e regjistrimit n\u00eb shum\u00eb rave dokumentesh n\u00eb indeksin e invers t\u00eb Lucene mbi baz\u00ebn e ngjarjeve n\u00eb log t\u00eb ndihmonte v\u00ebrtet.<\/p>\n<p>Pavar\u00ebsisht nga pamja teknike e vizualizimeve n\u00eb Kibana me konceptet e ul\u00ebta t\u00eb nivelit 'bucket' dhe gjuh\u00ebn e ri-shpikur t\u00eb algjebr\u00ebs relacional \u2014 mjeti na ndihmoi shum\u00eb n\u00eb k\u00ebto detyra:<\/p>\n<ul>\n<li>Sa gabime PHP kishte klienti Bitrix24 n\u00eb portalin p1 p\u00ebr or\u00ebn e kaluar dhe cilat? T\u00eb kuptoj dhe t\u00eb reagoj shpejt.<\/li>\n<li>Sa video-thirrje u b\u00ebn\u00eb n\u00eb portalet n\u00eb Gjermani p\u00ebr 24 or\u00ebt e fundit, me cil\u00ebsin\u00eb e cila ishte dhe a kishte ndonj\u00eb problem me kanalin\/rete?<\/li>\n<li>Sa mir\u00eb funksionon funksionaliteti sistemor (zgjerimi yn\u00eb n\u00eb C p\u00ebr PHP), i kompiluar nga burimet n\u00eb p\u00ebrdit\u00ebsimin e fundit t\u00eb sh\u00ebrbimit dhe i shp\u00ebrndar\u00eb p\u00ebr klient\u00ebt? A ka ndonj\u00eb segfault?<\/li>\n<li>A kan\u00eb t\u00eb dh\u00ebnat e klient\u00ebve vendosur n\u00eb memorjen PHP? A ka ndonj\u00eb gabim p\u00ebr tejkalimin e memories t\u00eb ndara p\u00ebr proceset: \u00abjasht\u00eb memories\u00bb? Gjej dhe neutralizo.<\/li>\n<\/ul>\n<p>\nJa nj\u00eb shembull konkret. Pavar\u00ebsisht testimit t\u00eb kujdessh\u00ebm dhe shum\u00ebnivel\u00ebsh, nj\u00eb klienti i \u00ebsht\u00eb shfaqur nj\u00eb gabim i bezdissh\u00ebm dhe befasues p\u00ebr nj\u00eb rast shum\u00eb t\u00eb pazakont\u00eb dhe t\u00eb dh\u00ebna hyr\u00ebse t\u00eb d\u00ebmtuara; alarmi \u00ebsht\u00eb ndezur dhe ka filluar procesi i riparimit t\u00eb shpejt\u00eb:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/8a802dba41b5d1a85c0dc41dfbf8b84e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nP\u00ebr m\u00eb tep\u00ebr, kibana lejon organizimin e njoftimeve p\u00ebr ngjarje t\u00eb caktuara dhe n\u00eb nj\u00eb koh\u00eb t\u00eb shkurt\u00ebr, ky mjet filloi t\u00eb p\u00ebrdorej nga dhjet\u00ebra punonj\u00ebs nga departamente t\u00eb ndryshme \u2014 nga mb\u00ebshtetje teknike dhe zhvillim deri n\u00eb QA.<\/p>\n<p>Aktiviteti i \u00e7do departamenti brenda kompanis\u00eb \u00ebsht\u00eb b\u00ebr\u00eb i leht\u00eb p\u00ebr t'u ndjekur dhe matur \u2014 n\u00eb vend t\u00eb analizave manuale t\u00eb skedave n\u00eb servera, mjafton t\u00eb konfiguroni nj\u00ebher\u00eb analiz\u00ebn e skedave dhe d\u00ebrgimin e tyre n\u00eb klasterin elastic, p\u00ebr t\u00eb shijuar, p\u00ebr shembull, pamjen n\u00eb panelin e kibana t\u00eb numrit t\u00eb koteleve me dy kryqe q\u00eb jan\u00eb printuar me printer 3-d p\u00ebr muajin e kaluar h\u00ebnor.<\/p>\n<h3>Analitika baz\u00eb e biznesit<\/h3>\n<p>\nT\u00eb gjith\u00eb e din\u00eb se shpesh analitika e biznesit n\u00eb kompanit\u00eb fillon me nj\u00eb p\u00ebrdorim jasht\u00ebzakonisht aktiv, po, po, Excel. Por, m\u00eb e r\u00ebnd\u00ebsishmja, \u00ebsht\u00eb q\u00eb ajo t\u00eb mos p\u00ebrfundoj\u00eb atje. Vaj i ri i zjarrit \u00ebsht\u00eb edhe Google Analytics n\u00eb re \u2014 n\u00eb t\u00eb mirat fillon t\u00eb besohet shpejt.<\/p>\n<p>N\u00eb kompanin\u00eb ton\u00eb q\u00eb po zhvillohet n\u00eb harmoni, jan\u00eb shfaqur k\u00ebtu e atje, \u00abprofet\u00eb\u00bb t\u00eb pun\u00ebs m\u00eb intensive me t\u00eb dh\u00ebna m\u00eb t\u00eb m\u00ebdha. K\u00ebrkesat p\u00ebr raporte m\u00eb t\u00eb thella dhe m\u00eb shum\u00eb dimensionale kan\u00eb filluar t\u00eb shfaqen rregullisht dhe me p\u00ebrpjekjet e djemve nga departamente t\u00eb ndryshme, disa koh\u00eb m\u00eb par\u00eb u organizua nj\u00eb zgjidhje e thjesht\u00eb dhe praktike \u2014 lidhja ClickHouse dhe PowerBI.<\/p>\n<p>Kjo zgjidhje fleksib\u00ebl ka ndihmuar mir\u00eb p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb, por gradualisht ka filluar t\u00eb kuptohet se ClickHouse nuk \u00ebsht\u00eb elastik dhe nuk mund t\u00eb abuzohet me t\u00eb.<\/p>\n<p>\u00cbsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb kuptohet mir\u00eb se ClickHouse, ashtu si Druid, si Vertica, si Amazon RedShift (i bazuar n\u00eb postgres), jan\u00eb motor\u00eb analitik\u00eb t\u00eb optimizuar p\u00ebr analitik\u00eb mjaft t\u00eb rehatshme (shuma, agregime, minimum-maksimum p\u00ebr kolon\u00eb dhe ndonj\u00eb gj\u00eb e vog\u00ebl p\u00ebr bashkimi), pasi jan\u00eb t\u00eb organizuara p\u00ebr ruajtjen efektive t\u00eb kolonave t\u00eb tabelave relacionales, ndryshe nga MySQL i njohur dhe bazat e tjera t\u00eb dh\u00ebnash (me orientim rresht).<\/p>\n<p>N\u00eb thelb, ClickHouse \u00ebsht\u00eb thjesht nj\u00eb \u2018baz\u00eb\u2019 m\u00eb e madhe e t\u00eb dh\u00ebnave, me nj\u00eb futje jo shum\u00eb praktike t\u00eb pikave (ashtu \u00ebsht\u00eb menduar, gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull), por me analitik\u00eb t\u00eb k\u00ebndshme dhe nj\u00eb grup funksionesh interesante dhe t\u00eb fuqishme p\u00ebr pun\u00ebn me t\u00eb dh\u00ebnat. Po, mund t\u00eb krijoni nj\u00eb klaster \u2014 por, e dini, q\u00eb t\u00eb godas\u00ebsh me nj\u00eb thik\u00eb nd\u00ebrtimore me mikroskopin nuk \u00ebsht\u00eb krejt\u00ebsisht e sakt\u00eb dhe ne filluam t\u00eb k\u00ebrkojm\u00eb zgjidhje t\u00eb tjera.<\/p>\n<h3>K\u00ebrkesa p\u00ebr python dhe analitik\u00ebt<\/h3>\n<p>\nN\u00eb kompanin\u00eb ton\u00eb ka shum\u00eb zhvillues q\u00eb shkruajn\u00eb kod pothuajse \u00e7do dit\u00eb p\u00ebr 10-20 vite n\u00eb PHP, JavaScript, C#, C\/C++, Java, Go, Rust, Python, Bash. Po ashtu shum\u00eb administratori t\u00eb sistemit me p\u00ebrvoj\u00eb, q\u00eb kan\u00eb p\u00ebrjetuar jo nj\u00eb katastrof\u00eb t\u00eb pabesueshme q\u00eb s'\u00ebsht\u00eb e p\u00ebrshtatshme p\u00ebr ligjet e statistik\u00ebs (p\u00ebr shembull, kur shumica e disqeve n\u00eb raid-10 shkat\u00ebrrohen nga goditjet e forta t\u00eb nj\u00eb shkrepjeje). N\u00eb k\u00ebto kushte p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb nuk ishte e qart\u00eb se \u00e7far\u00eb \u00ebsht\u00eb \u00abanalisti n\u00eb python\u00bb. Python \u00ebsht\u00eb si PHP, vet\u00ebm emri \u00ebsht\u00eb pak m\u00eb i gjat\u00eb dhe mbetjet e substancave q\u00eb ndryshojn\u00eb vet\u00ebdijen jan\u00eb pak m\u00eb t\u00eb vogla n\u00eb kodin burimor t\u00eb interpretuesit. Megjithat\u00eb, me krijimin e raporteve t\u00eb reja analitike, zhvilluesit e p\u00ebrvojsh\u00ebm filluan t\u00eb kuptojn\u00eb r\u00ebnd\u00ebsin\u00eb e specializimit t\u00eb ngusht\u00eb n\u00eb mjete si numpy, pandas, matplotlib, seaborn.<br \/>\nRoli vendimtar, ndoshta, u luajt nga p\u00ebrgjumja e papritur e punonj\u00ebsve nga kombinimi i fjal\u00ebve \u00abregresioni logjistik\u00bb dhe demonstruar nd\u00ebrtimin efektiv t\u00eb raporteve mbi t\u00eb dh\u00ebna voluminoze me ndihm\u00ebn e, po, pyspark.<\/p>\n<p>Apache Spark, paradigma e tij funksionale, n\u00eb t\u00eb cil\u00ebn p\u00ebrshtatet shum\u00eb mir\u00eb algebra relasionale, dhe mund\u00ebsit\u00eb e tij kan\u00eb b\u00ebr\u00eb nj\u00eb p\u00ebrshtypje t\u00eb madhe te zhvilluesit e zakonsh\u00ebm me MySQL, saq\u00eb nevoja p\u00ebr t\u00eb forcuar radh\u00ebt me analist\u00eb t\u00eb p\u00ebrvojsh\u00ebm u b\u00eb e qart\u00eb si dita.<\/p>\n<h3>P\u00ebrpjekjet e m\u00ebtejshme t\u00eb Apache Spark\/Hadoop p\u00ebr t\u00eb fluturuar dhe ajo q\u00eb nuk shkoi krejt si\u00e7 ishte parashikuar<\/h3>\n<p>\nMegjithat\u00eb, s\u00eb shpejti u kuptua se di\u00e7ka me Spark, duket, nuk ishte tamam ashtu si duhet, ose ndoshta duhej thjesht t\u00eb lahej m\u00eb mir\u00eb. N\u00ebse staku Hadoop\/MapReduce\/Lucene ishte zhvilluar nga programues mjaft t\u00eb aft\u00eb, e cila \u00ebsht\u00eb e qart\u00eb n\u00ebse shikon me v\u00ebmendje burimin n\u00eb Java ose idet\u00eb e Doug Cutting n\u00eb Lucene, at\u00ebher\u00eb Spark, papritur, \u00ebsht\u00eb shkruar n\u00eb nj\u00eb gjuh\u00eb shum\u00eb t\u00eb diskutueshme n\u00eb aspektin e praktik\u00ebs dhe q\u00eb aktualisht nuk po zhvillohet, gjuh\u00ebn eksotike Scala. Rr\u00ebzimet e rregullta t\u00eb llogaritjeve n\u00eb klasterin Spark p\u00ebr shkak t\u00eb menaxhimit jo t\u00eb logjiksh\u00ebm dhe jo shum\u00eb t\u00eb qart\u00eb t\u00eb memories p\u00ebr operacionet reduce (vijn\u00eb shum\u00eb \u00e7el\u00ebsa t\u00eb papritur) krijuan rreth tij nj\u00eb aur\u00eb t\u00eb di\u00e7kaje q\u00eb ka ende hap\u00ebsir\u00eb p\u00ebr t\u00eb ecur p\u00ebrpara. P\u00ebr m\u00eb tep\u00ebr, situat\u00ebn e p\u00ebrkeq\u00ebsonte numri i madh i porteve t\u00eb hapura t\u00eb \u00e7uditshme, skedar\u00ebve p\u00ebrkoh\u00ebshtar\u00eb q\u00eb rriteshin n\u00eb vende shum\u00eb t\u00eb pakuptueshme dhe var\u00ebsive t\u00eb jar-it \u2013 q\u00eb shkaktonte tek administrator\u00ebt sistemor\u00eb nj\u00eb ndjenj\u00eb t\u00eb njohur tashm\u00eb nga f\u00ebmij\u00ebria: nj\u00eb urrejtje t\u00eb thell\u00eb (ndoshta duhej t\u00eb lahej me sapun).<\/p>\n<p>Si rezultat, ne \"p\u00ebrsim\u00eb\" disa projekte t\u00eb brendshme analitike, duke p\u00ebrdorur aktivisht Apache Spark (p\u00ebrfshir\u00eb Spark Streaming, Spark SQL) dhe ekosistemin Hadoop (dhe e tjera). Megjithat\u00eb, me kalimin e koh\u00ebs, m\u00ebsuam t\u00eb \"p\u00ebrgatisim\" k\u00ebt\u00eb mjaft mir\u00eb dhe ta monitorojm\u00eb dhe \"ajo\" pothuajse nuk ndalonte m\u00eb papritur t\u00eb binte p\u00ebr shkak t\u00eb ndryshimit t\u00eb natyr\u00ebs s\u00eb t\u00eb dh\u00ebnave dhe disbalanc\u00ebs s\u00eb shp\u00ebrndarjes s\u00eb barabart\u00eb t\u00eb hash-it RDD, d\u00ebshira p\u00ebr t\u00eb marr\u00eb di\u00e7ka t\u00eb gatshme, t\u00eb azhurnuar dhe t\u00eb administruar diku n\u00eb cloud ishte duke u intensifikuar gjithnj\u00eb e m\u00eb shum\u00eb. N\u00eb k\u00ebt\u00eb koh\u00eb, provuam t\u00eb p\u00ebrdornim nj\u00eb paket\u00eb cloud t\u00eb gatshme nga Amazon Web Services - <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/emr\/\">EMR<\/a><\/noindex> dhe, m\u00eb pas, p\u00ebrpiqeshim t\u00eb zgjidhnim detyra tashm\u00eb n\u00eb t\u00eb. EMR \u00ebsht\u00eb nj\u00eb Apache Spark i p\u00ebrgatitur nga Amazon me softver shtes\u00eb nga ekosistema, pak si paketat e Cloudera\/Hortonworks.<\/p>\n<h3>\"Depo\" elastike p\u00ebr analitik\u00eb - nj\u00eb nevoj\u00eb urgjente<\/h3>\n<p>\nEksperienca e \"p\u00ebrgatitjes\" s\u00eb Hadoop\/Spark me djegie t\u00eb disa pjes\u00ebve t\u00eb trupit nuk ka kaluar kot. Nevoj\u00eb p\u00ebr krijimin e nj\u00eb depoje t\u00eb vetme t\u00eb besueshme dhe t\u00eb lir\u00eb, e cila do t\u00eb ishte e q\u00ebndrueshme ndaj aksidenteve harduerike dhe n\u00eb t\u00eb cil\u00ebn mund t\u00eb ruanim skedar\u00eb n\u00eb formate t\u00eb ndryshme nga sisteme t\u00eb ndryshme dhe t\u00eb b\u00ebnim seleksione efektive dhe n\u00eb nj\u00eb koh\u00eb t\u00eb arsyeshme p\u00ebr raporte p\u00ebr k\u00ebto t\u00eb dh\u00ebna, u b\u00eb gjithnj\u00eb e m\u00eb e qart\u00eb.<\/p>\n<p>Po ashtu, ne d\u00ebshirojm\u00eb q\u00eb p\u00ebrdit\u00ebsimi i softverit t\u00eb k\u00ebsaj platforme t\u00eb mos kthehet n\u00eb nj\u00eb makth nate festash me leximin e sh\u00ebnimeve dhe analizimin e kilometrave t\u00eb log\u00ebve t\u00eb detajuar t\u00eb pun\u00ebs s\u00eb klasterit me ndihm\u00ebn e Spark History Server dhe nj\u00eb lup\u00eb me ndri\u00e7im. D\u00ebshira ishte t\u00eb kishim nj\u00eb mjet t\u00eb thjesht\u00eb dhe t\u00eb qart\u00eb q\u00eb nuk k\u00ebrkonzh pak q\u00eb t\u00eb mendohet, n\u00ebse zhvilluesi s'mund t\u00eb ekzekutoj\u00eb nj\u00eb k\u00ebrkes\u00eb standarde MapReduce p\u00ebr shkak t\u00eb humbjes s\u00eb t\u00eb dh\u00ebnave nga diga e pun\u00ebs gjat\u00eb ndarjes s\u00eb t\u00eb dh\u00ebnave p\u00ebr shkak t\u00eb nj\u00eb algoritmi t\u00eb pap\u00ebrshtatsh\u00ebm t\u00eb ndarjes n\u00eb fillim.<\/p>\n<h3>A mundet Amazon S3 t\u00eb jet\u00eb nj\u00eb kandidat p\u00ebr DataLake?<\/h3>\n<p>\nP\u00ebrvoja me Hadoop\/MapReduce na ka m\u00ebsuar se na nevojitet nj\u00eb sistem skedari t\u00eb besuesh\u00ebm dhe t\u00eb shkall\u00ebzuesh\u00ebm dhe pun\u00ebtor\u00eb t\u00eb shkall\u00ebzuar mbi t\u00eb, \"duke ardhur\" m\u00eb af\u00ebr t\u00eb dh\u00ebnave, q\u00eb t\u00eb mos na duhet t\u00eb d\u00ebrgojm\u00eb t\u00eb dh\u00ebnat n\u00ebp\u00ebr rrjet. Pun\u00ebtor\u00ebt duhet t\u00eb jen\u00eb n\u00eb gjendje t\u00eb lexojn\u00eb t\u00eb dh\u00ebna n\u00eb formate t\u00eb ndryshme, por, idealisht, pa lexuar informacion t\u00eb panevojsh\u00ebm dhe q\u00eb t\u00eb jet\u00eb e mundur t\u00eb ruani t\u00eb dh\u00ebnat paraprakisht n\u00eb formate t\u00eb p\u00ebrshtatshme p\u00ebr pun\u00ebtor\u00ebt.<\/p>\n<p><b>Nj\u00eb her\u00eb tjet\u00ebr \u2014 ideja kryesore.<\/b> Nuk ka d\u00ebshir\u00eb t\u00eb \"ngarkohet\" nj\u00eb sasi e madhe t\u00eb dh\u00ebnash n\u00eb nj\u00eb motor analitik t\u00eb centralizuar q\u00eb do t\u00eb mbytet her\u00ebt apo von\u00eb dhe do t\u00eb duhet t\u00eb ndahet n\u00eb m\u00ebnyr\u00eb t\u00eb \u00e7rregullt. D\u00ebshira \u00ebsht\u00eb t\u00eb ruhen skedar\u00eb, thjesht skedar\u00eb, n\u00eb nj\u00eb format t\u00eb kuptuesh\u00ebm dhe t\u00eb b\u00ebhen k\u00ebrkesa efektive analitike p\u00ebrmes mjeteve t\u00eb ndryshme, por t\u00eb kuptueshme. Dhe numri i skedar\u00ebve n\u00eb formate t\u00eb ndryshme do t\u00eb rritet gjithnj\u00eb e m\u00eb shum\u00eb. Dhe m\u00eb mir\u00eb t\u00eb ndahet jo motori, por t\u00eb dh\u00ebnat origjinale. K\u00ebshtu, vendos\u00ebm se na nevojitet nj\u00eb DataLake i shkall\u00ebzuesh\u00ebm dhe universale...<\/p>\n<p>\u00c7far\u00eb ndodh n\u00ebse ruajm\u00eb skedar\u00ebt n\u00eb nj\u00eb depo t\u00eb njohur dhe t\u00eb shkall\u00ebzuar, Amazon S3, pa u marr\u00eb me p\u00ebrgatitjen e vet\u00eb Hadoop-it?<\/p>\n<p>E qart\u00eb, t\u00eb dh\u00ebnat personale \"nuk lejohet\", por a ka ndonj\u00eb mund\u00ebsi q\u00eb, n\u00ebse i nxjerrim ato dhe \"t\u00eb pastrojm\u00eb\" efikasitetin?<\/p>\n<h3>Ekosistemi analitik i Amazon Web Services m\u00eb klasteret e m\u00ebdhenj t\u00eb t\u00eb dh\u00ebnave - n\u00eb fjal\u00eb shum\u00eb t\u00eb thjeshta<\/h3>\n<p>\nSipas eksperienc\u00ebs son\u00eb me AWS, atje p\u00ebrdoret prej koh\u00ebsh dhe aktivisht Apache Hadoop\/MapReduce n\u00ebn sos gjithnj\u00eb t\u00eb ndryshme, p\u00ebr shembull n\u00eb sh\u00ebrbimin DataPipeline (e kam xhelozi p\u00ebr koleg\u00ebt, pse ata din\u00eb ta p\u00ebrgatitin si\u00e7 duhet). K\u00ebtu ne vendos\u00ebm backup nga sh\u00ebrbime t\u00eb ndryshme nga tabelat DynamoDB:<br \/>\n<img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/058dc54ed032a7bf3e9e129646202440.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDhe ato kryhen rregullisht n\u00eb klaster\u00ebt e integruar Hadoop\/MapReduce si or\u00eb tashm\u00eb p\u00ebr disa vite. \"E vendosa dhe e harrova\":<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/a6569da8cafdb96c63250bb32bf51704.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nGjithashtu, \u00ebsht\u00eb e mundur t\u00eb angazhohesh n\u00eb data\u015ftaniz\u00ebm, duke ngritur Jupiter-notebooks p\u00ebr analist\u00ebt n\u00eb cloud dhe duke p\u00ebrdorur p\u00ebr trajnimin dhe shp\u00ebrndarjen e modeleve AI sh\u00ebrbimin AWS SageMaker. Ja se si duket kjo p\u00ebr ne:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/c825d979c9278a8edf8e1e747ef6def8.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPo, mund t\u00eb ngresh nj\u00eb laptop analitik n\u00eb cloud dhe ta lidh\u00ebsh at\u00eb me nj\u00eb klas\u00ebr Hadoop\/Spark, t\u00eb b\u00ebsh llogaritjet dhe m\u00eb pas t\u00eb \"marr\u00ebsh\" t\u00eb gjitha.<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/73cea18c54d2a9ce8d0441463991808b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nN\u00eb t\u00eb v\u00ebrtet\u00eb, \u00ebsht\u00eb shum\u00eb e p\u00ebrshtatshme p\u00ebr projekte t\u00eb ve\u00e7anta analitike dhe p\u00ebr disa prej tyre kemi p\u00ebrdorur me sukses sh\u00ebrbimin EMR p\u00ebr llogaritje dhe analiz\u00eb t\u00eb m\u00ebdha. Si \u00ebsht\u00eb situata p\u00ebr nj\u00eb zgjidhje sistematike p\u00ebr DataLake, a do t'ia dalim? N\u00eb at\u00eb moment ne ishim n\u00eb prag t\u00eb shpres\u00ebs dhe d\u00ebshp\u00ebrimit dhe vazhduam k\u00ebrkimin.<\/p>\n<h3>AWS Glue \u2014 nj\u00eb Apache Spark i paketuar n\u00eb m\u00ebnyr\u00eb t\u00eb sakt\u00eb \"n\u00eb steroide\"<\/h3>\n<p>\nDoli se AWS ka nj\u00eb version \"t\u00eb vetin\" t\u00eb stack-ut \"Hive\/Pig\/Spark\". Roli i Hive, pra, katalogu i skedar\u00ebve dhe llojeve t\u00eb tyre n\u00eb DataLake, e kryen sh\u00ebrbimi \"Data catalog\", i cili nuk fshihet nga p\u00ebrputhshm\u00ebria e saj me formatin Apache Hive. N\u00eb k\u00ebt\u00eb sh\u00ebrbim duhet t\u00eb shtosh informacionin se ku ndodhen skedar\u00ebt dhe n\u00eb cilin format jan\u00eb. T\u00eb dh\u00ebnat mund t\u00eb jen\u00eb jo vet\u00ebm n\u00eb s3, por edhe n\u00eb nj\u00eb baz\u00eb t\u00eb dh\u00ebnash, por p\u00ebr k\u00ebt\u00eb nuk do t\u00eb flasim n\u00eb k\u00ebt\u00eb postim. Ja si \u00ebsht\u00eb organizuar katalogu i t\u00eb dh\u00ebnave DataLake tek ne:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/adb45d09698fdacbf41c86bbadde8bb2.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSkedar\u00ebt jan\u00eb regjistruar, shk\u00eblqyer. N\u00ebse skedar\u00ebt jan\u00eb p\u00ebrdit\u00ebsuar \u2014 ekzekutojm\u00eb manualisht ose sipas nj\u00eb orari crawlers, t\u00eb cil\u00ebt do t\u00eb p\u00ebrdit\u00ebsojn\u00eb informacionin dhe do ta ruajn\u00eb. M\u00eb pas, t\u00eb dh\u00ebnat nga liqeni mund t\u00eb p\u00ebrpunohen dhe rezultatet mund t\u00eb d\u00ebrgohen diku. N\u00eb rastin m\u00eb t\u00eb thjesht\u00eb \u2014 d\u00ebrgojm\u00eb gjithashtu n\u00eb s3. P\u00ebrpunimi i t\u00eb dh\u00ebnave mund t\u00eb b\u00ebhet kudo, por rekomandohet t\u00eb konfigurohet procesi i p\u00ebrpunimit n\u00eb nj\u00eb klas\u00ebr Apache Spark duke p\u00ebrdorur mund\u00ebsit\u00eb e avancuara p\u00ebrmes API AWS Glue. N\u00eb t\u00eb v\u00ebrtet\u00eb, mund t\u00eb marr\u00ebsh kodin tradicional dhe t\u00eb njohur n\u00eb python duke p\u00ebrdorur bibliotek\u00ebn pyspark dhe ta konfiguroni p\u00ebr ekzekutim n\u00eb N noda t\u00eb klasit me nj\u00ebfar\u00eb fuqie me monitorim, pa u nxjerr\u00eb n\u00eb detaje t\u00eb Hadoop-it dhe pa u marr\u00eb me konfliktet e var\u00ebsive.<\/p>\n<p><b>Nj\u00eb her\u00eb tjet\u00ebr \u2014 ide e thjesht\u00eb.<\/b> Nuk ka nevoj\u00eb t\u00eb konfigurosh Apache Spark, duhet vet\u00ebm t\u00eb shkruash kod n\u00eb python p\u00ebr pyspark, ta testosh at\u00eb lokal n\u00eb desktopin t\u00ebnd dhe m\u00eb pas ta ekzekutosh n\u00eb nj\u00eb klas\u00ebr t\u00eb madh n\u00eb cloud, duke specifikuar ku ndodhen t\u00eb dh\u00ebnat origjinale dhe ku t\u00eb vendoset rezultati. Nsometimes, kjo \u00ebsht\u00eb e nevojshme dhe e dobishme dhe ja si e kemi konfiguruar tek ne:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/dc03181573bb3f5cfcc3a8760bc7e07b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPra, n\u00ebse nevojitet t\u00eb b\u00ebsh ndonj\u00eb llogaritje n\u00eb klastri Spark me t\u00eb dh\u00ebna nga s3 \u2014 shkruaj kodin n\u00eb python\/pyspark, testojeni dhe p\u00ebrpara n\u00eb cloud.<\/p>\n<p>Dhe \u00e7far\u00eb me orkestrimin? \u00c7far\u00eb n\u00ebse detyra d\u00ebshton dhe humbet? Po, sugjerohet t\u00eb krijosh nj\u00eb pipeline t\u00eb bukur n\u00eb stilin e Apache Pig dhe madje e provuam, por vendos\u00ebm p\u00ebr momentin t\u00eb p\u00ebrdorim orkestrimin ton\u00eb t\u00eb thell\u00eb t\u00eb personalizuar n\u00eb PHP dhe JavaScript (e kuptoj, ndjen nj\u00eb disonanc\u00eb kognitive, por funksionon, p\u00ebr vite me radh\u00eb dhe pa gabime).<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/e00ed2047c5c6492e36fccc82e7278a3.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Formati i skedar\u00ebve t\u00eb ruajtur n\u00eb liqen \u2014 \u00e7el\u00ebsi p\u00ebr performanc\u00eb<\/h3>\n<p>\n\u00c7\u00ebshtje shum\u00eb, shum\u00eb e r\u00ebnd\u00ebsishme t\u00eb kuptohet edhe dy pika ky\u00e7e. P\u00ebr t\u00eb siguruar q\u00eb k\u00ebrkesat p\u00ebr t\u00eb dh\u00ebnat e skedar\u00ebve n\u00eb liqen t\u00eb realizohen sa m\u00eb shpejt dhe q\u00eb performanca t\u00eb mos degradohet gjat\u00eb shtimit t\u00eb informacionit t\u00eb ri, duhet:<\/p>\n<ul>\n<li>Kolonat e skedar\u00ebve t\u00eb ruhen ve\u00e7mas (nuk ka nevoj\u00eb t\u00eb lexosh t\u00eb gjitha rreshtat p\u00ebr t\u00eb kuptuar se \u00e7far\u00eb ka n\u00eb kolona). P\u00ebr k\u00ebt\u00eb mor\u00ebm formatin parquet me kompresim.<\/li>\n<li>Shum\u00eb e r\u00ebnd\u00ebsishme \u00ebsht\u00eb t\u00eb ndahen skedar\u00ebt n\u00eb dosje n\u00eb stilin: gjuh\u00eb, vit, muaj, dit\u00eb, jav\u00eb. Motor\u00ebt q\u00eb e kuptojn\u00eb k\u00ebt\u00eb lloj ndarjeje, do t\u00eb shikojn\u00eb vet\u00ebm n\u00eb dosjet e duhura, pa u marr\u00eb me t\u00eb gjitha t\u00eb dh\u00ebnat nj\u00eb p\u00ebr nj\u00eb.<\/li>\n<\/ul>\n<p>\nN\u00eb thelb, n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb, ju ofroni t\u00eb dh\u00ebnat origjinale n\u00eb m\u00ebnyr\u00ebn m\u00eb efektive p\u00ebr motor\u00ebt analitik\u00eb q\u00eb jan\u00eb t\u00eb aft\u00eb t\u00eb hyjn\u00eb dhe t\u00eb lexojn\u00eb selektivisht vet\u00ebm kolonat e nevojshme nga skedar\u00ebt. Nuk ka nevoj\u00eb t\u00eb \"derdhni\" t\u00eb dh\u00ebnat diku (ruajtja vet\u00ebm do t\u00eb bjer\u00eb) \u2014 thjesht vendosni menj\u00ebher\u00eb n\u00eb sistemin e skedar\u00ebve n\u00eb formatin e duhur. Natyrisht, duhet t\u00eb jet\u00eb e qart\u00eb se ruajtja e nj\u00eb skedari t\u00eb madh csv n\u00eb DataLake, t\u00eb cilin duhet ta lexoni fillimisht rresht p\u00ebr rresht nga klastri p\u00ebr t\u00eb nxjerr\u00eb kolonat \u2014 nuk \u00ebsht\u00eb shum\u00eb e arsyeshme. Rishikoni dy pikat e m\u00ebsip\u00ebrme n\u00ebse ende nuk kuptoni pse \u00ebsht\u00eb e gjith\u00eb kjo.<\/p>\n<h3>AWS Athena \u2014 \"djalli\" n\u00eb kutin\u00eb e duhanit<\/h3>\n<p>\nDhe k\u00ebtu, duke krijuar liqenin, ne, si t\u00eb thuash, gjet\u00ebm Amazon Athena. Papritmas u zbulua se duke e organizuar me kujdes skedar\u00ebt tan\u00eb t\u00eb log\u00ebve t\u00eb m\u00ebdha n\u00eb formatin e duhur (parquet) sipas ndarjeve - mund t\u00eb b\u00ebjm\u00eb seleksione shum\u00eb informative dhe t\u00eb nd\u00ebrtojm\u00eb raporte SHPEJT, pa pasur nevoj\u00eb p\u00ebr klastra Apache Spark\/Glue.<\/p>\n<p>Motori Athena, q\u00eb punon n\u00eb t\u00eb dh\u00ebnat n\u00eb s3, bazohet n\u00eb legjendarin <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/big-data\/what-is-presto\/\">Presto<\/a><\/noindex> - p\u00ebrfaq\u00ebsuesi i familjes MPP (procesim masiv paralel) t\u00eb qasjeve p\u00ebr p\u00ebrpunimin e t\u00eb dh\u00ebnave, merr t\u00eb dh\u00ebnat aty ku jan\u00eb, nga s3 dhe Hadoop deri te Cassandra dhe skedar\u00eb tekstual\u00eb t\u00eb thjesht\u00eb. Thjesht k\u00ebrkoni q\u00eb Athena t\u00eb ekzekutoj\u00eb nj\u00eb SQL-query, dhe m\u00eb pas gjith\u00e7ka \"punon shpejt dhe vet\u00eb\". \u00cbsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb theksohet se Athena \u00ebsht\u00eb \"e zgjuar\", hyn vet\u00ebm n\u00eb dosjet e ndara t\u00eb nevojshme dhe lexon vet\u00ebm kolonat e nevojshme n\u00eb query.<\/p>\n<p>K\u00ebrkesat ndaj Athena po ashtu jan\u00eb interesante. Ne paguajm\u00eb p\u00ebr <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/athena\/pricing\/\">v\u00ebllimin e t\u00eb dh\u00ebnave t\u00eb skanuara<\/a><\/noindex>. Pra, jo p\u00ebr numrin e makinave n\u00eb klaster p\u00ebr minut\u00eb, por\u2026 p\u00ebr t\u00eb dh\u00ebnat q\u00eb jan\u00eb skanuar n\u00eb 100-500 makina, t\u00eb cilat jan\u00eb v\u00ebrtet t\u00eb nevojshme p\u00ebr plot\u00ebsimin e k\u00ebrkes\u00ebs.<\/p>\n<p>Dhe duke k\u00ebrkuar vet\u00ebm kolonat e nevojshme nga dosjet e sharduara si\u00e7 duhet, u tregua se sh\u00ebrbimi Athena na kostonte disa dhjetra dollar\u00eb n\u00eb muaj. E shk\u00eblqyer, thuajse falas, krahasuar me analytik\u00ebn n\u00eb klastere!<\/p>\n<p>Ja, si i shardojm\u00eb t\u00eb dh\u00ebnat tona n\u00eb S3:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/00bd9ae48c1cd13f3c4f7d32692c9209.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSi rezultat, p\u00ebr nj\u00eb koh\u00eb t\u00eb shkurt\u00ebr, n\u00eb kompani, nj\u00eb gam\u00eb e gjer\u00eb ndarjesh, nga siguria informacionit deri te analiza, filluan t\u00eb b\u00ebjn\u00eb k\u00ebrkesa ndaj Athena dhe t\u00eb marrin p\u00ebrgjigje t\u00eb dobishme nga \"t\u00eb dh\u00ebnat e m\u00ebdha\" shpejt, brenda sekondave p\u00ebr periudha relativisht t\u00eb gjata: muaj, gjysm\u00ebviti etj.<\/p>\n<p>Por ne shkuam p\u00ebrtej dhe filluam t\u00eb k\u00ebrkojm\u00eb p\u00ebrgjigje n\u00eb re <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.aws.amazon.com\/athena\/latest\/ug\/connect-with-odbc.html\">n\u00ebp\u00ebrmjet ODBC driver-it.<\/a><\/noindex>: analisti n\u00eb konsol\u00ebn e njohur shkruan nj\u00eb k\u00ebrkes\u00eb SQL, e cila n\u00eb 100-500 makina \"p\u00ebr pak\" k\u00ebrkon t\u00eb dh\u00ebnat n\u00eb S3 dhe kthen p\u00ebrgjigjen zakonisht brenda disa sekondash. E rehatshme. Dhe shpejt. Akoma nuk e besojm\u00eb.<\/p>\n<p>N\u00eb fund, duke vendosur t\u00eb ruajm\u00eb t\u00eb dh\u00ebnat n\u00eb S3, n\u00eb nj\u00eb format kolonor efektiv dhe me shardim t\u00eb arsyesh\u00ebm t\u00eb t\u00eb dh\u00ebnave n\u00eb dosje\u2026 ne nd\u00ebrtuam DataLake dhe nj\u00eb motor analitik t\u00eb shpejt\u00eb dhe t\u00eb lir\u00eb \u2014 falas. Dhe ai b\u00ebri shum\u00eb popullor n\u00eb kompani, pasi kupton SQL dhe punon shum\u00eb m\u00eb shpejt se p\u00ebrmes nisjes\/ststopjeve\/konfigurimeve t\u00eb klastereve. \"N\u00ebse rezultati \u00ebsht\u00eb i nj\u00ebjt\u00eb, pse t\u00eb paguash m\u00eb shum\u00eb?\"<\/p>\n<p>Nj\u00eb k\u00ebrkes\u00eb ndaj Athena duket p\u00ebraf\u00ebrsisht k\u00ebshtu. N\u00ebse d\u00ebshiron, natyrisht, mund t\u00eb formosh nj\u00eb k\u00ebrkes\u00eb SQL mjaft <noindex><a rel=\"nofollow\" href=\"https:\/\/prestodb.io\/docs\/0.172\/index.html\">t\u00eb komplikuar dhe shum\u00ebfaq\u00ebshe<\/a><\/noindex>, por ne do t\u00eb kufizohemi n\u00eb nj\u00eb grup t\u00eb thjesht\u00eb. T\u00eb shohim se cilat kode p\u00ebrgjigjesh kishte klienti disa jav\u00eb m\u00eb par\u00eb n\u00eb log-\u00ebt e pun\u00ebs s\u00eb serverit web dhe t\u00eb sigurohemi q\u00eb nuk ka gabime:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake t\u00eb shk\u00eblqyer dhe t\u00eb p\u00ebrballuesh\u00ebm dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/30028991467b9e52f597faa617d374b5.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>P\u00ebrfundimet<\/h3>\n<p>\nPas nj\u00eb rrug\u00ebtimi q\u00eb nuk ishte i gjat\u00eb, por i dhimbsh\u00ebm, duke vler\u00ebsuar vazhdimisht rrezikun, nivelin e kompleksitetit dhe koston e mb\u00ebshtetjes, ne gjendem zgjidhjen p\u00ebr DataLake dhe analitik\u00ebn q\u00eb na g\u00ebzon vazhdimisht me shpejt\u00ebsin\u00eb dhe kostot e pron\u00ebsis\u00eb.<\/p>\n<p>Doli se ishte plot\u00ebsisht e mundur t\u00eb nd\u00ebrtohet nj\u00eb DataLake efektiv, t\u00eb shpejt\u00eb dhe t\u00eb lir\u00eb p\u00ebr nevojat e ndarjeve t\u00eb ndryshme t\u00eb kompanis\u00eb \u2014 madje edhe p\u00ebr zhvilluesit q\u00eb nuk kan\u00eb punuar kurr\u00eb si arkitekt\u00eb dhe nuk din\u00eb t\u00eb vizatojn\u00eb katror\u00eb mbi katror\u00eb me th\u00ebrrime dhe q\u00eb din\u00eb 50 terma nga ekosistemi Hadoop.<\/p>\n<p>N\u00eb fillim t\u00eb rrug\u00ebtimit, koka m\u00eb doli nga shum\u00eb kafshat\u00eb t\u00eb \u00e7uditshme t\u00eb softuer\u00ebve t\u00eb hapur dhe t\u00eb mbyllur dhe kuptimi i barr\u00ebs s\u00eb p\u00ebrgjegj\u00ebsis\u00eb p\u00ebr pasardh\u00ebsit. Thjesht filloni t\u00eb nd\u00ebrtoni DataLake tuaj nga mjete t\u00eb thjeshta: nagios\/munin -&gt; elastic\/kibana -&gt; Hadoop\/Spark\/s3 \u2026, duke mbledhur feedback dhe duke e kuptuar thell\u00ebsisht fizik\u00ebn e proceseve q\u00eb ndodhin. \u00c7do gj\u00eb e komplikuar dhe e turbullt \u2014 l\u00ebreni armik\u00ebve dhe konkurent\u00ebve.<\/p>\n<p>N\u00ebse nuk doni n\u00eb re dhe e doni t\u00eb mbani, p\u00ebrdit\u00ebsoni dhe aplikoni patch-e n\u00eb projekte t\u00eb hapura, mund t\u00eb nd\u00ebrtoni nj\u00eb skem\u00eb t\u00eb ngjashme me ton\u00ebn lokal, n\u00eb makina t\u00eb lira zyrtare me Hadoop dhe Presto sip\u00ebr. E r\u00ebnd\u00ebsishmja \u2014 mos ndaloni dhe ecni p\u00ebrpara, llogaritni, k\u00ebrkoni zgjidhje t\u00eb thjeshta dhe \u00e7do gj\u00eb do t\u00eb funksionoj\u00eb! Fat t\u00eb mir\u00eb t\u00eb gjith\u00ebve dhe deri n\u00eb takime t\u00eb reja!<br \/>\n<br \/>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/bitrix\/blog\/516374\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041c\u044b \u0436\u0438\u0432\u0435\u043c \u0432 \u0443\u0434\u0438\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f, \u043a\u043e\u0433\u0434\u0430 \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0441\u0442\u044b\u043a\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0433\u043e\u0442\u043e\u0432\u044b\u0445 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438\u0445 \u0441 \u00ab\u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u043c \u0441\u043e\u0437\u043d\u0430\u043d\u0438\u0435\u043c\u00bb \u043f\u043e \u0441\u043e\u0432\u0435\u0442\u0430\u043c stackoverflow, \u043d\u0435 \u0432\u043d\u0438\u043a\u0430\u044f \u0432 \u00ab\u043c\u043d\u043e\u0433\u043e\u0431\u0443\u043a\u0432\u00bb, \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432 \u043a\u043e\u043c\u043c\u0435\u0440\u0447\u0435\u0441\u043a\u0443\u044e \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u044e. \u0410 \u043a\u043e\u0433\u0434\u0430 \u043d\u0443\u0436\u043d\u043e \u0431\u0443\u0434\u0435\u0442 \u043e\u0431\u043d\u043e\u0432\u043b\u044f\u0442\u044c\u0441\u044f\/\u0440\u0430\u0441\u0448\u0438\u0440\u044f\u0442\u044c\u0441\u044f \u0438\u043b\u0438 \u043a\u0442\u043e-\u0442\u043e \u0441\u043b\u0443\u0447\u0430\u0439\u043d\u043e \u043f\u0435\u0440\u0435\u0437\u0430\u0433\u0440\u0443\u0437\u0438\u0442 \u043f\u0430\u0440\u0443 \u043c\u0430\u0448\u0438\u043d \u2014 \u043e\u0441\u043e\u0437\u043d\u0430\u0442\u044c, \u0447\u0442\u043e \u043d\u0430\u0447\u0430\u043b\u0441\u044f \u043a\u0430\u043a\u043e\u0439-\u0442\u043e \u043d\u0430\u0432\u044f\u0437\u0447\u0438\u0432\u044b\u0439 \u0434\u0443\u0440\u043d\u043e\u0439 \u0441\u043e\u043d \u043d\u0430\u044f\u0432\u0443, \u0432\u0441\u0435 \u0440\u0435\u0437\u043a\u043e \u0443\u0441\u043b\u043e\u0436\u043d\u0438\u043b\u043e\u0441\u044c \u0434\u043e [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":92509,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-92508","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.0.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041c\u044b \u0436\u0438\u0432\u0435\u043c \u0432 \u0443\u0434\u0438\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f, \u043a\u043e\u0433\u0434\u0430 \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0441\u0442\u044b\u043a\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0433\u043e\u0442\u043e\u0432\u044b\u0445 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438\u0445 \u0441 \u00ab\u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u043c \u0441\u043e\u0437\u043d\u0430\u043d\u0438\u0435\u043c\u00bb \u043f\u043e \u0441\u043e\u0432\u0435\u0442\u0430\u043c stackoverflow, \u043d\u0435 \u0432\u043d\u0438\u043a\u0430\u044f \u0432 \u00ab\u043c\u043d\u043e\u0433\u043e\u0431\u0443\u043a\u0432\u00bb, \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.0.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041a\u0430\u043a \u043c\u044b \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u0430\u043b\u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0435 \u0438 \u043d\u0435\u0434\u043e\u0440\u043e\u0433\u043e\u0435 DataLake \u0438 \u043f\u043e\u0447\u0435\u043c\u0443 \u0438\u043c\u0435\u043d\u043d\u043e \u0442\u0430\u043a | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041c\u044b \u0436\u0438\u0432\u0435\u043c \u0432 \u0443\u0434\u0438\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f, \u043a\u043e\u0433\u0434\u0430 \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0441\u0442\u044b\u043a\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0433\u043e\u0442\u043e\u0432\u044b\u0445 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438\u0445 \u0441 \u00ab\u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u043c \u0441\u043e\u0437\u043d\u0430\u043d\u0438\u0435\u043c\u00bb \u043f\u043e \u0441\u043e\u0432\u0435\u0442\u0430\u043c stackoverflow, \u043d\u0435 \u0432\u043d\u0438\u043a\u0430\u044f \u0432 \u00ab\u043c\u043d\u043e\u0433\u043e\u0431\u0443\u043a\u0432\u00bb, \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-08-28T05:42:10+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-08-28T05:42:10+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Si e organizuam DataLake-in me efikasitet t\u00eb lart\u00eb dhe kosto t\u00eb ul\u00ebt dhe pse k\u00ebshtu | ProHoster","description":"Ne jemi n\u00eb nj\u00eb koh\u00eb t\u00eb jasht\u00ebzakonshme, kur \u00ebsht\u00eb e mundur t\u00eb lidhni shpejt dhe leht\u00ebsisht disa mjete t\u00eb hapura, t'i konfiguroni ato me \"nderprerje mendore\" sipas k\u00ebshillave t\u00eb stackoverflow, pa u thelluar n\u00eb \"shum\u00eb shkronja\", t\u00eb nisni.","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041a\u0430\u043a \u043c\u044b \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u0430\u043b\u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0435 \u0438 \u043d\u0435\u0434\u043e\u0440\u043e\u0433\u043e\u0435 DataLake \u0438 \u043f\u043e\u0447\u0435\u043c\u0443 \u0438\u043c\u0435\u043d\u043d\u043e \u0442\u0430\u043a | ProHoster","og:description":"\u041c\u044b \u0436\u0438\u0432\u0435\u043c \u0432 \u0443\u0434\u0438\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f, \u043a\u043e\u0433\u0434\u0430 \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0441\u0442\u044b\u043a\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0433\u043e\u0442\u043e\u0432\u044b\u0445 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438\u0445 \u0441 \u00ab\u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u043c \u0441\u043e\u0437\u043d\u0430\u043d\u0438\u0435\u043c\u00bb \u043f\u043e \u0441\u043e\u0432\u0435\u0442\u0430\u043c stackoverflow, \u043d\u0435 \u0432\u043d\u0438\u043a\u0430\u044f \u0432 \u00ab\u043c\u043d\u043e\u0433\u043e\u0431\u0443\u043a\u0432\u00bb, \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-08-28T05:42:10+00:00","article:modified_time":"2020-08-28T05:42:10+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"92508","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 12:07:39","updated":"2022-10-01 09:50:53","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/92508","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=92508"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/92508\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/92509"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=92508"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=92508"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=92508"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}