{"id":92508,"date":"2020-08-28T07:42:10","date_gmt":"2020-08-28T05:42:10","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak"},"modified":"2020-08-28T07:42:10","modified_gmt":"2020-08-28T05:42:10","slug":"kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak","title":{"rendered":"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p>Ne jetojm\u00eb n\u00eb nj\u00eb koh\u00eb t\u00eb jasht\u00ebzakonshme, kur mund t\u00eb lidhen shpejt dhe leht\u00ebsisht disa mjete t\u00eb gatshme t\u00eb hapura, t'i konfigurojm\u00eb ato me \"dijeni t\u00eb \u00e7aktivizuar\" sipas k\u00ebshillave nga stackoverflow, pa u shqet\u00ebsuar p\u00ebr \"shum\u00eb shkronja\", dhe ta nisni at\u00eb n\u00eb p\u00ebrdorim tregtar. Por kur do t\u00eb vij\u00eb koha t\u00eb p\u00ebrdit\u00ebsohet\/zgjerohet ose ndokush rast\u00ebsisht t\u00eb rindez\u00eb disa makina \u2014 do t\u00eb kuptoni se ka filluar nj\u00eb makth i ngjash\u00ebm me nj\u00eb \u00ebnd\u00ebrr, gjith\u00e7ka \u00ebsht\u00eb p\u00ebrtej njohjes, rrug\u00eb e kthimit s'ka, e ardhmja \u00ebsht\u00eb e paqart\u00eb dhe m\u00eb e sigurt, p\u00ebrve\u00e7 programimit, t\u00eb rritni blet\u00eb dhe t\u00eb b\u00ebni djath\u00eb.<\/p>\n<p>Nuk \u00ebsht\u00eb rast\u00ebsi q\u00eb koleg\u00ebt m\u00eb t\u00eb p\u00ebrvojsh\u00ebm, me koke t\u00eb bardh\u00eb nga gabimet e shumta, duke v\u00ebzhguar shp\u00ebrthimin e pabesuesh\u00ebm t\u00eb grupeve t\u00eb \"kontejner\u00ebve\" n\u00eb \"kubik\u00eb\" n\u00eb dhjetra servera n\u00eb \"gjuh\u00eb t\u00eb mod\u00ebs\" me mb\u00ebshtetje t\u00eb integruar p\u00ebr hyrje\/ dalje asinkrone dhe jo-bllokuese \u2014 buz\u00ebqeshin modestisht. Dhe n\u00eb heshtje vazhdojn\u00eb t\u00eb rishikojn\u00eb \"man ps\", p\u00ebrpiqen deri n\u00eb gjakderdhje nga syt\u00eb n\u00eb kodin burimor t\u00eb \"nginx\" dhe shkruajn\u00eb-shkruajn\u00eb-shkruajn\u00eb teste nj\u00ebsie. Koleg\u00ebt e din\u00eb se gj\u00ebja m\u00eb interesante do t\u00eb ndodh\u00eb m\u00eb von\u00eb, kur \"t\u00eb gjitha k\u00ebto\" nj\u00eb nat\u00eb do t\u00eb kthehen n\u00eb nj\u00eb telash p\u00ebr nat\u00ebn e Vitit t\u00eb Ri. Dhe do t'i ndihmoj\u00eb vet\u00ebm njohja e thell\u00eb e natyr\u00ebs s\u00eb unix-it, tabela e m\u00ebsuar e gjendjeve TCP\/IP dhe algoritmet themelore t\u00eb renditjes-k\u00ebrkimit.<br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><br \/>\nAh po, ndoshta u nisa pak larg, por shpresoj se kam arritur t\u00eb transmetoj ndjenj\u00ebn e pritjes.<br \/>\nSot dua t\u00eb ndaj p\u00ebrvoj\u00ebn ton\u00eb n\u00eb zhvillimin e nj\u00eb staku t\u00eb leht\u00eb dhe t\u00eb p\u00ebrballuesh\u00ebm p\u00ebr DataLake, q\u00eb zgjidh shumic\u00ebn e problemeve analitike n\u00eb kompani p\u00ebr strukturat e ndryshme organizative.<\/p>\n<p>Disa koh\u00eb m\u00eb par\u00eb arrit\u00ebm t\u00eb kuptojm\u00eb se kompanit\u00eb kan\u00eb nevoj\u00eb gjithnj\u00eb e m\u00eb shum\u00eb p\u00ebr frytet e analitik\u00ebs produktive dhe teknike (pa e p\u00ebrmendur k\u00ebrcellin mbi tort\u00eb n\u00eb form\u00ebn e machine learning) dhe p\u00ebr t\u00eb kuptuar trendet dhe rreziqet \u2014 nevojitet t\u00eb grumbullojm\u00eb dhe analizojm\u00eb gjithnj\u00eb e m\u00eb shum\u00eb metrika.<\/p>\n<h3>Analitika themelore teknike n\u00eb \"Bitrix24\"<\/h3>\n<p>\nDisa vjet m\u00eb par\u00eb, n\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb me nisjen e sh\u00ebrbimit 'Bitrix24', ne investedh kontribuoi pak koh\u00eb dhe burime n\u00eb krijimin e nj\u00eb platforme analitike t\u00eb thjesht\u00eb dhe t\u00eb besueshme, e cila do t\u00eb ndihmonte n\u00eb m\u00ebnyr\u00eb t\u00eb shpejt\u00eb q\u00eb t\u00eb shiheshin problemet n\u00eb infrastruktur\u00eb dhe t\u00eb planifikoheshin hapat e ardhsh\u00ebm. Natyrisht, mjetet preferoheshin t'i merrnim t\u00eb gatshme dhe sa m\u00eb t\u00eb thjeshta dhe t\u00eb kuptueshme. Si rezultat, u zgjodh\u00ebn nagios p\u00ebr monitorim dhe munin p\u00ebr analitik\u00eb dhe vizualizim. Tani kemi mij\u00ebra verifikime n\u00eb nagios, qindra grafik\u00eb n\u00eb munin dhe koleg\u00ebt tan\u00eb i p\u00ebrdorin ato \u00e7do dit\u00eb me sukses. Metodikat jan\u00eb t\u00eb qarta, grafik\u00ebt jan\u00eb t\u00eb kuptuesh\u00ebm, sistemi punon besuesh\u00ebm p\u00ebr disa vjet tani dhe rregullisht shtohen teste dhe grafik\u00eb t\u00eb rinj: fillojm\u00eb nj\u00eb sh\u00ebrbim t\u00eb ri \u2014 shtojm\u00eb disa teste dhe grafik\u00eb. N\u00eb nj\u00eb rrug\u00eb t\u00eb mbar\u00eb.<\/p>\n<h3>Dora mbi puls \u2014 analitika teknike e avancuar<\/h3>\n<p>\nD\u00ebshira p\u00ebr t\u00eb marr\u00eb informacion mbi problemet 'sa m\u00eb shpejt q\u00eb t\u00eb jet\u00eb e mundur' na \u00e7oi n\u00eb eksperimentime aktive me mjete t\u00eb thjeshta dhe t\u00eb kuptueshme \u2014 pinba dhe xhprof.<\/p>\n<p>Pinba na d\u00ebrgonte statistik\u00ebn n\u00eb UDP-paketat p\u00ebr shpejt\u00ebsin\u00eb e funksionimit t\u00eb pjes\u00ebve t\u00eb faqeve t\u00eb web-it n\u00eb PHP dhe ishte e mundur t\u00eb shihnim n\u00eb koh\u00eb reale n\u00eb depozita MySQL (pinba vjen me motorin e saj MySQL p\u00ebr analitik\u00eb t\u00eb shpejt\u00eb t\u00eb ngjarjeve) nj\u00eb list\u00eb t\u00eb shkurt\u00ebr t\u00eb problemeve dhe t\u00eb reagonim ndaj tyre. Xhprof n\u00eb m\u00ebnyr\u00eb automatike lejonte grumbullimin e grafik\u00ebve t\u00eb ekzekutimit t\u00eb faqeve m\u00eb t\u00eb ngadalta PHP t\u00eb klient\u00ebve dhe t\u00eb analizohej se \u00e7far\u00eb mund t\u00eb kishte \u00e7uar n\u00eb k\u00ebt\u00eb \u2014 qet\u00ebsisht, duke pir\u00eb \u00e7aj ose di\u00e7ka m\u00eb t\u00eb fort\u00eb.<\/p>\n<p>Disa koh\u00eb m\u00eb par\u00eb, mjetet e pun\u00ebs u plot\u00ebsuan me nj\u00eb tjet\u00ebr motor t\u00eb thjesht\u00eb dhe t\u00eb kuptuesh\u00ebm n\u00eb baz\u00eb t\u00eb algoritmit t\u00eb indeksimit invers, i realizuar mjaft mir\u00eb n\u00eb bibliotek\u00ebn legjendare Lucene \u2014 Elastic\/Kibana. Ideja e thjesht\u00eb e regjistrimit shum\u00eb-faqesh t\u00eb dokumenteve n\u00eb indeksin e kthyer Lucene n\u00eb baz\u00eb t\u00eb ngjarjeve n\u00eb log dhe k\u00ebrkimit t\u00eb shpejt\u00eb p\u00ebr to duke p\u00ebrdorur ndarjen e fasetave \u2014 v\u00ebrtet ishte e dobishme.<\/p>\n<p>Pavar\u00ebsisht pamjes mjaft teknike t\u00eb vizualizimeve n\u00eb Kibana me konceptet e nivelit t\u00eb ul\u00ebt 'bucket' dhe nj\u00eb gjuh\u00eb t\u00eb rinovuar t\u00eb algjebr\u00ebs relacione, instrumenti na ndihmoi mir\u00eb n\u00eb detyrat e m\u00ebposhtme:<\/p>\n<ul>\n<li>Sa gabime PHP kishte klienti Bitrix24 n\u00eb portalin p1 gjat\u00eb or\u00ebs s\u00eb fundit dhe \u00e7far\u00eb? Kuptoni, falni dhe rregulloni shpejt.<\/li>\n<li>Sa u b\u00ebn\u00eb thirrje video n\u00eb platforma n\u00eb Gjermani gjat\u00eb 24 or\u00ebve t\u00eb fundit, me cila cil\u00ebsi dhe a kishte v\u00ebshtir\u00ebsi me kanalin\/rete?n?<\/li>\n<li>Sa mir\u00eb funksionon funksionaliteti sistematik (shtesa jon\u00eb n\u00eb C p\u00ebr PHP), e nd\u00ebrtuar nga burimi n\u00eb p\u00ebrdit\u00ebsimin m\u00eb t\u00eb fundit t\u00eb sh\u00ebrbimit dhe shp\u00ebrndar\u00eb p\u00ebr klient\u00ebt? A ka segfaults?<\/li>\n<li>A ruhen t\u00eb dh\u00ebnat e klient\u00ebve n\u00eb memorien PHP? A ka gabime q\u00eb tejkalojn\u00eb memorjen e ndar\u00eb p\u00ebr proceset: \"jasht\u00eb memorie\"? Gjejini dhe ndaloni.<\/li>\n<\/ul>\n<p>\nJa nj\u00eb shembull konkret. Pavar\u00ebsisht testimeve t\u00eb kujdesshme dhe shum\u00eb-niveli, nj\u00eb klient p\u00ebrballi nj\u00eb gabim t\u00eb pad\u00ebshiruar dhe t\u00eb papritur n\u00eb nj\u00eb rast shum\u00eb t\u00eb pazakont\u00eb dhe me t\u00eb dh\u00ebna hyr\u00ebse t\u00eb d\u00ebmtuara, filloi alarmi dhe procesi p\u00ebr ta rregulluar shpejt at\u00eb:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/8a802dba41b5d1a85c0dc41dfbf8b84e.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nP\u00ebr m\u00eb tep\u00ebr, Kibana mund\u00ebson organizimin e njoftimeve p\u00ebr ngjarje t\u00eb caktuara dhe brenda nj\u00eb kohe t\u00eb shkurt\u00ebr, shumica e punonj\u00ebsve nga departamente t\u00eb ndryshme \u2014 nga mb\u00ebshtetje teknike dhe zhvillimi deri n\u00eb QA \u2014 filluan t\u00eb p\u00ebrdorin k\u00ebt\u00eb mjet.<\/p>\n<p>Aktiviteti i \u00e7do dege brenda kompanis\u00eb u b\u00eb leht\u00eb i ndjeksh\u00ebm dhe i matsh\u00ebm \u2014 n\u00eb vend t\u00eb analiz\u00ebs manuale t\u00eb log-eve n\u00eb server\u00eb, mjafton q\u00eb nj\u00ebher\u00eb t\u00eb konfiguroni parsing-un e log-eve dhe d\u00ebrgimin e tyre n\u00eb klasterin elastic, p\u00ebr t\u00eb shijuar, p\u00ebr shembull, numrin e kat\u00ebrmij\u00ebsht t\u00eb koteleve dy-koka, t\u00eb printuara n\u00eb 3-d printer p\u00ebr muajin e kaluar h\u00ebnor.<\/p>\n<h3>Analitika e biznesit baz\u00eb<\/h3>\n<p>\nT\u00eb gjith\u00eb din\u00eb se shpesh analitika e biznesit n\u00eb kompani fillon me p\u00ebrdorimin ekstrem t\u00eb Excel-it. Por, m\u00eb e r\u00ebnd\u00ebsishmja, \u00ebsht\u00eb q\u00eb t\u00eb mos p\u00ebrfundoj\u00eb aty. Pushtimi i Google Analytics n\u00eb re gjithashtu kontribuon n\u00eb k\u00ebt\u00eb \u2014 nisin t\u00eb b\u00ebhesh t\u00eb shk\u00eblqyer me t\u00eb.<\/p>\n<p>N\u00eb kompanin\u00eb ton\u00eb, e cila po zhvillohet n\u00eb m\u00ebnyr\u00eb harmonike, filluan t\u00eb shfaqen her\u00eb pas here \"profet\u00eb\" t\u00eb pun\u00ebs m\u00eb intensive me t\u00eb dh\u00ebna m\u00eb t\u00eb m\u00ebdha. K\u00ebrkesat p\u00ebr raporte m\u00eb thell\u00ebsisht dhe shum\u00ebdimensionale u shfaq\u00ebn rregullisht dhe me p\u00ebrpjekjet e djemve nga departamente t\u00eb ndryshme, disa koh\u00eb m\u00eb par\u00eb u organizua nj\u00eb zgjidhje e thjesht\u00eb dhe praktike \u2014 lidhja ClickHouse dhe PowerBI.<\/p>\n<p>Kjo zgjidhje fleksibile ndihmoi shum\u00eb gjat\u00eb nj\u00eb periudhe t\u00eb gjat\u00eb, por gradualisht filloi t\u00eb kuptohej se ClickHouse \u2014 nuk \u00ebsht\u00eb elastik dhe nuk mund t\u00eb keqinterpretohet.<\/p>\n<p>\u00cbsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb kuptoni mir\u00eb se ClickHouse, si Druid, si Vertica, si Amazon RedShift (i cili bazohet n\u00eb Postgres), jan\u00eb motor\u00eb analitik\u00eb, t\u00eb optimizuar p\u00ebr nj\u00eb analiz\u00eb mjaft t\u00eb thjesht\u00eb (shuma, agregime, minimum-maksimum p\u00ebr kolon\u00eb dhe ndonj\u00ebher\u00eb mund t\u00eb b\u00ebhen bashkime), pasi jan\u00eb organizuar p\u00ebr ruajtjen efikase t\u00eb kolonave t\u00eb tabelave relacionale, n\u00eb ndryshim nga MySQL dhe databazat e tjera (me orientim rresht).<\/p>\n<p>N\u00eb thelb, ClickHouse \u00ebsht\u00eb thjesht nj\u00eb \u00abbaza\u00bb t\u00eb dh\u00ebnash m\u00eb e madhe, me nj\u00eb inserim t\u00eb pik\u00ebzuar jo shum\u00eb t\u00eb p\u00ebrshtatsh\u00ebm (\u00ebsht\u00eb menduar k\u00ebshtu, gjith\u00e7ka \u00ebsht\u00eb n\u00eb rregull), por me nj\u00eb analiz\u00eb t\u00eb k\u00ebndshme dhe nj\u00eb grup interesant t\u00eb funksioneve t\u00eb fuqishme p\u00ebr pun\u00ebn me t\u00eb dh\u00ebnat. Po ashtu, mund t\u00eb krijoni nj\u00eb klaster \u2014 por, e kuptoni, t\u00eb godas\u00ebsh me hammer nj\u00eb gozhd\u00eb me mikroskop nuk \u00ebsht\u00eb krejt\u00ebsisht e duhur dhe ne filluam t\u00eb k\u00ebrkojm\u00eb zgjidhje t\u00eb tjera.<\/p>\n<h3>K\u00ebrkesa p\u00ebr Python dhe analitik\u00ebt<\/h3>\n<p>\nN\u00eb kompanin\u00eb ton\u00eb ka shum\u00eb zhvillues, t\u00eb cil\u00ebt shkruajn\u00eb kod pothuajse \u00e7do dit\u00eb p\u00ebr 10-20 vjet n\u00eb PHP, JavaScript, C#, C\/C++, Java, Go, Rust, Python, Bash. Po ashtu, ka shum\u00eb administator\u00eb t\u00eb sistemeve me eksperienc\u00eb, q\u00eb kan\u00eb p\u00ebrjetuar jo nj\u00eb, por disa katastrofa t\u00eb pabesueshme, q\u00eb nuk p\u00ebrfshihen n\u00eb ligjet e statistik\u00ebs (p\u00ebr shembull, kur shkat\u00ebrrohen shumica e disqeve n\u00eb RAID-10 nga nj\u00eb goditje e fort\u00eb vet\u00ebtimash). N\u00eb k\u00ebto kushte, p\u00ebr nj\u00eb koh\u00eb t\u00eb gjat\u00eb ka qen\u00eb e paqart\u00eb se \u00e7far\u00eb \u00ebsht\u00eb nj\u00eb \u00abanalist n\u00eb Python\u00bb. Python \u00ebsht\u00eb si PHP, vet\u00ebm emri \u00ebsht\u00eb pak m\u00eb i gjat\u00eb dhe ndot\u00ebsit e substancave q\u00eb ndryshojn\u00eb mendjen n\u00eb kodin burimor t\u00eb interpretorit jan\u00eb pak m\u00eb t\u00eb pak\u00ebt. Megjithat\u00eb, me krijimin e raporteve gjithnj\u00eb e m\u00eb analitike, zhvilluesit e eksperienc\u00ebs kan\u00eb filluar t\u00eb kuptojn\u00eb r\u00ebnd\u00ebsin\u00eb e specializimit t\u00eb ngusht\u00eb n\u00eb mjetet si numpy, pandas, matplotlib, seaborn.<br \/>\nRoli vendimtar, me siguri, ka qen\u00eb nj\u00eb ndihm\u00eb e papritur nga fjetja e punonj\u00ebsve nga kombinimi i fjal\u00ebve \u00abregresioni logjistik\u00bb dhe demonstrimi i nd\u00ebrtimit t\u00eb raporteve efikase mbi t\u00eb dh\u00ebna voluminoze me an\u00eb t\u00eb pyspark.<\/p>\n<p>Apache Spark, paradigmat e tij funksionale, mbi t\u00eb cilat bazohet shum\u00eb mir\u00eb algebra relacional, pati nj\u00eb impakt t\u00eb till\u00eb te zhvilluesit q\u00eb ishin m\u00ebsuar me MySQL, saq\u00eb nevoja p\u00ebr t\u00eb forcuar radh\u00ebt me analist\u00eb t\u00eb njohur u b\u00eb e qart\u00eb si dita.<\/p>\n<h3>Tentativat e m\u00ebtejshme t\u00eb Apache Spark\/Hadoop p\u00ebr t\u00eb fluturuar dhe se \u00e7far\u00eb nuk shkoi krejt\u00ebsisht sipas skenarit<\/h3>\n<p>\nMegjithat\u00eb, s\u00eb shpejti u b\u00eb e qart\u00eb se me Spark, duket se di\u00e7ka nuk ishte sistematikisht n\u00eb rregull ose thjesht duhen lar\u00eb m\u00eb mir\u00eb duar. N\u00ebse stoku Hadoop\/MapReduce\/Lucene ishte zhvilluar nga programues t\u00eb mjaftuesh\u00ebm t\u00eb p\u00ebrvojsh\u00ebm, \u00e7ka \u00ebsht\u00eb evidente n\u00ebse shikon me v\u00ebmendje kodin burimor n\u00eb Java ose idet\u00eb e Doug Cutting n\u00eb Lucene, at\u00ebher\u00eb Spark, papritur, \u00ebsht\u00eb shkruar n\u00eb nj\u00eb gjuh\u00eb t\u00eb diskutueshme nga pik\u00ebpamja e praktik\u00ebs dhe tani nuk po zhvillohet, gjuha ekzotike Scala. P\u00ebrdorimi i rregullt t\u00eb llogaritjeve n\u00eb p\u00ebrb\u00ebrjen Spark p\u00ebr shkak t\u00eb pun\u00ebsjo t\u00eb paqart\u00eb dhe jo shum\u00eb transparente me ndarjen e memories p\u00ebr operacionet e reduktimit (vijn\u00eb shum\u00eb \u00e7el\u00ebsa menj\u00ebher\u00eb) \u2014 krijoi rreth tij nj\u00eb aur\u00eb t\u00eb di\u00e7kaje q\u00eb ka hap\u00ebsir\u00eb p\u00ebr t\u00eb u rritur. Shtes\u00eb, situata u p\u00ebrkeq\u00ebsua nga numri i madh i porteve t\u00eb \u00e7uditshme t\u00eb hapura, skedar\u00ebve temporar\u00eb q\u00eb rriteshin n\u00eb vende t\u00eb paqart\u00eb dhe var\u00ebsive jar \u2014 q\u00eb shkaktonte ndjenja t\u00eb njohura tek administruesit e sistemit: nj\u00eb urrejtje e fort\u00eb (ndoshta do t\u00eb duhej t\u00eb lahnin duar me sapun).<\/p>\n<p>Ne, p\u00ebr pasoj\u00eb, kemi \"kanosur\" disa projekte analitike t\u00eb brendshme q\u00eb p\u00ebrdorin aktivisht Apache Spark (p\u00ebrfshir\u00eb Spark Streaming, Spark SQL) dhe ekosistemin Hadoop (dhe t\u00eb tjera). Megjith\u00ebse me kalimin e koh\u00ebs m\u00ebsuam ta p\u00ebrgatisim dhe monitorojm\u00eb mjaft mir\u00eb dhe \"ato\" praktikisht ndaluan t\u00eb bien papritur p\u00ebr shkak t\u00eb ndryshimit t\u00eb natyr\u00ebs s\u00eb t\u00eb dh\u00ebnave dhe disbalanc\u00ebs s\u00eb heshimit t\u00eb barabart\u00eb RDD, d\u00ebshira p\u00ebr t\u00eb marr\u00eb di\u00e7ka t\u00eb gatshme, t\u00eb p\u00ebrdit\u00ebsuar dhe t\u00eb administruar pakt\u00eb diku n\u00eb cloud u intensifikua gjithnj\u00eb e m\u00eb shum\u00eb. Pik\u00ebrisht n\u00eb k\u00ebt\u00eb koh\u00eb, provuam t\u00eb p\u00ebrdornim nj\u00eb nd\u00ebrtim cloud t\u00eb gatsh\u00ebm nga Amazon Web Services \u2014 <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/emr\/\">EMR<\/a><\/noindex> dhe, m\u00eb von\u00eb, p\u00ebrpiqeshim t\u00eb zgjidhim detyrat n\u00eb t\u00eb. EMR \u00ebsht\u00eb nj\u00eb Apache Spark i p\u00ebrgatitur nga Amazon me softuer shtes\u00eb nga ekosistemi, di\u00e7ka si nd\u00ebrtimet Cloudera\/Hortonworks.<\/p>\n<h3>\"Ruajtja e skedar\u00ebve\" p\u00ebr analitik\u00eb \u2014 nj\u00eb nevoj\u00eb e ngutshme<\/h3>\n<p>\nEksperienca e \"gatimit\" t\u00eb Hadoop\/Spark me djegie t\u00eb ndryshme trupore nuk kaloi kot. Nevoja p\u00ebr t\u00eb krijuar nj\u00eb depo t\u00eb vetme t\u00eb lir\u00eb dhe t\u00eb besueshme u b\u00eb gjithnj\u00eb e m\u00eb e qart\u00eb, e cila do t\u00eb ishte e q\u00ebndrueshme ndaj depozitave t\u00eb pajisjeve dhe ku mund t\u00eb ruajm\u00eb skedar\u00eb n\u00eb formate t\u00eb ndryshme nga sisteme t\u00eb ndryshme dhe t\u00eb b\u00ebjm\u00eb p\u00ebr k\u00ebto t\u00eb dh\u00ebna zgjedhje efektive dhe q\u00eb realizohen n\u00eb koh\u00eb t\u00eb arsyeshme p\u00ebr raporte.<\/p>\n<p>Gjithashtu, doja q\u00eb azhurnimi i softuerit t\u00eb k\u00ebsaj platforme t\u00eb mos kthehej n\u00eb nj\u00eb makth t\u00eb nat\u00ebs s\u00eb Vitit t\u00eb Ri me leximin e sh\u00ebnimeve 20-faq\u00ebshe t\u00eb Java dhe analizimin e kilometrave t\u00eb detajeve t\u00eb log\u00ebve t\u00eb pun\u00ebs s\u00eb klasterit me ndihm\u00ebn e Spark History Server dhe nj\u00eb lup\u00eb me ndri\u00e7im. Doja t\u00eb kishim nj\u00eb mjet t\u00eb thjesht\u00eb dhe transparent, q\u00eb nuk k\u00ebrkon nj\u00eb zhytje t\u00eb rregullt n\u00ebn kapak, n\u00ebse k\u00ebrkuesi ndalon s\u00eb ekzekutuari nj\u00eb k\u00ebrkes\u00eb standarde MapReduce kur ndodhin humbjet e memories s\u00eb punonj\u00ebsit t\u00eb t\u00eb dh\u00ebnave t\u00eb reduktimit p\u00ebr shkak t\u00eb nj\u00eb algoritmi jo shum\u00eb t\u00eb p\u00ebrshtatsh\u00ebm t\u00eb ndarjes s\u00eb t\u00eb dh\u00ebnave origjinale.<\/p>\n<h3>Amazon S3 \u2014 nj\u00eb kandidat p\u00ebr DataLake?<\/h3>\n<p>\nEksperienca me Hadoop\/MapReduce m\u00eb m\u00ebsoi se nevojitet nj\u00eb sistem i q\u00ebndruesh\u00ebm skedar\u00ebsh q\u00eb \u00ebsht\u00eb gjithashtu i shkall\u00ebzuesh\u00ebm dhe sip\u00ebr tij pun\u00ebtor\u00eb t\u00eb shkall\u00ebzuesh\u00ebm, \"duke ardhur\" af\u00ebr t\u00eb dh\u00ebnave, n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb mos l\u00ebvizin t\u00eb dh\u00ebnat p\u00ebrmes rrjetit. Punonj\u00ebsit duhet t\u00eb jen\u00eb n\u00eb gjendje t\u00eb lexojn\u00eb t\u00eb dh\u00ebnat n\u00eb formate t\u00eb ndryshme, por, sa m\u00eb mir\u00eb, t\u00eb mos lexojn\u00eb informacione t\u00eb panevojshme, dhe q\u00eb t\u00eb mund t\u00eb ruajn\u00eb t\u00eb dh\u00ebnat paraprakisht n\u00eb formate t\u00eb p\u00ebrshtatshme p\u00ebr punonj\u00ebsit.<\/p>\n<p><b>Nj\u00ebher\u00eb tjet\u00ebr \u2014 ideja kryesore.<\/b> Nuk kam d\u00ebshir\u00eb t\u00eb \"ngarkoj\" t\u00eb dh\u00ebna t\u00eb m\u00ebdha n\u00eb nj\u00eb motor analitik klasterik t\u00eb vet\u00ebm, i cili do t\u00eb d\u00ebshp\u00ebrohet dhe do t\u00eb duhet ta ndajm\u00eb n\u00eb m\u00ebnyr\u00eb t\u00eb pak\u00ebndshme. Dua t\u00eb ruaj skedar\u00eb, thjesht skedar\u00eb, n\u00eb nj\u00eb format t\u00eb kuptuesh\u00ebm dhe t\u00eb ekzekutimin e k\u00ebrkesave efektive analitike n\u00eb m\u00ebnyr\u00eb t\u00eb ndryshme, por edhe t\u00eb kuptueshme. Dhe numri i skedar\u00ebve n\u00eb formate t\u00eb ndryshme do t\u00eb jet\u00eb gjithnj\u00eb e m\u00eb i madh. Dhe do t\u00eb ishte m\u00eb mir\u00eb t\u00eb ndahej jo motori, por t\u00eb dh\u00ebnat origjinale. Na nevojitet nj\u00eb DataLake i zgjeruesh\u00ebm dhe universale, vendos\u00ebm ne...<\/p>\n<p>\u00c7far\u00eb n\u00ebse ruajm\u00eb skedar\u00ebt n\u00eb nj\u00eb depo t\u00eb njohur dhe t\u00eb shkall\u00ebzuar t\u00eb njohur t\u00eb Amazon S3, pa u marr\u00eb me p\u00ebrgatitjen e vet t\u00eb pllakave nga Hadoop?<\/p>\n<p>E qart\u00eb, t\u00eb dh\u00ebnat personale \u201cnuk lejohet\u201d, por t\u00eb dh\u00ebnat e tjera n\u00ebse i nxjerrim atje dhe i \u201cp\u00ebrpunojm\u00eb me efektivitet\u201d?<\/p>\n<h3>Ekosistemi analitik t\u00eb dh\u00ebnave t\u00eb m\u00ebdha t\u00eb Amazon Web Services \u2014 me fjal\u00eb shum\u00eb t\u00eb thjeshta<\/h3>\n<p>\nDuke u bazuar n\u00eb p\u00ebrvoj\u00ebn ton\u00eb me AWS, aty p\u00ebrdoret prej koh\u00ebsh dhe me intensitet Apache Hadoop\/MapReduce n\u00ebn forma t\u00eb ndryshme, p\u00ebr shembull n\u00eb sh\u00ebrbimin DataPipeline (i ziliqem koleg\u00ebve, ata e din\u00eb ta p\u00ebrgatisin si\u00e7 duhet). K\u00ebtu kemi vendosur backup nga sh\u00ebrbime t\u00eb ndryshme nga tabelat DynamoDB:<br \/>\n<img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/058dc54ed032a7bf3e9e129646202440.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDhe ato jan\u00eb duke u ekzekutuar rregullisht n\u00eb klasteret e integruara Hadoop\/MapReduce si or\u00ebt p\u00ebr disa vite. \"P\u00ebrgatite dhe harroje\":<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/a6569da8cafdb96c63250bb32bf51704.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nPo ashtu, mund t\u00eb angazhoheni n\u00eb datascienc\u00eb n\u00eb m\u00ebnyr\u00eb efektive duke ngritur notebook-\u00ebt Jupiter p\u00ebr analist\u00ebt n\u00eb cloud dhe duke p\u00ebrdorur sh\u00ebrbimin AWS SageMaker p\u00ebr trajnim dhe vendosjen e modeleve AI n\u00eb p\u00ebrdorim. Ja si duket kjo n\u00eb ne:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/c825d979c9278a8edf8e1e747ef6def8.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nDhe, po, mund t\u00eb ngresh p\u00ebr vete ose p\u00ebr analist\u00ebt nj\u00eb notebook n\u00eb cloud dhe ta lidh\u00ebsh at\u00eb me klasterin Hadoop\/Spark, t\u00eb b\u00ebsh llogaritje dhe pastaj 't\u00eb p\u00ebrfundosh' gjith\u00e7ka:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/73cea18c54d2a9ce8d0441463991808b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nV\u00ebrtet\u00eb e p\u00ebrshtatshme p\u00ebr projekte analitike t\u00eb ve\u00e7anta dhe p\u00ebr disa prej tyre kemi p\u00ebrdorur me sukses sh\u00ebrbimin EMR p\u00ebr kalkulime dhe analitika t\u00eb m\u00ebdha. Por \u00e7far\u00eb do t\u00eb thot\u00eb p\u00ebr nj\u00eb zgjidhje sistematike p\u00ebr DataLake, do t\u00eb funksionoj\u00eb? N\u00eb at\u00eb moment ishim n\u00eb prag t\u00eb shpres\u00ebs dhe d\u00ebshp\u00ebrimit dhe vazhduam k\u00ebrkimin.<\/p>\n<h3>AWS Glue \u2014 nj\u00eb Apache Spark i paketuar me kujdes 'n\u00eb steroide'<\/h3>\n<p>\nDoli se AWS ka nj\u00eb version t\u00eb tijin t\u00eb stack-ut 'Hive\/Pig\/Spark'. Rolii i Hive, dmth, katalogu i skedar\u00ebve dhe tipeve t\u00eb tyre n\u00eb DataLake, ekzekutohet nga sh\u00ebrbimi 'Data catalog', i cili nuk e fsheh kompatibilitetin e tij me formatin Apache Hive. N\u00eb k\u00ebt\u00eb sh\u00ebrbim duhen shtuar informacione se ku ndodhen skedar\u00ebt tuaj dhe n\u00eb \u00e7far\u00eb formati jan\u00eb. T\u00eb dh\u00ebnat mund t\u00eb jen\u00eb jo vet\u00ebm n\u00eb s3, por edhe n\u00eb nj\u00eb baz\u00eb t\u00eb dh\u00ebnash, por p\u00ebr k\u00ebt\u00eb nuk do flasim n\u00eb k\u00ebt\u00eb postim. Ja si \u00ebsht\u00eb organizuar katalogu i dh\u00ebnave DataLake p\u00ebr ne:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/adb45d09698fdacbf41c86bbadde8bb2.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSkedar\u00ebt jan\u00eb regjistruar, shk\u00eblqyesh\u00ebm. N\u00ebse skedar\u00ebt jan\u00eb p\u00ebrdit\u00ebsuar \u2014 aktivizojm\u00eb manualisht ose sipas nj\u00eb skeduli crawlers, t\u00eb cil\u00ebt do t\u00eb p\u00ebrdit\u00ebsojn\u00eb informacionin mbi ta nga liqeni dhe do ta ruajn\u00eb. M\u00eb pas, t\u00eb dh\u00ebnat nga liqeni mund t\u00eb p\u00ebrpunohen dhe rezultatet diku t\u00eb skadohen. N\u00eb rastin m\u00eb t\u00eb thjesht\u00eb \u2014 skadojm\u00eb gjithashtu n\u00eb s3. P\u00ebrpunimi i t\u00eb dh\u00ebnave mund t\u00eb b\u00ebhet kudo, por ofrohet q\u00eb t\u00eb konfigurohet procesi i p\u00ebrpunimit n\u00eb klasterin Apache Spark duke p\u00ebrdorur funksionalitetet e avancuara p\u00ebrmes API AWS Glue. N\u00eb thelb, mund t\u00eb marrim kodin e vjet\u00ebr t\u00eb njohur n\u00eb python duke p\u00ebrdorur bibliotek\u00ebn pyspark dhe t\u00eb konfigurojm\u00eb ekzekutimin e tij n\u00eb N nodet e klasterit t\u00eb ndonj\u00eb fuqie me monitorim, pa g\u00ebrmim n\u00eb thell\u00ebsit\u00eb e Hadoop dhe transportim t\u00eb kontejner\u00ebve docker-mokers dhe eliminimi i konflikteve t\u00eb var\u00ebsis\u00eb.<\/p>\n<p><b>Edhe nj\u00eb her\u00eb \u2014 nj\u00eb ide e thjesht\u00eb.<\/b> Nuk \u00ebsht\u00eb e nevojshme t\u00eb konfigurohet Apache Spark, duhet vet\u00ebm t\u00eb shkruhet kodi n\u00eb python p\u00ebr pyspark, ta testosh lokal n\u00eb desktop dhe pastaj ta aktivizosh n\u00eb nj\u00eb klaster t\u00eb madh n\u00eb cloud, duke treguar ku ndodhen t\u00eb dh\u00ebnat burimore dhe ku duhet t\u00eb vendoset rezultati. Ndonj\u00ebher\u00eb kjo \u00ebsht\u00eb e nevojshme dhe e dobishme dhe ja si \u00ebsht\u00eb konfiguruar p\u00ebr ne:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/dc03181573bb3f5cfcc3a8760bc7e07b.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nK\u00ebshtu, n\u00ebse kemi nevoj\u00eb t\u00eb b\u00ebjm\u00eb nj\u00eb kalkulim n\u00eb klasterin Spark mbi t\u00eb dh\u00ebnat n\u00eb s3 \u2014 shkruajm\u00eb kodin n\u00eb python\/pyspark, e testojm\u00eb dhe shkojm\u00eb n\u00eb rrug\u00ebn e mir\u00eb n\u00eb cloud.<\/p>\n<p>\u00c7far\u00eb ndodhi me orkestrimin? \u00c7far\u00eb b\u00ebhet n\u00ebse nj\u00eb detyr\u00eb d\u00ebshton dhe humb? Po, propozohet t\u00eb krijohet nj\u00eb pipeline t\u00eb bukur n\u00eb stilin e Apache Pig dhe madje edhe ne e provuam at\u00eb, por vendos\u00ebm t\u00eb p\u00ebrdorim orkestrimin ton\u00eb t\u00eb thell\u00eb t\u00eb personalizuar n\u00eb PHP dhe JavaScript (e kuptoj, ka nj\u00eb disonanc\u00eb kognitive, por funksionon, p\u00ebr vite me radh\u00eb dhe pa gabime).<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/e00ed2047c5c6492e36fccc82e7278a3.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>Formati i skedar\u00ebve t\u00eb ruajtur n\u00eb liqen \u2014 \u00e7el\u00ebsi p\u00ebr performanc\u00ebn<\/h3>\n<p>\n\u00cbsht\u00eb shum\u00eb, shum\u00eb e r\u00ebnd\u00ebsishme t\u00eb kuptojm\u00eb edhe dy momente ky\u00e7e. P\u00ebr t'u siguruar q\u00eb k\u00ebrkesat p\u00ebr t\u00eb dh\u00ebnat e skedar\u00ebve n\u00eb liqen t\u00eb zbatohen sa m\u00eb shpejt t\u00eb jet\u00eb e mundur dhe performanca t\u00eb mos degradohet gjat\u00eb shtimit t\u00eb informacionit t\u00eb ri, nevojitet:<\/p>\n<ul>\n<li>T\u00eb ruajm\u00eb kolonat e skedar\u00ebve ve\u00e7mas (p\u00ebr t\u00eb mos qen\u00eb e nevojshme t\u00eb lexojm\u00eb t\u00eb gjitha rreshtat p\u00ebr t\u00eb kuptuar se \u00e7far\u00eb ka n\u00eb kolona). P\u00ebr k\u00ebt\u00eb, ne mor\u00ebm formatin parquet me kompresim<\/li>\n<li>\u00cbsht\u00eb shum\u00eb e r\u00ebnd\u00ebsishme t\u00eb shardojm\u00eb skedar\u00ebt n\u00eb dosje si: gjuha, viti, muaji, dita, java. Motor\u00ebt q\u00eb kuptojn\u00eb k\u00ebt\u00eb lloj shardimi do t\u00eb shikojn\u00eb vet\u00ebm n\u00eb dosjet e nevojshme, pa e p\u00ebrzgjedhur t\u00eb gjith\u00eb informacionin nj\u00eblloj.<\/li>\n<\/ul>\n<p>\nShkurtimisht, n\u00eb k\u00ebt\u00eb m\u00ebnyr\u00eb, ju nxirrni t\u00eb dh\u00ebnat origjinale n\u00eb form\u00ebn m\u00eb efikase p\u00ebr motor\u00ebt analitik\u00eb q\u00eb e din\u00eb si t\u00eb hyjn\u00eb selektivisht n\u00eb dosjet e sharduara dhe t\u00eb lexojn\u00eb vet\u00ebm kolonat e nevojshme nga skedar\u00ebt. Nuk \u00ebsht\u00eb nevoja t\u00eb \"derdhni\" t\u00eb dh\u00ebnat diku (magazina do t\u00eb shp\u00ebrthej\u00eb) \u2014 thjesht vendosini menj\u00ebher\u00eb n\u00eb sistemin e skedar\u00ebve n\u00eb formatin e duhur. Natyrisht, k\u00ebtu duhet t\u00eb jet\u00eb e qart\u00eb se ruajtja e nj\u00eb CSV t\u00eb madh n\u00eb DataLake, i cili duhet t\u00eb lexohet rresht p\u00ebr rresht nga klasteri p\u00ebr t\u00eb nxjerr\u00eb kolonat \u2014 nuk \u00ebsht\u00eb shum\u00eb e arsyeshme. Mendoni p\u00ebrs\u00ebri mbi dy pikat e m\u00ebsip\u00ebrme n\u00ebse nuk e kuptoni ende pse \u00ebsht\u00eb e gjitha kjo.<\/p>\n<h3>AWS Athena \u2014 \"djalli\" nga kutia<\/h3>\n<p>\nDhe k\u00ebtu, duke krijuar liqenin, ne, n\u00eb nj\u00eb m\u00ebnyr\u00eb, has\u00ebm n\u00eb Amazon Athena. Papritur u zbulua se duke grumbulluar me kujdes n\u00eb dosjet e sharduara n\u00eb formatin e duhur (parquet) skedar\u00ebt tan\u00eb t\u00eb m\u00ebdhenj t\u00eb logjeve \u2014 mund t\u00eb b\u00ebjm\u00eb shum\u00eb shpejt zgjedhje shum\u00eb informuese dhe t\u00eb nd\u00ebrtojm\u00eb raporte PA, pa klasterin Apache Spark\/Glue.<\/p>\n<p>Motori Athena, q\u00eb punon me t\u00eb dh\u00ebnat n\u00eb s3, bazohet n\u00eb legjend\u00ebn <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/big-data\/what-is-presto\/\">Presto<\/a><\/noindex> \u2014 nj\u00eb p\u00ebrfaq\u00ebsues i familjes MPP (procesimi masiv paralel) t\u00eb qasjes p\u00ebr p\u00ebrpunimin e t\u00eb dh\u00ebnave, duke marr\u00eb t\u00eb dh\u00ebnat aty ku ndodhen, nga s3 dhe Hadoop te Cassandra dhe skedar\u00ebt e zakonsh\u00ebm tekstual\u00eb. Thjesht duhet t\u00eb k\u00ebrkosh nga Athena t\u00eb ekzekutoj\u00eb nj\u00eb k\u00ebrkes\u00eb SQL, dhe pastaj gjith\u00e7ka \"funksionon shpejt dhe vet\u00eb\". \u00cbsht\u00eb e r\u00ebnd\u00ebsishme t\u00eb theksohet se Athena \u00ebsht\u00eb \"e men\u00e7ur\", shkon vet\u00ebm n\u00eb dosjet e nevojshme t\u00eb ndara dhe lexon vet\u00ebm kolonat e nevojshme n\u00eb k\u00ebrkes\u00eb.<\/p>\n<p>Tarifat p\u00ebr k\u00ebrkesat n\u00eb Athena jan\u00eb gjithashtu interesante. Ne paguajm\u00eb p\u00ebr <noindex><a rel=\"nofollow\" href=\"https:\/\/aws.amazon.com\/ru\/athena\/pricing\/\">v\u00ebllimin e t\u00eb dh\u00ebnave t\u00eb skanuara<\/a><\/noindex>. Pra, jo p\u00ebr numrin e server\u00ebve n\u00eb klaster \u00e7do minut\u00eb, por\u2026 p\u00ebr t\u00eb dh\u00ebnat q\u00eb jan\u00eb realisht skanuar n\u00eb 100-500 server\u00eb q\u00eb jan\u00eb vet\u00ebm t\u00eb nevojshme p\u00ebr t\u00eb p\u00ebrmbushur k\u00ebrkes\u00ebn.<\/p>\n<p>Dhe duke k\u00ebrkuar vet\u00ebm kolonat e nevojshme nga dosjet e duhura t\u00eb ndara, rezultoi se sh\u00ebrbimi Athena na kushton vet\u00ebm disa dhjet\u00ebra dollar\u00eb n\u00eb muaj. Well, p\u00ebrshtatshme, thuajse falas, krahasuar me analiz\u00ebn n\u00eb klaster\u00eb!<\/p>\n<p>Ja, si i ndajm\u00eb t\u00eb dh\u00ebnat tona n\u00eb s3:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/00bd9ae48c1cd13f3c4f7d32692c9209.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<br \/>\nSi pasoj\u00eb, brenda nj\u00eb kohe t\u00eb shkurt\u00ebr, n\u00eb kompani jan\u00eb b\u00ebr\u00eb k\u00ebrkesa aktive n\u00eb Athena nga departamente t\u00eb ndryshme, nga siguria informative te analiza, dhe ato marrin shpejt, brenda sekondash, p\u00ebrgjigje t\u00eb dobishme nga \"t\u00eb dh\u00ebnat e m\u00ebdha\" p\u00ebr periudha mjaft t\u00eb gjera: muaj, gjysm\u00eb viti etj.<\/p>\n<p>Por ne shkuam m\u00eb tej dhe filluam t\u00eb k\u00ebrkojm\u00eb p\u00ebrgjigje n\u00eb cloud <noindex><a rel=\"nofollow\" href=\"https:\/\/docs.aws.amazon.com\/athena\/latest\/ug\/connect-with-odbc.html\">n\u00ebp\u00ebrmjet ODBC-draivrit<\/a><\/noindex>: analisti n\u00eb konsol\u00ebn e tij t\u00eb zakonshme shkruan nj\u00eb k\u00ebrkes\u00eb SQL, e cila \"me pak para\" k\u00ebrkon t\u00eb dh\u00ebnat n\u00eb s3 n\u00eb 100-500 server\u00eb dhe kthen p\u00ebrgjigjen zakonisht brenda disa sekondash. E ndihmon. Dhe \u00ebsht\u00eb shpejt. Akoma nuk e besojm\u00eb.<\/p>\n<p>N\u00eb p\u00ebrfundim, pas vendimit p\u00ebr t\u00eb ruajtur t\u00eb dh\u00ebnat n\u00eb s3, n\u00eb nj\u00eb format efikas kol\u00f3nar dhe me ndarje t\u00eb arsyeshme t\u00eb t\u00eb dh\u00ebnave n\u00eb dosje\u2026 ne mor\u00ebm DataLake dhe nj\u00eb mekaniz\u00ebm analitik t\u00eb shpejt\u00eb dhe t\u00eb lir\u00eb \u2014 falas. Dhe ai u b\u00eb shum\u00eb popullor n\u00eb kompani, pasi kupton SQL dhe punon shum\u00eb m\u00eb shpejt, sesa p\u00ebrmes ndjekjeve\/njohjeve\/konfigurimeve t\u00eb klaster\u00ebve. \"E n\u00ebse rezultati \u00ebsht\u00eb i nj\u00ebjt\u00eb, pse t\u00eb paguani m\u00eb shum\u00eb?\"<\/p>\n<p>Nj\u00eb k\u00ebrkes\u00eb n\u00eb Athena duket m\u00eb pak k\u00ebshtu. N\u00ebse d\u00ebshiron, sigurisht, mund t\u00eb formosh nj\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/prestodb.io\/docs\/0.172\/index.html\">k\u00ebrkes\u00eb SQL mjaft t\u00eb komplikuar dhe shum\u00ebfaq\u00ebshe<\/a><\/noindex>, por ne do t\u00eb kufizohemi n\u00eb nj\u00eb grupim t\u00eb thjesht\u00eb. Le t\u00eb shohim se cilat kode p\u00ebrgjigjesh kishte klienti disa jav\u00eb m\u00eb par\u00eb n\u00eb log-at e pun\u00ebs s\u00eb serverit web dhe t\u00eb sigurohemi q\u00eb nuk ka gabime:<\/p>\n<p><img decoding=\"async\" alt=\"Si e organizuam nj\u00eb DataLake me performanc\u00eb t\u00eb lart\u00eb dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht k\u00ebshtu\" src=\"\/wp-content\/uploads\/2020\/08\/30028991467b9e52f597faa617d374b5.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<\/p>\n<h3>P\u00ebrfundimet<\/h3>\n<p>\nPas duke kaluar n\u00ebp\u00ebr nj\u00eb rrug\u00eb q\u00eb nuk ishte e gjat\u00eb, por e dhimbshme, duke vler\u00ebsuar vazhdimisht rrezikun dhe nivelin e v\u00ebshtir\u00ebsis\u00eb dhe koston e mb\u00ebshtetjes, gjet\u00ebm zgjidhjen p\u00ebr DataLake dhe analiz\u00ebn q\u00eb na g\u00ebzon vazhdimisht me shpejt\u00ebsin\u00eb dhe kostot e zbatimit.<\/p>\n<p>Doli se nd\u00ebrtimi i nj\u00eb DataLake efektiv, t\u00eb shpejt\u00eb dhe me kosto t\u00eb ul\u00ebt p\u00ebr nevojat e nj\u00eb numri t\u00eb ndrysh\u00ebm nj\u00ebsish brenda kompanis\u00eb - \u00ebsht\u00eb plot\u00ebsisht i realizuesh\u00ebm edhe p\u00ebr zhvilluesit e p\u00ebrvojsh\u00ebm, t\u00eb cil\u00ebt nuk kan\u00eb punuar kurr\u00eb si arkitekt\u00eb dhe nuk din\u00eb t\u00eb vizatojn\u00eb katror\u00eb mbi katror\u00eb me shigjeta dhe t\u00eb njohin 50 terma nga ekosistemi Hadoop.<\/p>\n<p>N\u00eb fillim t\u00eb rrug\u00ebs, koka m\u00eb dhembte nga shum\u00eb zoologji t\u00eb \u00e7mendura t\u00eb softuer\u00ebve t\u00eb hapur dhe t\u00eb mbyllur dhe nga ndjenja e barr\u00ebs s\u00eb p\u00ebrgjegj\u00ebsis\u00eb ndaj pasardh\u00ebsve. Thjesht filloni t\u00eb nd\u00ebrtoni DataLake tuaj me mjete t\u00eb thjeshta: nagios\/munin -&gt; elastic\/kibana -&gt; Hadoop\/Spark\/s3 \u2026, duke mbledhur reagime dhe duke kuptuar thell\u00ebsisht fizik\u00ebn e proceseve q\u00eb ndodhin. \u00c7do gj\u00eb e komplikuar dhe e turbullt \u2014 ia l\u00ebshoni armiqve dhe konkurent\u00ebve.<\/p>\n<p>N\u00ebse nuk d\u00ebshironi t\u00eb shkoni n\u00eb cloud dhe preferoni t\u00eb mb\u00ebshtesni, p\u00ebrdit\u00ebsoni dhe aplikoni patches n\u00eb projekte t\u00eb hapura, mund t\u00eb nd\u00ebrtoni nj\u00eb skem\u00eb t\u00eb ngjashme me ton\u00ebn lokal, n\u00eb makina t\u00eb lira zyre me Hadoop dhe Presto p\u00ebrmbi. E r\u00ebnd\u00ebsishme \u00ebsht\u00eb t\u00eb mos ndaloni dhe t\u00eb ecni p\u00ebrpara, t\u00eb llogaritni, t\u00eb k\u00ebrkoni zgjidhje t\u00eb thjeshta dhe t\u00eb qarta dhe gjith\u00e7ka do t\u00eb dal\u00eb n\u00eb vend! Fat t\u00eb mir\u00eb t\u00eb gjith\u00ebve dhe deri n\u00eb takimin e ardhsh\u00ebm!<br \/>\n<br \/>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/bitrix\/blog\/516374\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041c\u044b \u0436\u0438\u0432\u0435\u043c \u0432 \u0443\u0434\u0438\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f, \u043a\u043e\u0433\u0434\u0430 \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0441\u0442\u044b\u043a\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0433\u043e\u0442\u043e\u0432\u044b\u0445 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438\u0445 \u0441 \u00ab\u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u043c \u0441\u043e\u0437\u043d\u0430\u043d\u0438\u0435\u043c\u00bb \u043f\u043e \u0441\u043e\u0432\u0435\u0442\u0430\u043c stackoverflow, \u043d\u0435 \u0432\u043d\u0438\u043a\u0430\u044f \u0432 \u00ab\u043c\u043d\u043e\u0433\u043e\u0431\u0443\u043a\u0432\u00bb, \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432 \u043a\u043e\u043c\u043c\u0435\u0440\u0447\u0435\u0441\u043a\u0443\u044e \u044d\u043a\u0441\u043f\u043b\u0443\u0430\u0442\u0430\u0446\u0438\u044e. \u0410 \u043a\u043e\u0433\u0434\u0430 \u043d\u0443\u0436\u043d\u043e \u0431\u0443\u0434\u0435\u0442 \u043e\u0431\u043d\u043e\u0432\u043b\u044f\u0442\u044c\u0441\u044f\/\u0440\u0430\u0441\u0448\u0438\u0440\u044f\u0442\u044c\u0441\u044f \u0438\u043b\u0438 \u043a\u0442\u043e-\u0442\u043e \u0441\u043b\u0443\u0447\u0430\u0439\u043d\u043e \u043f\u0435\u0440\u0435\u0437\u0430\u0433\u0440\u0443\u0437\u0438\u0442 \u043f\u0430\u0440\u0443 \u043c\u0430\u0448\u0438\u043d \u2014 \u043e\u0441\u043e\u0437\u043d\u0430\u0442\u044c, \u0447\u0442\u043e \u043d\u0430\u0447\u0430\u043b\u0441\u044f \u043a\u0430\u043a\u043e\u0439-\u0442\u043e \u043d\u0430\u0432\u044f\u0437\u0447\u0438\u0432\u044b\u0439 \u0434\u0443\u0440\u043d\u043e\u0439 \u0441\u043e\u043d \u043d\u0430\u044f\u0432\u0443, \u0432\u0441\u0435 \u0440\u0435\u0437\u043a\u043e \u0443\u0441\u043b\u043e\u0436\u043d\u0438\u043b\u043e\u0441\u044c \u0434\u043e [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":92509,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-92508","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041c\u044b \u0436\u0438\u0432\u0435\u043c \u0432 \u0443\u0434\u0438\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f, \u043a\u043e\u0433\u0434\u0430 \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0441\u0442\u044b\u043a\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0433\u043e\u0442\u043e\u0432\u044b\u0445 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438\u0445 \u0441 \u00ab\u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u043c \u0441\u043e\u0437\u043d\u0430\u043d\u0438\u0435\u043c\u00bb \u043f\u043e \u0441\u043e\u0432\u0435\u0442\u0430\u043c stackoverflow, \u043d\u0435 \u0432\u043d\u0438\u043a\u0430\u044f \u0432 \u00ab\u043c\u043d\u043e\u0433\u043e\u0431\u0443\u043a\u0432\u00bb, \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041a\u0430\u043a \u043c\u044b \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u0430\u043b\u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0435 \u0438 \u043d\u0435\u0434\u043e\u0440\u043e\u0433\u043e\u0435 DataLake \u0438 \u043f\u043e\u0447\u0435\u043c\u0443 \u0438\u043c\u0435\u043d\u043d\u043e \u0442\u0430\u043a | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041c\u044b \u0436\u0438\u0432\u0435\u043c \u0432 \u0443\u0434\u0438\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f, \u043a\u043e\u0433\u0434\u0430 \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0441\u0442\u044b\u043a\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0433\u043e\u0442\u043e\u0432\u044b\u0445 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438\u0445 \u0441 \u00ab\u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u043c \u0441\u043e\u0437\u043d\u0430\u043d\u0438\u0435\u043c\u00bb \u043f\u043e \u0441\u043e\u0432\u0435\u0442\u0430\u043c stackoverflow, \u043d\u0435 \u0432\u043d\u0438\u043a\u0430\u044f \u0432 \u00ab\u043c\u043d\u043e\u0433\u043e\u0431\u0443\u043a\u0432\u00bb, \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-08-28T05:42:10+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-08-28T05:42:10+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Si organizuam nj\u00eb DataLake t\u00eb lart\u00eb efektiv dhe t\u00eb lir\u00eb dhe pse pik\u00ebrisht ashtu | ProHoster","description":"Ne jetojm\u00eb n\u00eb nj\u00eb koh\u00eb t\u00eb jasht\u00ebzakonshme, kur mund t\u00eb lidhen shpejt dhe thjesht\u00eb disa mjete t\u00eb hapura t\u00eb gatshme, t'i konfigurojm\u00eb me \"nd\u00ebrgjegje t\u00eb \u00e7aktivizuar\" sipas k\u00ebshillave t\u00eb stackoverflow, pa u thelluar n\u00eb \"terma t\u00eb gjat\u00eb\", dhe t\u00eb nisim.","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041a\u0430\u043a \u043c\u044b \u043e\u0440\u0433\u0430\u043d\u0438\u0437\u043e\u0432\u0430\u043b\u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u044d\u0444\u0444\u0435\u043a\u0442\u0438\u0432\u043d\u043e\u0435 \u0438 \u043d\u0435\u0434\u043e\u0440\u043e\u0433\u043e\u0435 DataLake \u0438 \u043f\u043e\u0447\u0435\u043c\u0443 \u0438\u043c\u0435\u043d\u043d\u043e \u0442\u0430\u043a | ProHoster","og:description":"\u041c\u044b \u0436\u0438\u0432\u0435\u043c \u0432 \u0443\u0434\u0438\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f, \u043a\u043e\u0433\u0434\u0430 \u043c\u043e\u0436\u043d\u043e \u0431\u044b\u0441\u0442\u0440\u043e \u0438 \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0441\u0442\u044b\u043a\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0441\u043a\u043e\u043b\u044c\u043a\u043e \u0433\u043e\u0442\u043e\u0432\u044b\u0445 \u043e\u0442\u043a\u0440\u044b\u0442\u044b\u0445 \u0438\u043d\u0441\u0442\u0440\u0443\u043c\u0435\u043d\u0442\u043e\u0432, \u043d\u0430\u0441\u0442\u0440\u043e\u0438\u0442\u044c \u0438\u0445 \u0441 \u00ab\u043e\u0442\u043a\u043b\u044e\u0447\u0435\u043d\u043d\u044b\u043c \u0441\u043e\u0437\u043d\u0430\u043d\u0438\u0435\u043c\u00bb \u043f\u043e \u0441\u043e\u0432\u0435\u0442\u0430\u043c stackoverflow, \u043d\u0435 \u0432\u043d\u0438\u043a\u0430\u044f \u0432 \u00ab\u043c\u043d\u043e\u0433\u043e\u0431\u0443\u043a\u0432\u00bb, \u0437\u0430\u043f\u0443\u0441\u0442\u0438\u0442\u044c \u0432.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-my-organizovali-vysokoeffektivnoe-i-nedorogoe-datalake-i-pochemu-imenno-tak","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-08-28T05:42:10+00:00","article:modified_time":"2020-08-28T05:42:10+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"92508","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 12:07:39","updated":"2022-10-01 09:50:53","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/92508","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=92508"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/92508\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/92509"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=92508"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=92508"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=92508"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}