{"id":90783,"date":"2020-08-06T01:42:19","date_gmt":"2020-08-05T23:42:19","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1"},"modified":"2020-08-06T01:42:19","modified_gmt":"2020-08-05T23:42:19","slug":"kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1","title":{"rendered":"Si BigQuery nga Google demokratizoi analiz\u00ebn e t\u00eb dh\u00ebnave. Pjesa 1","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><b><i>P\u00ebrsh\u00ebndetje, Habr! Aktualisht n\u00eb OTUS \u00ebsht\u00eb hapur regjistrimi p\u00ebr nj\u00eb grup t\u00eb ri t\u00eb kursit <noindex><a rel=\"nofollow\" href=\"https:\/\/otus.pw\/wDZQ\/\">Inxhinier i t\u00eb Dh\u00ebnave<\/a><\/noindex>. N\u00eb prag t\u00eb fillimit t\u00eb kursit, ne kemi p\u00ebrgatitur tradicionalisht p\u00ebr ju nj\u00eb p\u00ebrkthim t\u00eb nj\u00eb materiali interesant. <\/i><\/b><\/p>\n<p>\n\u00c7do dit\u00eb m\u00eb shum\u00eb se nj\u00ebqind milion njer\u00ebz vizitojn\u00eb Twitter-in p\u00ebr t\u00eb m\u00ebsuar se \u00e7far\u00eb po ndodh n\u00eb bot\u00eb dhe p\u00ebr ta diskutuar at\u00eb. \u00c7do tweet dhe \u00e7do veprim tjet\u00ebr i p\u00ebrdoruesit gjeneron nj\u00eb ngjarje, e cila \u00ebsht\u00eb e disponueshme p\u00ebr analiz\u00ebn e brendshme t\u00eb t\u00eb dh\u00ebnave n\u00eb Twitter. Qindra punonj\u00ebs analizojn\u00eb dhe vizualizojn\u00eb k\u00ebto t\u00eb dh\u00ebna dhe p\u00ebrmir\u00ebsimi i p\u00ebrvoj\u00ebs s\u00eb tyre \u00ebsht\u00eb prioriteti kryesor p\u00ebr ekipin e Platform\u00ebs s\u00eb t\u00eb Dh\u00ebnave t\u00eb Twitter. <\/p>\n<p>Ne mendojm\u00eb se p\u00ebrdoruesit me nj\u00eb gam\u00eb t\u00eb gjer\u00eb aft\u00ebsish teknike duhet t\u00eb ken\u00eb mund\u00ebsin\u00eb t\u00eb gjejn\u00eb t\u00eb dh\u00ebnat dhe t\u00eb ken\u00eb qasje n\u00eb mjete t\u00eb mira p\u00ebr analiz\u00eb dhe vizualizim t\u00eb bazuara n\u00eb SQL. Kjo do t'i lejonte nj\u00eb grup t\u00eb ri p\u00ebrdoruesish me nj\u00eb zbehje m\u00eb t\u00eb vog\u00ebl teknike, duke p\u00ebrfshir\u00eb analist\u00ebt e t\u00eb dh\u00ebnave dhe menaxher\u00ebt e produkteve, t\u00eb nxjerrin informacion nga t\u00eb dh\u00ebnat, duke u lejuar atyre t\u00eb kuptojn\u00eb dhe t\u00eb p\u00ebrdorin m\u00eb mir\u00eb mund\u00ebsit\u00eb e Twitter. K\u00ebshtu po demokratizojm\u00eb analiz\u00ebn e t\u00eb dh\u00ebnave n\u00eb Twitter. <noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p>Me p\u00ebrmir\u00ebsimin e mjeteve tona dhe mund\u00ebsive p\u00ebr analiz\u00ebn e brendshme t\u00eb t\u00eb dh\u00ebnave, kemi par\u00eb p\u00ebrmir\u00ebsime n\u00eb sh\u00ebrbimin e Twitter-it. Megjithat\u00eb, ka ende shum\u00eb p\u00ebr t\u00eb b\u00ebr\u00eb. Mjetet aktuale, si Scalding, k\u00ebrkojn\u00eb p\u00ebrvoj\u00eb n\u00eb programim. Mjetet p\u00ebr analiz\u00ebn e bazuar n\u00eb SQL, si Presto dhe Vertica, kan\u00eb probleme me performanc\u00ebn n\u00eb nj\u00eb shkall\u00eb t\u00eb madhe. Ne gjithashtu kemi nj\u00eb problem me shp\u00ebrndarjen e t\u00eb dh\u00ebnave p\u00ebrmes disa sistemeve pa nj\u00eb qasje t\u00eb vazhdueshme n\u00eb to.<\/p>\n<p>Vit t\u00eb kaluar ne shpall\u00ebm <noindex><a rel=\"nofollow\" href=\"https:\/\/blog.twitter.com\/engineering\/en_us\/topics\/infrastructure\/2018\/a-new-collaboration-with-google-cloud.html\">nj\u00eb bashk\u00ebpunim t\u00eb ri me Google<\/a><\/noindex>, n\u00eb kuad\u00ebr t\u00eb t\u00eb cilit po transferojm\u00eb pjes\u00eb t\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/blog.twitter.com\/engineering\/en_us\/topics\/infrastructure\/2019\/the-start-of-a-journey-into-the-cloud.html\">infrastruktur\u00ebs son\u00eb t\u00eb t\u00eb dh\u00ebnave<\/a><\/noindex> n\u00eb Google Cloud Platform (GCP). Ne arrit\u00ebm n\u00eb p\u00ebrfundimin se mjetet e Google Cloud <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/products\/big-data\/\">Big Data<\/a><\/noindex> mund t\u00eb na ndihmojn\u00eb n\u00eb iniciativat tona p\u00ebr demokratizimin e analiz\u00ebs, vizualizimit dhe m\u00ebsimit t\u00eb makinerive n\u00eb Twitter:<\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery\/\">BigQuery<\/a><\/noindex>: nj\u00eb depo e t\u00eb dh\u00ebnave t\u00eb korporatave me nj\u00eb motor SQL t\u00eb bazuar n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/ai.google\/research\/pubs\/pub36632\">Dremel<\/a><\/noindex>, i njohur p\u00ebr shpejt\u00ebsin\u00eb, thjesht\u00ebsin\u00eb dhe p\u00ebrballimin e <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery-ml\/docs\/bigqueryml-intro\">m\u00ebsimit t\u00eb makinave<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/datastudio.google.com\/overview\">Data Studio:<\/a><\/noindex> nj\u00eb mjet p\u00ebr vizualizimin e t\u00eb dh\u00ebnave t\u00eb m\u00ebdha me funksione bashk\u00ebpunuese, si n\u00eb Google Docs.<\/li>\n<\/ul>\n<p>\nNga ky artikull do t\u00eb m\u00ebsoni p\u00ebr p\u00ebrvoj\u00ebn ton\u00eb me k\u00ebto mjete: \u00e7far\u00eb kemi b\u00ebr\u00eb, \u00e7far\u00eb kemi m\u00ebsuar dhe \u00e7far\u00eb do t\u00eb b\u00ebjm\u00eb m\u00eb pas. Tani do t\u00eb p\u00ebrqendrohemi n\u00eb analiz\u00ebn e grupuar dhe interaktive. Analiz\u00ebn n\u00eb koh\u00eb reale do ta diskutojm\u00eb n\u00eb artikullin e ardhsh\u00ebm.<\/p>\n<h2>Historia e depozitave t\u00eb t\u00eb dh\u00ebnave n\u00eb Twitter<\/h2>\n<p>\nPara se t\u00eb thellohemi n\u00eb BigQuery, vlen t\u00eb p\u00ebrmendet shkurtimisht historia e depozitave t\u00eb t\u00eb dh\u00ebnave n\u00eb Twitter. N\u00eb vitin 2011, analiza e t\u00eb dh\u00ebnave n\u00eb Twitter b\u00ebhej n\u00eb Vertica dhe Hadoop. P\u00ebr t\u00eb krijuar MapReduce p\u00ebr funksionin Hadoop, ne p\u00ebrdor\u00ebm Pig. N\u00eb vitin 2012, e z\u00ebvend\u00ebsuam Pig me Scalding, i cili kishte nj\u00eb API n\u00eb Scala me p\u00ebrpar\u00ebsi si krijimi i tubacioneve komplekse dhe leht\u00ebsia e testimit. Sidoqoft\u00eb, p\u00ebr shum\u00eb analist\u00eb t\u00eb t\u00eb dh\u00ebnave dhe menaxher\u00eb produktesh, t\u00eb cil\u00ebt ndiheshin m\u00eb rehat t\u00eb punonin me SQL, kjo was nj\u00eb p\u00ebrshtatje e v\u00ebshtir\u00eb e m\u00ebsimit. Rreth vitit 2016 nisi p\u00ebrdorimi i Presto si nd\u00ebrfaqe SQL p\u00ebr t\u00eb dh\u00ebnat e Hadoop. Spark ofronte nj\u00eb nd\u00ebrfaqe n\u00eb Python, q\u00eb e b\u00ebnte at\u00eb nj\u00eb zgjedhje t\u00eb mir\u00eb p\u00ebr k\u00ebrkime ad hoc t\u00eb t\u00eb dh\u00ebnave dhe m\u00ebsim makinerie.<\/p>\n<p>Q\u00eb nga viti 2018 kemi p\u00ebrdorur k\u00ebto mjete p\u00ebr analiz\u00ebn dhe vizualizimin e t\u00eb dh\u00ebnave:<\/p>\n<ul>\n<li>Scalding p\u00ebr tubacionet prodhuese<\/li>\n<li>Scalding dhe Spark p\u00ebr analiza ad hoc t\u00eb t\u00eb dh\u00ebnave dhe m\u00ebsim makinerie<\/li>\n<li>Vertica dhe Presto p\u00ebr analiz\u00eb ad hoc dhe interaktive SQL <\/li>\n<li>Druid p\u00ebr qasje interaktive t\u00eb vog\u00ebl, eksploruese dhe me vones\u00eb t\u00eb vog\u00ebl n\u00eb metrikat e serive temporale<\/li>\n<li>Tableau, Zeppelin dhe Pivot p\u00ebr vizualizimin e t\u00eb dh\u00ebnave<\/li>\n<\/ul>\n<p>\nKemi zbuluar se, nd\u00ebrsa k\u00ebto mjete ofrojn\u00eb mund\u00ebsi shum\u00eb t\u00eb fuqishme, kemi hasur v\u00ebshtir\u00ebsi n\u00eb realizimin e aksesit t\u00eb k\u00ebtyre mund\u00ebsive p\u00ebr nj\u00eb audienc\u00eb m\u00eb t\u00eb gjer\u00eb n\u00eb Twitter. Duke zgjeruar platform\u00ebn ton\u00eb me Google Cloud, ne p\u00ebrqendrohemi n\u00eb thjeshtimin e mjeteve tona analitike p\u00ebr t\u00eb gjith\u00eb Twitterin.<\/p>\n<h2>Depozita e dh\u00ebnave BigQuery nga Google <\/h2>\n<p>\nDisa prej ekipeve n\u00eb Twitter tashm\u00eb e kan\u00eb p\u00ebrfshir\u00eb BigQuery n\u00eb disa nga pipeline-t e tyre t\u00eb prodhimit. Duke p\u00ebrdorur p\u00ebrvoj\u00ebn e tyre, ne filluam t\u00eb vler\u00ebsojm\u00eb mund\u00ebsit\u00eb e BigQuery p\u00ebr t\u00eb gjitha skenaret e p\u00ebrdorimit n\u00eb Twitter. Q\u00ebllimi yn\u00eb ishte t\u00eb ofronim BigQuery p\u00ebr t\u00eb gjith\u00eb kompanin\u00eb, si dhe t\u00eb standardizonim dhe mb\u00ebshtetnim at\u00eb brenda grupit t\u00eb mjeteve t\u00eb Platform\u00ebs s\u00eb t\u00eb Dh\u00ebnave. Kjo ishte e v\u00ebshtir\u00eb p\u00ebr shum\u00eb arsye. Na nevojitej t\u00eb zhvillonim nj\u00eb infrastruktur\u00eb p\u00ebr t\u00eb pranuar n\u00eb m\u00ebnyr\u00eb t\u00eb besueshme sasi t\u00eb m\u00ebdha t\u00eb dh\u00ebnash, p\u00ebr t\u00eb mb\u00ebshtetur menaxhimin e t\u00eb dh\u00ebnave n\u00eb shkall\u00eb t\u00eb gjer\u00eb, p\u00ebr t\u00eb siguruar kontroll t\u00eb duhur t\u00eb aksesit dhe p\u00ebr t\u00eb ruajtur privat\u00ebsin\u00eb e klient\u00ebve. Ne gjithashtu duhej t\u00eb krijonim sisteme p\u00ebr shp\u00ebrndarjen e burimeve, monitorimin dhe rikthimin e pagesave, n\u00eb m\u00ebnyr\u00eb q\u00eb ekipet t\u00eb mund t\u00eb p\u00ebrdornin n\u00eb m\u00ebnyr\u00eb efektive BigQuery.<\/p>\n<p>N\u00eb n\u00ebntor 2018, ne l\u00ebshuam nj\u00eb version alfa t\u00eb BigQuery dhe Data Studio p\u00ebr t\u00eb gjith\u00eb kompanin\u00eb. Ne u ofruam punonj\u00ebsve t\u00eb Twitter disa nga tabelat tona t\u00eb p\u00ebrdorura m\u00eb shpesh me t\u00eb dh\u00ebna personale t\u00eb pastra. BigQuery e kan\u00eb p\u00ebrdorur m\u00eb shum\u00eb se 250 p\u00ebrdorues nga ekipe t\u00eb ndryshme, duke p\u00ebrfshir\u00eb inxhinieri, financa dhe marketing. S\u00eb fundmi, ata realizuan rreth 8,000 k\u00ebrkesa, duke p\u00ebrpunuar rreth 100 PB n\u00eb muaj, p\u00ebrve\u00e7 k\u00ebrkesave t\u00eb planifikuara. Duke marr\u00eb feedback shum\u00eb pozitiv, ne vendos\u00ebm t\u00eb vazhdonim p\u00ebrpara dhe t\u00eb ofronim BigQuery si burimin kryesor p\u00ebr nd\u00ebrveprimin me t\u00eb dh\u00ebnat n\u00eb Twitter.<\/p>\n<p>K\u00ebtu \u00ebsht\u00eb skema e arkitektur\u00ebs s\u00eb nivelit t\u00eb lart\u00eb t\u00eb depozit\u00ebs son\u00eb t\u00eb t\u00eb dh\u00ebnave Google BigQuery. <\/p>\n<p><img decoding=\"async\" alt=\"Si BigQuery nga Google demokratizoi analiz\u00ebn e t\u00eb dh\u00ebnave. Pjesa 1\" src=\"\/wp-content\/uploads\/2020\/08\/6a3e0b43752ebbac641c43519c8c2fba.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nNe kopjojm\u00eb t\u00eb dh\u00ebnat nga klaster\u00ebt lokal\u00eb Hadoop n\u00eb Google Cloud Storage (GCS), duke p\u00ebrdorur nj\u00eb mjet t\u00eb brendsh\u00ebm Cloud Replicator. M\u00eb pas, ne p\u00ebrdorim Apache Airflow p\u00ebr t\u00eb krijuar pipeline-t q\u00eb p\u00ebrdorin \"<noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery\/docs\/reference\/bq-cli-reference#bq_load\">bq_load<\/a><\/noindex>\" p\u00ebr t\u00eb ngarkuar t\u00eb dh\u00ebnat nga GCS n\u00eb BigQuery. Ne p\u00ebrdorim Presto p\u00ebr t\u00eb pyetur grupe t\u00eb dh\u00ebnash Parquet ose Thrift-LZO n\u00eb GCS. BQ Blaster \u00ebsht\u00eb nj\u00eb mjet i brendsh\u00ebm Scalding p\u00ebr ngarkimin e grupeve t\u00eb dh\u00ebnash HDFS Vertica dhe Thrift-LZO n\u00eb BigQuery.<\/p>\n<p>N\u00eb seksionet n\u00eb vazhdim do t\u00eb diskutojm\u00eb qasjen ton\u00eb dhe njohurit\u00eb n\u00eb lidhje me leht\u00ebsin\u00eb e p\u00ebrdorimit, performanc\u00ebn, menaxhimin e t\u00eb dh\u00ebnave, disponueshm\u00ebrin\u00eb e sistemit dhe kostot.<\/p>\n<h4>Leht\u00ebsia e p\u00ebrdorimit<\/h4>\n<p>\nNe kemi zbuluar se p\u00ebrdoruesit ishin t\u00eb leht\u00eb t\u00eb fillonin me BigQuery, pasi nuk k\u00ebrkonte instalimin e softuerit dhe p\u00ebrdoruesit mund t\u00eb kishin qasje n\u00eb t\u00eb p\u00ebrmes nj\u00eb nd\u00ebrfaqeje intuitive n\u00eb internet. Megjithat\u00eb, ishte e nevojshme q\u00eb p\u00ebrdoruesit t\u00eb njiheshin me disa funksione t\u00eb GCP dhe konceptet e tij, t\u00eb p\u00ebrfshira burime si projektet, grumbujt e t\u00eb dh\u00ebnave dhe tabelat. Ne zhvilluam materiale m\u00ebsimore dhe tutoriale p\u00ebr t\u00eb ndihmuar p\u00ebrdoruesit t\u00eb fillonin pun\u00ebn. Me nj\u00eb kuptim themelor, p\u00ebrdoruesit e gjet\u00ebn m\u00eb t\u00eb leht\u00eb t\u00eb l\u00ebviznin n\u00ebp\u00ebr grumbujt e t\u00eb dh\u00ebnave, t\u00eb shihnin skem\u00ebn dhe t\u00eb dh\u00ebnat e tabelave, t\u00eb kryenin k\u00ebrkesa t\u00eb thjeshta dhe t\u00eb vizualizonin rezultatet n\u00eb Data Studio.<\/p>\n<p>Q\u00ebllimi yn\u00eb n\u00eb lidhje me futjen e t\u00eb dh\u00ebnave n\u00eb BigQuery ishte q\u00eb t\u00eb sigurohej ngarkimi i qet\u00eb i grumbujve t\u00eb t\u00eb dh\u00ebnave HDFS ose GCS me nj\u00eb klik t\u00eb vet\u00ebm. Ne shqyrtuam <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/composer\/\">Cloud Composer<\/a><\/noindex> (Airflow i menaxhuar), por nuk arrit\u00ebm ta p\u00ebrdornim p\u00ebr shkak t\u00eb modelit ton\u00eb t\u00eb siguris\u00eb 'Domain Restricted Sharing' (m\u00eb shum\u00eb mbi k\u00ebt\u00eb n\u00eb seksionin 'Menaxhimi i t\u00eb Dh\u00ebnave' m\u00eb posht\u00eb). Ne eksperimentuam me p\u00ebrdorimin e Sh\u00ebrbimit t\u00eb Transferimit t\u00eb T\u00eb Dh\u00ebnave t\u00eb Google (DTS) p\u00ebr t\u00eb organizuar detyrat e ngarkesave n\u00eb BigQuery. Nd\u00ebrsa DTS u konfiguruar shpejt, nuk ishte fleksib\u00ebl p\u00ebr nd\u00ebrtimin e konvejereve me var\u00ebsi. P\u00ebr versionin ton\u00eb alfa, ne krijuam nj\u00eb ambient tonin Apache Airflow n\u00eb GCE dhe po e p\u00ebrgatisim p\u00ebr prodhim dhe p\u00ebr mund\u00ebsit\u00eb p\u00ebr t\u00eb mb\u00ebshtetur m\u00eb shum\u00eb burime t\u00eb dh\u00ebnash si Vertica.<\/p>\n<p>P\u00ebr t\u00eb transformuar t\u00eb dh\u00ebnat n\u00eb BigQuery, p\u00ebrdoruesit krijojn\u00eb kontejner\u00eb t\u00eb thjesht\u00eb t\u00eb t\u00eb dh\u00ebnave SQL, duke p\u00ebrdorur k\u00ebrkesa t\u00eb planifikuara. P\u00ebr kontejner\u00eb t\u00eb nd\u00ebrlikuar me shum\u00eb hapa me var\u00ebsi, ne planifikojm\u00eb t\u00eb p\u00ebrdorim ose infrastruktur\u00ebn ton\u00eb Airflow ose Cloud Composer s\u00eb bashku me <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/dataflow\/\">Cloud Dataflow<\/a><\/noindex>.<\/p>\n<h4>Performanca<\/h4>\n<p>\nBigQuery \u00ebsht\u00eb krijuar p\u00ebr k\u00ebrkesa SQL p\u00ebr p\u00ebrdorim t\u00eb p\u00ebrgjithsh\u00ebm, q\u00eb p\u00ebrpunojn\u00eb sasi t\u00eb m\u00ebdha t\u00eb dh\u00ebnash. Ai nuk \u00ebsht\u00eb i dizajnuar p\u00ebr k\u00ebrkesa me vonesa t\u00eb ul\u00ebta, me kapacitet t\u00eb lart\u00eb q\u00eb jan\u00eb t\u00eb nevojshme p\u00ebr bazat e t\u00eb dh\u00ebnave transaksionale, ose p\u00ebr analiz\u00ebn e serive temporale me vones\u00eb t\u00eb ul\u00ebt, q\u00eb \u00ebsht\u00eb realizuar <noindex><a rel=\"nofollow\" href=\"https:\/\/druid.apache.org\/\">Apache Druid<\/a><\/noindex>. N\u00eb pyetjet interaktive analitike, p\u00ebrdoruesit tan\u00eb presin nj\u00eb koh\u00eb p\u00ebrgjigjeje m\u00eb pak se nj\u00eb minut\u00eb. Duhej t\u00eb dizajnojm\u00eb p\u00ebrdorimin e BigQuery n\u00eb m\u00ebnyr\u00eb q\u00eb t\u00eb p\u00ebrputhej me k\u00ebto pritshm\u00ebri. P\u00ebr t\u00eb siguruar nj\u00eb performanc\u00eb t\u00eb parashikueshme p\u00ebr p\u00ebrdoruesit tan\u00eb, p\u00ebrdor\u00ebm funksionalitetin e BigQuery q\u00eb \u00ebsht\u00eb n\u00eb dispozicion p\u00ebr klient\u00ebt me pages\u00eb fikse, e cila lejon pronar\u00ebt e projekteve t\u00eb rezervojn\u00eb nj\u00eb num\u00ebr minimal slots p\u00ebr pyetjet e tyre. <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery\/docs\/slots\">Slot<\/a><\/noindex> BigQuery \u00ebsht\u00eb nj\u00eb nj\u00ebsi e fuqis\u00eb llogarit\u00ebse q\u00eb nevojitet p\u00ebr t\u00eb kryer pyetje SQL. <\/p>\n<p>Analizuam m\u00eb shum\u00eb se 800 pyetje q\u00eb procesojn\u00eb rreth 1 TB t\u00eb dh\u00ebnash \u00e7do nj\u00eb, dhe zbuluam se koha mesatare e ekzekutimit ishte 30 sekonda. M\u00ebsojm\u00eb gjithashtu se performanca varet shum\u00eb nga p\u00ebrdorimi i slot-it ton\u00eb n\u00eb projekte dhe detyra t\u00eb ndryshme. Duhej t\u00eb ndanim qart\u00eb rezervat tona t\u00eb slot-ve p\u00ebr prodhim dhe ad hoc, p\u00ebr t\u00eb mb\u00ebshtetur performanc\u00ebn p\u00ebr skenaret e p\u00ebrdorimit prodhues dhe analiz\u00ebn interaktive. Kjo kishte nj\u00eb ndikim t\u00eb fort\u00eb n\u00eb dizajnin ton\u00eb p\u00ebr rezervimin e slot-ve dhe hierarkin\u00eb e projekteve.<\/p>\n<p><i>P\u00ebr menaxhimin e t\u00eb dh\u00ebnave, funksionalitetin dhe koston e sistemeve, do t\u00eb flasim n\u00eb dit\u00ebt n\u00eb vijim n\u00eb pjes\u00ebn e dyt\u00eb t\u00eb p\u00ebrkthimit, por tani ftojm\u00eb t\u00eb gjith\u00eb t\u00eb interesuarit n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/otus.pw\/wDZQ\/\">webinari t\u00eb drejtp\u00ebrdrejt falas<\/a><\/noindex>, n\u00eb kuad\u00ebr t\u00eb cilit do t\u00eb keni mund\u00ebsi t\u00eb m\u00ebsoni n\u00eb detaje rreth kursit, si dhe t\u00eb b\u00ebni pyetje ekspertit ton\u00eb \u2014 Yegor Mateushku (Inxhinier i t\u00eb Dh\u00ebnave t\u00eb Larta, MaximaTelecom). <\/i><\/p>\n<p><\/p>\n<h2>Lexoni m\u00eb shum\u00eb:<\/h2>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/501380\/\">Data Build Tool ose \u00e7far\u00eb kan\u00eb t\u00eb p\u00ebrbashk\u00ebt Depozita e t\u00eb Dh\u00ebnave dhe Smoothie<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/502324\/\">The Role of Delta Lake: Schema Enforcement and Evolution<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/503132\/\">Apache Parquet me shpejt\u00ebsi t\u00eb lart\u00eb n\u00eb Python me Apache Arrow<\/a><\/noindex><\/li>\n<\/ul>\n<p>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/513780\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u044f\u043c\u043e \u0441\u0435\u0439\u0447\u0430\u0441 \u0432 OTUS \u043e\u0442\u043a\u0440\u044b\u0442 \u043d\u0430\u0431\u043e\u0440 \u043d\u0430 \u043d\u043e\u0432\u044b\u0439 \u043f\u043e\u0442\u043e\u043a \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb. \u0412 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 \u043c\u044b \u0442\u0440\u0430\u0434\u0438\u0446\u0438\u043e\u043d\u043d\u043e \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u0438\u043b\u0438 \u0434\u043b\u044f \u0432\u0430\u0441 \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u0438\u043d\u0442\u0435\u0440\u0435\u0441\u043d\u043e\u0433\u043e \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0430. \u041a\u0430\u0436\u0434\u044b\u0439 \u0434\u0435\u043d\u044c \u0431\u043e\u043b\u0435\u0435 \u0441\u0442\u0430 \u043c\u0438\u043b\u043b\u0438\u043e\u043d\u043e\u0432 \u0447\u0435\u043b\u043e\u0432\u0435\u043a \u043f\u043e\u0441\u0435\u0449\u0430\u044e\u0442 Twitter, \u0447\u0442\u043e\u0431\u044b \u0443\u0437\u043d\u0430\u0442\u044c, \u0447\u0442\u043e \u043f\u0440\u043e\u0438\u0441\u0445\u043e\u0434\u0438\u0442 \u0432 \u043c\u0438\u0440\u0435, \u0438 \u043e\u0431\u0441\u0443\u0434\u0438\u0442\u044c \u044d\u0442\u043e. \u041a\u0430\u0436\u0434\u044b\u0439 \u0442\u0432\u0438\u0442 \u0438 \u043b\u044e\u0431\u043e\u0435 \u0434\u0440\u0443\u0433\u043e\u0435 \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044f \u0433\u0435\u043d\u0435\u0440\u0438\u0440\u0443\u044e\u0442 \u0441\u043e\u0431\u044b\u0442\u0438\u0435, \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0435 \u0434\u043b\u044f \u0432\u043d\u0443\u0442\u0440\u0435\u043d\u043d\u0435\u0433\u043e [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":90784,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-90783","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u044f\u043c\u043e \u0441\u0435\u0439\u0447\u0430\u0441 \u0432 OTUS \u043e\u0442\u043a\u0440\u044b\u0442 \u043d\u0430\u0431\u043e\u0440 \u043d\u0430 \u043d\u043e\u0432\u044b\u0439 \u043f\u043e\u0442\u043e\u043a \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041a\u0430\u043a BigQuery \u043e\u0442 Google \u0434\u0435\u043c\u043e\u043a\u0440\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043b \u0430\u043d\u0430\u043b\u0438\u0437 \u0434\u0430\u043d\u043d\u044b\u0445. \u0427\u0430\u0441\u0442\u044c 1 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u044f\u043c\u043e \u0441\u0435\u0439\u0447\u0430\u0441 \u0432 OTUS \u043e\u0442\u043a\u0440\u044b\u0442 \u043d\u0430\u0431\u043e\u0440 \u043d\u0430 \u043d\u043e\u0432\u044b\u0439 \u043f\u043e\u0442\u043e\u043a \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-08-05T23:42:19+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-08-05T23:42:19+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Si si BigQuery nga Google demokratizoi analiz\u00ebn e t\u00eb dh\u00ebnave. Pjesa 1 | ProHoster","description":"P\u00ebrsh\u00ebndetje, Habr! Tani p\u00ebr tani, n\u00eb OTUS \u00ebsht\u00eb e hapur regjistrimi p\u00ebr grupin e ri t\u00eb kursit \"Inxhinier i T\u00eb Dh\u00ebnave\".","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041a\u0430\u043a BigQuery \u043e\u0442 Google \u0434\u0435\u043c\u043e\u043a\u0440\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043b \u0430\u043d\u0430\u043b\u0438\u0437 \u0434\u0430\u043d\u043d\u044b\u0445. \u0427\u0430\u0441\u0442\u044c 1 | ProHoster","og:description":"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u044f\u043c\u043e \u0441\u0435\u0439\u0447\u0430\u0441 \u0432 OTUS \u043e\u0442\u043a\u0440\u044b\u0442 \u043d\u0430\u0431\u043e\u0440 \u043d\u0430 \u043d\u043e\u0432\u044b\u0439 \u043f\u043e\u0442\u043e\u043a \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb.","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-08-05T23:42:19+00:00","article:modified_time":"2020-08-05T23:42:19+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"90783","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 12:42:25","updated":"2022-10-03 03:00:18","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/90783","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=90783"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/90783\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/90784"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=90783"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=90783"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=90783"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}