{"id":90783,"date":"2020-08-06T01:42:19","date_gmt":"2020-08-05T23:42:19","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1"},"modified":"2020-08-06T01:42:19","modified_gmt":"2020-08-05T23:42:19","slug":"kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1","status":"publish","type":"post","link":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1","title":{"rendered":"Jak BigQuery od Google demokratizoval anal\u00fdzu dat. \u010c\u00e1st 1","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><b><i>Ahoj, Habr! Pr\u00e1v\u011b te\u010f je v OTUS otev\u0159en\u00e1 registrace na nov\u00fd b\u011bh kurzu <noindex><a rel=\"nofollow\" href=\"https:\/\/otus.pw\/wDZQ\/\">\u201eDatov\u00fd in\u017een\u00fdr\u201c<\/a><\/noindex>P\u0159ed za\u010d\u00e1tkem kurzu jsme tradi\u010dn\u011b p\u0159ipravili p\u0159eklad zaj\u00edmav\u00e9ho materi\u00e1lu pro v\u00e1s. <\/i><\/b><\/p>\n<p>\nKa\u017ed\u00fd den nav\u0161t\u00edv\u00ed Twitter v\u00edce ne\u017e sto mili\u00f3n\u016f lid\u00ed, aby se dozv\u011bd\u011bli, co se d\u011bje ve sv\u011bt\u011b, a o tom diskutovali. Ka\u017ed\u00fd tweet a jak\u00e1koli jin\u00e1 u\u017eivatelsk\u00e1 akce generuj\u00ed ud\u00e1lost, kter\u00e1 je k dispozici pro intern\u00ed anal\u00fdzu dat na Twitteru. Stovky zam\u011bstnanc\u016f tyto \u00fadaje analyzuj\u00ed a vizualizuj\u00ed, a zlep\u0161en\u00ed jejich zku\u0161enost\u00ed je hlavn\u00edm prioritou t\u00fdmu Twitter Data Platform. <\/p>\n<p>Domn\u00edv\u00e1me se, \u017ee u\u017eivatel\u00e9 s \u0161irok\u00fdm spektrum technick\u00fdch dovednost\u00ed by m\u011bli m\u00edt mo\u017enost naj\u00edt data a m\u00edt p\u0159\u00edstup k dob\u0159e funguj\u00edc\u00edm n\u00e1stroj\u016fm pro anal\u00fdzu a vizualizaci zalo\u017een\u00fdm na SQL. To by umo\u017enilo cel\u00e9 nov\u00e9 skupin\u011b u\u017eivatel\u016f s men\u0161\u00edm technick\u00fdm z\u00e1zem\u00edm, v\u010detn\u011b datov\u00fdch analytik\u016f a produktov\u00fdch mana\u017eer\u016f, \u010derpat informace z dat, co\u017e by jim umo\u017enilo l\u00e9pe porozum\u011bt a vyu\u017e\u00edvat mo\u017enosti Twitteru. Takto demokrati\u010dt\u011bji p\u0159istupujeme k anal\u00fdze dat na Twitteru. <noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p>S postupn\u00fdm vylep\u0161ov\u00e1n\u00edm na\u0161ich n\u00e1stroj\u016f a schopnost\u00ed pro intern\u00ed anal\u00fdzu dat jsme byli sv\u011bdky zlep\u0161en\u00ed slu\u017eby Twitter. N\u011bco v\u0161ak st\u00e1le zb\u00fdv\u00e1. Sou\u010dasn\u00e9 n\u00e1stroje, jako je Scalding, vy\u017eaduj\u00ed program\u00e1torsk\u00e9 dovednosti. N\u00e1stroje pro anal\u00fdzu zalo\u017een\u00e9 na SQL, jako jsou Presto a Vertica, maj\u00ed probl\u00e9my s v\u00fdkonem ve velk\u00e9m m\u011b\u0159\u00edtku. Tak\u00e9 m\u00e1me probl\u00e9m s distribuc\u00ed dat mezi r\u016fzn\u00fdmi syst\u00e9my bez trval\u00e9ho p\u0159\u00edstupu k nim.<\/p>\n<p>Minul\u00fd rok jsme ozn\u00e1mili <noindex><a rel=\"nofollow\" href=\"https:\/\/blog.twitter.com\/engineering\/en_us\/topics\/infrastructure\/2018\/a-new-collaboration-with-google-cloud.html\">nov\u00e9 spolupr\u00e1ce s Googlem<\/a><\/noindex>, v r\u00e1mci kter\u00e9 p\u0159esouv\u00e1me \u010d\u00e1sti na\u0161\u00ed <noindex><a rel=\"nofollow\" href=\"https:\/\/blog.twitter.com\/engineering\/en_us\/topics\/infrastructure\/2019\/the-start-of-a-journey-into-the-cloud.html\">datov\u00e9 infrastruktury<\/a><\/noindex> na Google Cloud Platform (GCP). Do\u0161li jsme k z\u00e1v\u011bru, \u017ee n\u00e1stroje Google Cloud <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/products\/big-data\/\">Big Data<\/a><\/noindex> n\u00e1s mohou podpo\u0159it v na\u0161ich iniciativ\u00e1ch zam\u011b\u0159en\u00fdch na demokratizaci anal\u00fdzy, vizualizace a strojov\u00e9ho u\u010den\u00ed na Twitteru:<\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery\/\">BigQuery<\/a><\/noindex>: \u00falo\u017ei\u0161t\u011b podnikov\u00fdch dat se SQL enginem zalo\u017een\u00fdm na <noindex><a rel=\"nofollow\" href=\"https:\/\/ai.google\/research\/pubs\/pub36632\">Dremelu<\/a><\/noindex>, kter\u00e9 se vyzna\u010duje rychlost\u00ed a jednoduchost\u00ed a vyrovn\u00e1 se s <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery-ml\/docs\/bigqueryml-intro\">strojov\u00e9ho u\u010den\u00ed<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/datastudio.google.com\/overview\">Data Studio:<\/a><\/noindex> n\u00e1stroj pro vizualizaci velk\u00fdch dat s funkcemi spolupr\u00e1ce, jako v Google Docs.<\/li>\n<\/ul>\n<p>\nZ tohoto \u010dl\u00e1nku se dozv\u00edte o na\u0161\u00ed zku\u0161enosti s t\u011bmito n\u00e1stroji: co jsme ud\u011blali, co jsme se nau\u010dili a co budeme d\u011blat d\u00e1l. Nyn\u00ed se zam\u011b\u0159\u00edme na d\u00e1vkovou a interaktivn\u00ed anal\u00fdzu. Anal\u00fdzu v re\u00e1ln\u00e9m \u010dase probereme v dal\u0161\u00edm \u010dl\u00e1nku.<\/p>\n<h2>Historie datov\u00fdch \u00falo\u017ei\u0161\u0165 na Twitteru<\/h2>\n<p>\nNe\u017e se pono\u0159\u00edme do BigQuery, stoj\u00ed za to stru\u010dn\u011b shrnout historii datov\u00fdch sklad\u016f na Twitteru. V roce 2011 se anal\u00fdza dat na Twitteru prov\u00e1d\u011bla pomoc\u00ed Vertica a Hadoop. Pro vytv\u00e1\u0159en\u00ed MapReduce pracovn\u00edch \u00faloh Hadoop jsme pou\u017e\u00edvali Pig. V roce 2012 jsme nahradili Pig za Scalding, kter\u00fd m\u011bl Scala API s v\u00fdhodami jako mo\u017enost vytv\u00e1\u0159et slo\u017eit\u00e9 potrub\u00ed a jednoduchost testov\u00e1n\u00ed. Nicm\u00e9n\u011b, pro mnoh\u00e9 datov\u00e9 analytiky a produktov\u00e9 mana\u017eery, pro n\u011b\u017e bylo pohodln\u011bj\u0161\u00ed pracovat s SQL, to byla dost strm\u00e1 k\u0159ivka u\u010den\u00ed. P\u0159ibli\u017en\u011b v roce 2016 jsme za\u010dali pou\u017e\u00edvat Presto jako SQL rozhran\u00ed pro data Hadoop. Spark nab\u00edzel Python rozhran\u00ed, co\u017e ho \u010dinilo vhodnou volbou pro ad hoc datov\u00e9 v\u00fdzkumy a strojov\u00e9 u\u010den\u00ed.<\/p>\n<p>Od roku 2018 jsme pou\u017e\u00edvali n\u00e1sleduj\u00edc\u00ed n\u00e1stroje pro anal\u00fdzu a vizualizaci dat:<\/p>\n<ul>\n<li>Scalding pro v\u00fdrobn\u00ed potrub\u00ed<\/li>\n<li>Scalding a Spark pro ad hoc anal\u00fdzu dat a strojov\u00e9 u\u010den\u00ed<\/li>\n<li>Vertica a Presto pro ad hoc a interaktivn\u00ed SQL anal\u00fdzu <\/li>\n<li>Druid pro malou interaktivn\u00ed, pr\u016fzkumnou a p\u0159\u00edstup s n\u00edzkou latenc\u00ed k metrik\u00e1m \u010dasov\u00fdch \u0159ad<\/li>\n<li>Tableau, Zeppelin a Pivot pro vizualizaci dat<\/li>\n<\/ul>\n<p>\nZjistili jsme, \u017ee a\u010dkoli tyto n\u00e1stroje nab\u00edzej\u00ed velmi siln\u00e9 mo\u017enosti, m\u011bli jsme pot\u00ed\u017ee s implementac\u00ed dostupnosti t\u011bchto mo\u017enost\u00ed pro \u0161ir\u0161\u00ed publikum na Twitteru. P\u0159i roz\u0161i\u0159ov\u00e1n\u00ed na\u0161\u00ed platformy prost\u0159ednictv\u00edm Google Cloud se soust\u0159ed\u00edme na zjednodu\u0161en\u00ed na\u0161ich analytick\u00fdch n\u00e1stroj\u016f pro cel\u00fd Twitter.<\/p>\n<h2>Datov\u00fd sklad BigQuery od Googlu <\/h2>\n<p>\nN\u011bkolik t\u00fdm\u016f na Twitteru ji\u017e za\u010dlenilo BigQuery do n\u011bkter\u00fdch sv\u00fdch v\u00fdrobn\u00edch potrub\u00ed. Vyu\u017e\u00edvaj\u00edc jejich zku\u0161enosti, jsme za\u010dali hodnotit mo\u017enosti BigQuery pro v\u0161echny sc\u00e9n\u00e1\u0159e pou\u017eit\u00ed na Twitteru. Na\u0161\u00edm c\u00edlem bylo nab\u00eddnout BigQuery cel\u00e9 spole\u010dnosti, stejn\u011b jako standardizovat a podporovat ji v r\u00e1mci sady n\u00e1stroj\u016f Data Platform. To bylo n\u00e1ro\u010dn\u00e9 z mnoha d\u016fvod\u016f. Museli jsme vyvinout infrastrukturu pro spolehliv\u00e9 p\u0159ij\u00edm\u00e1n\u00ed velk\u00fdch objem\u016f dat, podporu spr\u00e1vy dat v m\u011b\u0159\u00edtku cel\u00e9 firmy, zaji\u0161t\u011bn\u00ed spr\u00e1vn\u00e9 kontroly p\u0159\u00edstupu a zaji\u0161t\u011bn\u00ed d\u016fv\u011brnosti klient\u016f. Tak\u00e9 jsme museli vytvo\u0159it syst\u00e9my pro rozd\u011blov\u00e1n\u00ed zdroj\u016f, monitorov\u00e1n\u00ed a vracen\u00ed plateb, aby t\u00fdmy mohly efektivn\u011b vyu\u017e\u00edvat BigQuery.<\/p>\n<p>V listopadu 2018 jsme vydali alpha verzi BigQuery a Data Studio pro celou spole\u010dnost. Zam\u011bstnanc\u016fm Twitteru jsme nab\u00eddli n\u011bkter\u00e9 z na\u0161ich nej\u010dast\u011bji pou\u017e\u00edvan\u00fdch tabulek s vy\u010di\u0161t\u011bn\u00fdmi osobn\u00edmi \u00fadaji. BigQuery pou\u017e\u00edvalo v\u00edce ne\u017e 250 u\u017eivatel\u016f z r\u016fzn\u00fdch t\u00fdm\u016f, v\u010detn\u011b in\u017een\u00fdrsk\u00fdch, finan\u010dn\u00edch a marketingov\u00fdch. Ned\u00e1vno vykon\u00e1vali p\u0159ibli\u017en\u011b 8 tis\u00edc dotaz\u016f, zpracov\u00e1vaj\u00edc kolem 100 PB m\u011bs\u00ed\u010dn\u011b, mimo napl\u00e1novan\u00e9 dotazy. Po obdr\u017een\u00ed velmi pozitivn\u00edch zp\u011btn\u00fdch vazeb jsme se rozhodli pokra\u010dovat a nab\u00eddnout BigQuery jako hlavn\u00ed zdroj pro pr\u00e1ci s daty na Twitteru.<\/p>\n<p>Zde je sch\u00e9ma vysoce\u00farov\u0148ov\u00e9 architektury na\u0161eho datov\u00e9ho skladu Google BigQuery. <\/p>\n<p><img decoding=\"async\" alt=\"Jak BigQuery od Google demokratizoval anal\u00fdzu dat. \u010c\u00e1st 1\" src=\"\/wp-content\/uploads\/2020\/08\/6a3e0b43752ebbac641c43519c8c2fba.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\nKop\u00edrujeme data z lok\u00e1ln\u00edch klastr\u016f Hadoop do Google Cloud Storage (GCS) pomoc\u00ed intern\u00edho n\u00e1stroje Cloud Replicator. Pot\u00e9 pou\u017e\u00edv\u00e1me Apache Airflow k vytv\u00e1\u0159en\u00ed potrub\u00ed, kter\u00e1 pou\u017e\u00edvaj\u00ed \u201e<noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery\/docs\/reference\/bq-cli-reference#bq_load\">bq_load<\/a><\/noindex>\u201c pro na\u010d\u00edt\u00e1n\u00ed dat z GCS do BigQuery. Pou\u017e\u00edv\u00e1me Presto pro dotazov\u00e1n\u00ed na datov\u00e9 sady Parquet nebo Thrift-LZO v GCS. BQ Blaster je intern\u00ed n\u00e1stroj Scalding pro na\u010d\u00edt\u00e1n\u00ed datov\u00fdch sad HDFS Vertica a Thrift-LZO do BigQuery.<\/p>\n<p>V n\u00e1sleduj\u00edc\u00edch sekc\u00edch budeme diskutovat o na\u0161em p\u0159\u00edstupu a znalostech v oblasti snadnosti pou\u017eit\u00ed, v\u00fdkonu, spr\u00e1vy dat, provozn\u00ed efektivity a n\u00e1klad\u016f.<\/p>\n<h4>Snadnost pou\u017eit\u00ed<\/h4>\n<p>\nZjistili jsme, \u017ee u\u017eivatel\u00e9 se snadno za\u010dali s BigQuery, proto\u017ee nevy\u017eadoval instalaci softwaru a u\u017eivatel\u00e9 k n\u011bmu mohli p\u0159istupovat prost\u0159ednictv\u00edm intuitivn\u00edho webov\u00e9ho rozhran\u00ed. Nicm\u00e9n\u011b u\u017eivatel\u00e9 se museli sezn\u00e1mit s n\u011bkter\u00fdmi funkcemi GCP a jeho koncepty, v\u010detn\u011b takov\u00fdch zdroj\u016f, jako jsou projekty, datov\u00e9 sady a tabulky. Vyvinuli jsme vzd\u011bl\u00e1vac\u00ed materi\u00e1ly a tutori\u00e1ly, abychom pomohli u\u017eivatel\u016fm za\u010d\u00edt. Po z\u00edsk\u00e1n\u00ed z\u00e1kladn\u00edho porozum\u011bn\u00ed se u\u017eivatel\u016fm stalo snadn\u00e9 navigovat datov\u00fdmi sadami, prohl\u00ed\u017eet sch\u00e9ma a data tabulek, prov\u00e1d\u011bt jednoduch\u00e9 dotazy a vizualizovat v\u00fdsledky v Data Studio.<\/p>\n<p>Na\u0161\u00edm c\u00edlem ohledn\u011b vstupu dat do BigQuery bylo zajistit hladk\u00e9 na\u010d\u00edt\u00e1n\u00ed datov\u00fdch sad HDFS nebo GCS jedn\u00edm kliknut\u00edm. Zva\u017eovali jsme <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/composer\/\">Cloud Composer<\/a><\/noindex> (\u0159\u00edzen\u00fd Airflow), ale nemohli jsme jej pou\u017e\u00edt kv\u016fli na\u0161emu modelu bezpe\u010dnosti \u201eDomain Restricted Sharing\u201c (v\u00edce o tom v \u010d\u00e1sti \u201eSpr\u00e1va dat\u201c n\u00ed\u017ee). Experimentovali jsme s pou\u017eit\u00edm Google Data Transfer Service (DTS) pro organizaci zat\u011b\u017eovac\u00edch \u00faloh BigQuery. I kdy\u017e se DTS rychle nastavilo, nebylo flexibiln\u00ed pro vytv\u00e1\u0159en\u00ed potrub\u00ed se z\u00e1vislostmi. Pro na\u0161i alfa verzi jsme vytvo\u0159ili vlastn\u00ed prost\u0159ed\u00ed Apache Airflow v GCE a p\u0159ipravujeme je na provoz a schopnost podporovat v\u00edce zdroj\u016f dat, jako je Vertica.<\/p>\n<p>Pro transformaci dat v BigQuery u\u017eivatel\u00e9 vytv\u00e1\u0159ej\u00ed jednoduch\u00e9 datov\u00e9 potrub\u00ed SQL pomoc\u00ed napl\u00e1novan\u00fdch dotaz\u016f. Pro slo\u017eit\u00e1 v\u00edcestup\u0148ov\u00e1 potrub\u00ed se z\u00e1vislostmi pl\u00e1nujeme pou\u017e\u00edt bu\u010f na\u0161i vlastn\u00ed infrastrukturu Airflow, nebo Cloud Composer spolu s <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/dataflow\/\">Cloud Dataflow<\/a><\/noindex>.<\/p>\n<h4>V\u00fdkon<\/h4>\n<p>\nBigQuery je navr\u017een pro obecn\u00e9 SQL dotazy, kter\u00e9 zpracov\u00e1vaj\u00ed velk\u00e9 objemy dat. Nen\u00ed ur\u010den pro dotazy s n\u00edzkou latenc\u00ed a vysokou propustnost\u00ed, kter\u00e9 jsou pot\u0159ebn\u00e9 pro transak\u010dn\u00ed datab\u00e1ze, nebo pro anal\u00fdzu \u010dasov\u00fdch \u0159ad s n\u00edzkou latenc\u00ed, realizovanou <noindex><a rel=\"nofollow\" href=\"https:\/\/druid.apache.org\/\">Apache Druid<\/a><\/noindex>. Pro interaktivn\u00ed analytick\u00e9 dotazy na\u0161i u\u017eivatel\u00e9 o\u010dek\u00e1vaj\u00ed dobu odezvy pod jednu minutu. Museli jsme navrhnout pou\u017eit\u00ed BigQuery tak, aby vyhovovalo t\u011bmto o\u010dek\u00e1v\u00e1n\u00edm. Abychom zajistili p\u0159edv\u00eddateln\u00fd v\u00fdkon pro na\u0161e u\u017eivatele, vyu\u017eili jsme funkci BigQuery, dostupnou pro klienty na pevn\u00e9 platb\u011b, kter\u00e1 umo\u017e\u0148uje vlastn\u00edk\u016fm projekt\u016f rezervovat minim\u00e1ln\u00ed sloty pro sv\u00e9 dotazy. <noindex><a rel=\"nofollow\" href=\"https:\/\/cloud.google.com\/bigquery\/docs\/slots\">Slot<\/a><\/noindex> BigQuery je jednotka v\u00fdpo\u010detn\u00ed s\u00edly pot\u0159ebn\u00e1 pro prov\u00e1d\u011bn\u00ed SQL dotaz\u016f. <\/p>\n<p>Analyzovali jsme v\u00edce ne\u017e 800 dotaz\u016f, kter\u00e9 zpracov\u00e1vaj\u00ed p\u0159ibli\u017en\u011b 1 TB dat ka\u017ed\u00fd, a zjistili jsme, \u017ee pr\u016fm\u011brn\u00fd \u010das proveden\u00ed byl 30 sekund. Tak\u00e9 jsme se dozv\u011bd\u011bli, \u017ee v\u00fdkon siln\u011b z\u00e1vis\u00ed na pou\u017eit\u00ed na\u0161eho slotu v r\u016fzn\u00fdch projektech a \u00fakolech. Museli jsme jasn\u011b odd\u011blit na\u0161e v\u00fdrobn\u00ed a ad hoc rezervy slot\u016f, abychom udr\u017eeli v\u00fdkon pro v\u00fdrobn\u00ed sc\u00e9n\u00e1\u0159e pou\u017eit\u00ed a interaktivn\u00ed anal\u00fdzu. To velmi ovlivnilo n\u00e1\u0161 design rezervace slot\u016f a hierarchie projekt\u016f.<\/p>\n<p><i>O spr\u00e1v\u011b dat, funk\u010dnosti a n\u00e1kladech syst\u00e9m\u016f si promluv\u00edme ji\u017e brzy ve druh\u00e9 \u010d\u00e1sti p\u0159ekladu, ale nyn\u00ed zveme v\u0161echny z\u00e1jemce na <noindex><a rel=\"nofollow\" href=\"https:\/\/otus.pw\/wDZQ\/\">bezplatn\u00fd \u017eiv\u00fd webin\u00e1\u0159<\/a><\/noindex>, v r\u00e1mci kter\u00e9ho se budete moci podrobn\u011b dozv\u011bd\u011bt o kurzu a tak\u00e9 se pt\u00e1t na\u0161eho experta \u2014 Egora Mateshuka (Senior Data Engineer, MaximaTelecom). <\/i><\/p>\n<p><\/p>\n<h2>\u010c\u00edst v\u00edc:<\/h2>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/501380\/\">Data Build Tool neboli co maj\u00ed spole\u010dn\u00e9ho Datov\u00e9 Sklady a Smoothie<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/502324\/\">Pohled na Delta Lake: nucen\u00e9 aplikace a evoluce sch\u00e9matu<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/503132\/\">Vysokorychlostn\u00ed Apache Parquet na Pythonu s Apache Arrow<\/a><\/noindex><\/li>\n<\/ul>\n<p>Zdroj: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/otus\/blog\/513780\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u044f\u043c\u043e \u0441\u0435\u0439\u0447\u0430\u0441 \u0432 OTUS \u043e\u0442\u043a\u0440\u044b\u0442 \u043d\u0430\u0431\u043e\u0440 \u043d\u0430 \u043d\u043e\u0432\u044b\u0439 \u043f\u043e\u0442\u043e\u043a \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb. \u0412 \u043f\u0440\u0435\u0434\u0434\u0432\u0435\u0440\u0438\u0438 \u0441\u0442\u0430\u0440\u0442\u0430 \u043a\u0443\u0440\u0441\u0430 \u043c\u044b \u0442\u0440\u0430\u0434\u0438\u0446\u0438\u043e\u043d\u043d\u043e \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u0438\u043b\u0438 \u0434\u043b\u044f \u0432\u0430\u0441 \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u0438\u043d\u0442\u0435\u0440\u0435\u0441\u043d\u043e\u0433\u043e \u043c\u0430\u0442\u0435\u0440\u0438\u0430\u043b\u0430. \u041a\u0430\u0436\u0434\u044b\u0439 \u0434\u0435\u043d\u044c \u0431\u043e\u043b\u0435\u0435 \u0441\u0442\u0430 \u043c\u0438\u043b\u043b\u0438\u043e\u043d\u043e\u0432 \u0447\u0435\u043b\u043e\u0432\u0435\u043a \u043f\u043e\u0441\u0435\u0449\u0430\u044e\u0442 Twitter, \u0447\u0442\u043e\u0431\u044b \u0443\u0437\u043d\u0430\u0442\u044c, \u0447\u0442\u043e \u043f\u0440\u043e\u0438\u0441\u0445\u043e\u0434\u0438\u0442 \u0432 \u043c\u0438\u0440\u0435, \u0438 \u043e\u0431\u0441\u0443\u0434\u0438\u0442\u044c \u044d\u0442\u043e. \u041a\u0430\u0436\u0434\u044b\u0439 \u0442\u0432\u0438\u0442 \u0438 \u043b\u044e\u0431\u043e\u0435 \u0434\u0440\u0443\u0433\u043e\u0435 \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u044f \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044f \u0433\u0435\u043d\u0435\u0440\u0438\u0440\u0443\u044e\u0442 \u0441\u043e\u0431\u044b\u0442\u0438\u0435, \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0435 \u0434\u043b\u044f \u0432\u043d\u0443\u0442\u0440\u0435\u043d\u043d\u0435\u0433\u043e [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":90784,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-90783","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.3 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u044f\u043c\u043e \u0441\u0435\u0439\u0447\u0430\u0441 \u0432 OTUS \u043e\u0442\u043a\u0440\u044b\u0442 \u043d\u0430\u0431\u043e\u0440 \u043d\u0430 \u043d\u043e\u0432\u044b\u0439 \u043f\u043e\u0442\u043e\u043a \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.3\" \/>\n\t\t<meta property=\"og:locale\" content=\"cs_CZ\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041a\u0430\u043a BigQuery \u043e\u0442 Google \u0434\u0435\u043c\u043e\u043a\u0440\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043b \u0430\u043d\u0430\u043b\u0438\u0437 \u0434\u0430\u043d\u043d\u044b\u0445. \u0427\u0430\u0441\u0442\u044c 1 | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u044f\u043c\u043e \u0441\u0435\u0439\u0447\u0430\u0441 \u0432 OTUS \u043e\u0442\u043a\u0440\u044b\u0442 \u043d\u0430\u0431\u043e\u0440 \u043d\u0430 \u043d\u043e\u0432\u044b\u0439 \u043f\u043e\u0442\u043e\u043a \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-08-05T23:42:19+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-08-05T23:42:19+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Jak Google BigQuery demokratizoval anal\u00fdzu dat. \u010c\u00e1st 1 | ProHoster","description":"Ahoj, Habr! Pr\u00e1v\u011b te\u010f je v OTUS otev\u0159en z\u00e1pis na nov\u00fd b\u011bh kurzu \"Data Engineer\".","canonical_url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"cs_CZ","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041a\u0430\u043a BigQuery \u043e\u0442 Google \u0434\u0435\u043c\u043e\u043a\u0440\u0430\u0442\u0438\u0437\u0438\u0440\u043e\u0432\u0430\u043b \u0430\u043d\u0430\u043b\u0438\u0437 \u0434\u0430\u043d\u043d\u044b\u0445. \u0427\u0430\u0441\u0442\u044c 1 | ProHoster","og:description":"\u041f\u0440\u0438\u0432\u0435\u0442, \u0425\u0430\u0431\u0440! \u041f\u0440\u044f\u043c\u043e \u0441\u0435\u0439\u0447\u0430\u0441 \u0432 OTUS \u043e\u0442\u043a\u0440\u044b\u0442 \u043d\u0430\u0431\u043e\u0440 \u043d\u0430 \u043d\u043e\u0432\u044b\u0439 \u043f\u043e\u0442\u043e\u043a \u043a\u0443\u0440\u0441\u0430 \u00abData Engineer\u00bb.","og:url":"https:\/\/prohoster.info\/cs\/blog\/administrirovanie\/kak-bigquery-ot-google-demokratiziroval-analiz-dannyh-chast-1","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-08-05T23:42:19+00:00","article:modified_time":"2020-08-05T23:42:19+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"90783","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 12:42:25","updated":"2022-10-03 03:00:18","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/90783","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/comments?post=90783"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/posts\/90783\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media\/90784"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/media?parent=90783"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/categories?post=90783"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/cs\/wp-json\/wp\/v2\/tags?post=90783"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}