Ngiphakamisa ukuthi ufunde okulotshiweyo kombiko ka-2017 ka-Igor Stryhar "ClickHouse - ukuhlaziya okubonakalayo okusheshayo nokucacile kwedatha ku-Tabix."
I-Web interface ye-ClickHouse kuphrojekthi ye-Tabix.
Izici ezibalulekile:
- Isebenza nge-ClickHouse ngqo kusuka kusiphequluli, ngaphandle kwesidingo sokufaka isofthiwe eyengeziwe;
- Umhleli wombuzo onokugqamisa kwe-syntax;
- Ukuqedela ngokuzenzakalela imiyalo;
- Amathuluzi okuhlaziywa kwesithombe sokwenziwa kombuzo;
- Izikimu zombala ongakhetha kuzo.
Ngingumqondisi wezobuchwepheshe we-SMI2. Singumhlanganisi wezindaba wokushintshisana ngezindaba. Sigcina idatha eningi esiyithola kozakwethu futhi siyibhalisa ku-ClickHouse - izicelo ezingaba ngu-30 ngomzuzwana.
Lena idatha efana nalena:
- Ukuchofoza ezindabeni.
- Izindaba ziboniswa kusihlanganisi.
- Izibhengezo ziboniswa kunethiwekhi yethu.
- Futhi sibhalisa imicimbi kwikhawunta yethu, efana ne-Yandex.Metrica. Lokhu ukuhlaziya kwethu okuncane.
Besiphila impilo ematasatasa ngaphambi kweClickHouse. Sihlupheke kakhulu, sizama ukugcina le datha ndawana thize futhi siyihlaziye ngandlela thize.
Impilo ngaphambi kweClickHouse - infiniDB
Into yokuqala esasinayo kwakuyi-infiniDB. Wahlala nathi iminyaka emi-4. Siyethule ngobunzima.
- Ayisekeli ukuhlanganisa noma ukwahlukanisa. Azikho izinto ezihlakaniphile ezinjalo eziphume ebhokisini ngokuzenzakalelayo.
- Unenkinga yokulayisha idatha. Kuphela insiza ethile yekhonsoli engalayisha kuphela amafayela e-CSV futhi ngendlela ethile engacacile.
- Isizindalwazi sinomucu owodwa. Ungabhala noma ufunde. Kodwa kwenze kwaba nokwenzeka ukucubungula inani elikhulu ledatha.
- Futhi wayenedondolo elithakazelisayo. Njalo ebusuku iseva kwakudingeka iqalwe kabusha, ngaphandle kwalokho ngeke isebenze.
Usisebenzele kwaze kwaba sekupheleni kuka-2016, lapho sashintshela ngokuphelele ku-ClickHouse.
Impilo ngaphambi kweClickHouse - uCassandra
Njengoba i-infiniDB yayinentambo eyodwa, sinqume ukuthi sidinga uhlobo oluthile lwesizindalwazi esinezintambo eziningi esingabhala kuso imicu eminingi ngesikhathi esisodwa.
Sazama izinto eziningi ezithakazelisayo. Sabe sesinquma ukuzama uCassandra. Konke kwakukuhle ngoCassandra. Izicelo eziyi-10 ngomzuzwana ngebhidi ngayinye. Izicelo ezingu-000 ndawana thize ukuze zifundwe.
Kodwa naye wayenezithakazelo zakhe. Kanye ngenyanga noma kanye njalo ngemva kwezinyanga ezimbili wabhekana nokuchithwa kwedatha. Futhi kwadingeka ngivuke ngigijime ngiyolungisa uCassandra. Amaseva aqalwe kabusha ngayinye ngayinye. Futhi konke kwaba bushelelezi futhi enhle.
Impilo ngaphambi kweClickHouse - Druid
Sabe sesibona ukuthi sidinga ukubhala idatha eyengeziwe. Ngo-2016 saqala ukubuka iDruid.
I-Druid iyisofthiwe yomthombo ovulekile ebhalwe nge-Java. Okuqondile kakhulu. Futhi ibifanele i-clickstream, lapho sidinga ukugcina uhlobo oluthile lokusakazwa kwemicimbi bese senza ukuhlanganiswa kuzo noma senze imibiko yokuhlaziya.
UDruid wayenenguqulo engu-0.9.X.
I-database ngokwayo inzima kakhulu ukuyifaka. Lokhu kuyinkimbinkimbi yengqalasizinda. Ukuze kusetshenziswe, kwakudingeka kufakwe okuningi, insimbi eningi. Futhi ingxenye ngayinye yehadiwe yayinesibopho sendima yayo ehlukile.
Ukulayisha idatha kuyo, kwakudingeka ukusebenzisa uhlobo oluthile lwe-shamanism. Kukhona iphrojekthi ye-OpenSource - Tranquility, ebilahlekelwa idatha yethu ngesikhathi sokusakaza. Ngesikhathi silayisha idatha kuyo, ilahlekile.
Kodwa ngandlela-thile saqala ukukusebenzisa. Thina, njengama-hedgehogs asebenzisa izidakamizwa kodwa aqhubeka edla umdolofiya, saqala ukuyethula. Kusithathe isikhathi esingangenyanga ukuyilungiselela yonke ingqalasizinda. Okusho ukuthi, amaseva oda, lungiselela izindima, futhi ukhiphe ngokuzenzakalelayo ngokugcwele. Okusho ukuthi, uma kwenzeka ukwehluleka kweqoqo, iqoqo lesibili lizosetshenziswa ngokuzenzakalelayo.
Kodwa kwabe sekwenzeka isimangaliso. Bengiseholidini futhi ozakwethu bangithumelele isixhumanisi , okusho ukuthi i-Yandex inqume ukuvula i-ClickHouse. Ngithi asizame.
Futhi ngokoqobo ezinsukwini ezi-2 sikhiphe iqoqo lokuhlola le-ClickHouse. Siqale ukulayisha idatha kuyo. Uma kuqhathaniswa ne-infiniDB, lokhu kuyisisekelo; uma kuqhathaniswa ne-Druid, lokhu kuyisisekelo. Uma kuqhathaniswa neCassandra, nayo iyisisekelo. Ngoba uma ulayisha idatha kusuka ku-PHP ku-Cassandra, lokhu akuyona into eyisisekelo.
Sitholeni? Ukusebenza ngesivinini. Ukusebenza ekugcinweni kwedatha. Okusho ukuthi, isikhala sediski esincane kakhulu sisetshenziswa. I-ClickHouse iyashesha, iyashesha kakhulu uma iqhathaniswa neminye imikhiqizo.
Ngesikhathi sokwethulwa, lapho i-Yandex ishicilela i-ClickHouse ku-OpenSource, kwakukhona iklayenti le-console kuphela. Thina enkampanini yethu i-SMI2 sinqume ukuzama ukwenza iklayenti lendabuko kuwebhu, ukuze sikwazi ukuvula ikhasi kusuka kusiphequluli, sibhale isicelo futhi sithole umphumela, ngoba saqala ukubhala izicelo eziningi. Ukubhala kukhonsoli kunzima. Futhi senze inguqulo yethu yokuqala.
Futhi endaweni ethile eduze nobusika bangonyaka odlule, amathuluzi enkampani yangaphandle okusebenza ne-ClickHouse aqala ukuvela. Lawa ngamathuluzi afana nalawa:
- .
- .
- . (Manje )
- .
- .
Ngizobheka amanye ala mathuluzi, okungukuthi, lawo engisebenze ngawo.
Ithuluzi elihle, kodwa le-Druid. Ngenkathi i-Druid isetshenziswa, bengihlola i-SuperSet. Ngangimthanda. Ku-Druid kuyashesha kakhulu.
Ayilungele i-ClickHouse. Okusho ukuthi, iyalingana, iyaqala, kodwa ilungele ukucubungula kuphela imibuzo eyisisekelo efana nokuthi: KHETHA umcimbi, GROUP BY umcimbi. Ayisekeli i-syntax ye-ClickHouse eyinkimbinkimbi.
Ithuluzi elilandelayo i-Apache Zeppelin. Lokhu kuyinto enhle futhi ethakazelisayo. Iyasebenza. Isekela ama-notebook, amadeshibhodi, futhi isekela okuguquguqukayo. Ngiyazi othile emphakathini we-ClickHouse uyayisebenzisa.
Kodwa akukho ukusekelwa kwe-syntax ye-ClickHouse, okusho ukuthi kuzodingeka ubhale imibuzo ku-console noma kwenye indawo. Okulandelayo, hlola ukuthi konke kuyasebenza. Kuyaphazamisa nje. Kodwa inokusekelwa okuhle kwedeshibhodi.
Ithuluzi elilandelayo yi-Redash.IO. I-Redash isingathwa ku-inthanethi. Okusho ukuthi, ngokungafani namathuluzi wangaphambilini, ayidingi ukufakwa. Futhi lena ideshibhodi enekhono lokuhlanganisa idatha evela ku-DataSource ehlukene. Okusho ukuthi, ungalanda ku-ClickHouse, ku-MySQL, ku-PostgreSQL nakweminye imininingwane.
Enyangeni edlule (March 2017), ukwesekwa kwavela eGrafana. Uma wakha imibiko e-Grafana, ngokwesibonelo, ngesimo sezingxenyekazi zekhompuyutha yakho noma kwamanye amamethrikhi, manje usungakwazi ukwakha igrafu efanayo noma uhlobo oluthile lwephaneli kusuka kudatha esuka ku-ClickHouse ngokuqondile. Lokhu kulula kakhulu, futhi sikusebenzisa ngokwethu. Lokhu kukuvumela ukuthi uthole okudidayo. Okusho ukuthi, uma kwenzeka okuthile futhi enye i-hardware iwe noma iba nzima, ungabheka isizathu uma le datha ikwazile ukungena ku-ClickHouse.
Ngikuthole kunzima kakhulu ukubhala kulawa mathuluzi noma kukhonsoli. Futhi nginqume ukuthuthukisa isikhombimsebenzisi sethu sokuqala. Futhi ngithole umbono ku-EventSQL, SeperSet, Zeppelin.
Ubufunani? Bengifuna ukuthola ihluzo, umhleli othuthukisiwe, nokusebenzisa ukusekelwa kwezichazamazwi zokusikisela. Ngoba i-ClickHouse inesici esihle - izichazamazwi. Kodwa kunzima ukusebenza ngezichazamazwi, ngoba udinga ukukhumbula ifomethi yamanani agciniwe, okungukuthi inombolo noma iyunithi yezinhlamvu, njll. Futhi njengoba sivame ukusebenzisa izichazamazwi ngokuhlukahluka kwazo okuhlukene, bekunzima kakhulu ukubhala imibuzo.
Sekudlule izinyanga ezi-3 kwakhululwa inguqulo yethu yokuqala. Ngenze izibophezelo ezingaba ngu-330 egatsheni elizimele futhi kwavela ukuthi yi-Tabix.
Ngokungafani nenguqulo yangaphambilini, eyayibizwa nge-ClickHouse-Frontend, nginqume ukuyiqamba kabusha ngegama elilula. Futhi kwavela Tabix.
Kwavelani?
Udweba amagrafu. Isekela i-syntax ye-ClickHouse SQL. Inikeza iseluleko ngemisebenzi futhi ingenza izinto eziningi ezithakazelisayo.
Yilokhu okubukeka kuhlelo olujwayelekile lwe-Tabix. Kwesokunxele kunesihlahla. Phakathi nendawo kukhona umhleli wombuzo. Futhi ngezansi umphumela walesi sicelo.
Okulandelayo ngizokukhombisa ukuthi umhleli wemibuzo usebenza kanjani.
Lapha ukuqedela ngokuzenzakalela kusebenza ngokuzenzakalela kuthebula futhi kuyala, ngokufanelekile, ukuqedela ngokuzenzakalela kwezinkambu. Futhi namacebo mayelana nemisebenzi. Uma ucindezela u-ctrl enter, isicelo sizokwenziwa noma sehluleke ngephutha. Isicelo esilula sithunyelwa ku-Tabix futhi umphumela utholakele, okungukuthi ungasebenza ngokushesha nge-ClickHouse.
Izichazamazwi, njengoba sengishilo, ziyinto ethokozisayo kakhulu esisebenza ngayo kakhulu. Futhi okwasivumela ukuba senze izinto eziningi. Ake sithi sigcina wonke amadolobha ngezichazamazwi. Sigcina isihlonzi sedolobha kanye negama ledolobha, ububanzi balo nobude. Futhi kusizindalwazi sigcina kuphela isihlonzi sedolobha. Ngokuvumelana nalokho, sicindezela idatha kakhulu.
Lokhu kubonakala kuyinto elula, kodwa kusiza ku-ClickHouse ngendlela ethakazelisa kakhulu. Ngenxa yokuthi i-ClickHouse isekela kuphela ukujoyina okufakwe isidleke, umbuzo ukhula ubheke phansi futhi ububanzi ngokuqinile. Futhi uma ubakaki uvuleka futhi kungena isisho eside, into elula njengokugoqa umbuzo ikwenza kube lula ukusebenza nombuzo ngokwawo. Ngoba uma umbuzo uyimigqa engu-200-300 ubude futhi mkhulu kakhulu ngobubanzi, kuyasiza kakhulu ukugoqa umbuzo bese uthola indawo ethile noma wenze kube okwasendaweni ngandlela thile.
Isihlahla sezinto, imibuzo eminingi kanye namathebhu (Ividiyo 13:46 )
Okulandelayo ngizokukhombisa ngesihlahla namathebhu. Kwesokunxele kunesihlahla; phezulu ungakha amathebhu amaningana. Amathebhu afana nendawo yokusebenza. Ungakha amathebhu amaningana futhi uqambe ngalinye ngokuhlukile. Kufana nesistimu encane yokwakha umbiko.
Amathebhu alondolozwa ngokuzenzakalelayo. Uma uqala kabusha isiphequluli sakho noma uvala noma uvula i-Tabix, konke lokhu kuzolondolozwa.
I-Hotkey - elula (Ividiyo 14:39 )
Kukhona ama-hotkeys futhi maningi kakhulu. Ngikhiphe ezinye zazo lapha njengesibonelo. Lokhu ukushintsha amathebhu, ukwenza isicelo noma ukwenza izicelo ezimbalwa.
Ngizokukhombisa ukuthi usebenza kanjani ngomphumela. Sithumela isicelo. Lapha ngidweba isono, cos kanye ne-tg. Ungakwazi ukugqamisa umphumela, okungukuthi ukudweba imephu evamile yekholomu. Ungagqamisa amanani amahle noma amabi. Noma vele ufake umbala entweni ethile yetafula. Lokhu kulungele uma ithebula likhulu futhi udinga ukuthola okudidayo ngamehlo akho. Ngenkathi ngifuna okudidayo, ngagqamisa imigqa ethile, ezinye izakhi zibe luhlaza noma ngokubomvu.
Kukhona izinto eziningi ezithakazelisayo lapho. Isibonelo, ungakopisha kanjani ku-Redmine Markdown. Uma udinga ukukopisha umphumela endaweni ethile, lokhu kulula kakhulu. Ungamane ukhethe indawo, uthi “Kopishela ku-Redmine” futhi izokopisha ku-Redmine Markdown noma udale umbuzo othi Lapho.
Okulandelayo ukulungiselelwa kombuzo. Ngake ngakhohlwa ukucacisa inkambu "yosuku". Futhi isicelo sami ku-ClickHouse asicutshungulwanga kakhulu, ngokushesha okukhulu, kodwa ngokushesha, okungukuthi ngaphansi kwesekhondi. Lapho ngibona ukuthi mingaki imigqa adlula kuyo, ngafikelwa ukwesaba. Asibhali imigqa eminingi kangaka kuleli thebula ngosuku. Ngaqala ukuhlaziya isicelo ngabona ukuthi ngiphuthelwe usuku endaweni eyodwa. Okusho ukuthi, ngikhohliwe ukukhombisa ukuthi angidingi idatha yetafula lonke, kodwa isikhathi esithile.
I-Tabix inethebhu "Izibalo", egcina wonke umlando wezicelo ezithunyelwe, okungukuthi lapho ungabona ukuthi zingaki imigqa efundwe yilesi sicelo nokuthi kuthathe isikhathi esingakanani ukwenza. Lokhu kuvumela ukulungiselelwa.
Ungakha ithebula le-pivot phezu komphumela wombuzo. Uthumele isicelo ku-ClickHouse futhi wathola idatha. Bese ungakwazi ukuhambisa le datha ngegundane lakho futhi wakhe uhlobo oluthile lwethebula le-pivot.
Into elandelayo ethokozisayo ukwakha itulo. Ake sithi sinesicelo esilandelayo: ngesono, cos kusuka ku-0 kuya ku-299. Futhi ukuyidweba, udinga ukukhetha ithebhu ethi "Dweba" futhi uzothola igrafu ngesono sakho ne-cos.
Ungakwazi ukuhlukanisa lokhu ngezimbazo ezihlukene, okungukuthi ungadweba amagrafu amabili ngapha nangapha ngesikhathi esisodwa. Bhala umyalo owodwa nowesibili.
Ungakwazi ukudweba histograms.
Ungakwazi ukuhlukanisa lokhu kube i-matrix yamagrafu.
Ungakha imephu yokushisa.
Ungakha ikhalenda elishisayo. Ngendlela, lokhu kuyinto elula kakhulu uma udinga ukuhlaziya okungaqondakali ngaphezu konyaka, okungukuthi, thola ama-spikes noma amaconsi. Lokhu kubukwa kwedatha kungisizile kulokhu.
Okulandelayo yi-Treemap.
I-Sankeys iyishadi elithakazelisayo. Kuphakathi kokuthi u-Streamgrahps noma uMfula. Kodwa ngiwubiza ngoMfula. Iphinde ikuvumela ukuthi ubheke noma yikuphi okudidayo. Kuntofontofo kakhulu. Ngincoma ukuyisebenzisela ukusesha.
Into elandelayo ethokozisayo ukudweba imephu eguqukayo. Uma ugcina i-latitude, i-longitude kusizindalwazi sakho futhi, yithi, ugcine indawo oya kuyo, uma, isibonelo, unamaloli noma izindiza ezindizayo, ungakwazi ukudweba izindlela lapho uya khona. Futhi lapho ungasetha isivinini kanye nosayizi walezi zinto lapho zindiza khona.
Kodwa inkinga ngale mephu ukuthi idweba imephu yomhlaba kuphela, akukho mininingwane.
Kamuva ngengeza i-Google map. Uma ugcina i-latitude, i-longitude, ungadonsa umphumela kumephu ye-Google, kodwa ngaphandle kokusekelwa kwendiza.
Sixoxile ngemisebenzi esemqoka yokusebenza ngemiphumela nemibuzo ku-Tabix.
Okulandelayo ukuhlaziya iseva yakho ye-ClickHouse. Kukhona ithebhu ehlukile "Yemethrikhi", lapho ungabona khona usayizi wedatha egciniwe yekholomu ngayinye. Isithombe-skrini sibonisa ukuthi le nkambu "ye-referrer" ithatha cishe i-730 Gb. Uma sishiya lo mkhakha, sizogcina ama-shards amathathu we-700 GB ngalinye, okungukuthi mayelana ne-2 TB esingayidingi.
Futhi sinenkambu ethi "request_id" esiyigcina kuyunithi yezinhlamvu. Kodwa uma siqala ukuyigcina ngendlela yezinombolo, le nkambu izoncipha kakhulu.
Iphinde ibonise ukucushwa kweseva kanye nohlu lwamanodi kuqoqo lakho.
Ithebhu elandelayo amamethrikhi. Bangena esikhathini sangempela nge-ClickHouse futhi bavele bakuvumele ukuthi uhlaziye isimo seseva futhi uqonde ukuthi kwenzekani kuso. Lokhu akusona isikhala sikaGrafana ogcwele. Lokhu kuyadingeka ukuze kuhlaziywe ngokushesha.
Ithebhu elandelayo yizinqubo. Kusuka kubo ungaqonda ukuthi kwenzekani kuseva. Qonda ukuthi kwenzakalani lapho. Nganginesicelo esasidla u-200 GB wokufunda isikhathi ngasinye. Ngibone lokhu ngenxa yalesi interface. Ngambamba ngamlungisa. Futhi kuvele kucishe kube ngu-30 GB, okungukuthi ukusebenza kwesinye isikhathi.
Ngiyabonga! Futhi lokhu ku-OpenSource
Ngiqedile. Futhi-ke, i-OpenSource, imahhala futhi awudingi ukuyilanda. Ivule esipheqululini futhi yonke into izosebenza.
Imibuzo yakho
Igor, yini elandelayo? Uzolithuthukisa kuphi leli thuluzi?
Okulandelayo, amadeshibhodi azovela, okungukuthi, mhlawumbe amadeshibhodi azovela. Ukuhlanganiswa nezinye izingosi zolwazi. Ngikwenzile lokhu, kepha angikakakushicileli ku-OpenSource. Lena i-MySQL futhi mhlawumbe ne-PostgreSQL. Okusho ukuthi, kuzokwazi ukuthumela izicelo ezivela ku-Tabix hhayi kuphela ku-ClickHouse, kodwa nakwamanye amathuluzi.
Kuyacaca ukuthi mkhulu umsebenzi osewenziwe. Kwaba umqondo ophelele. Lokhu kwenziwa esipheqululini, ngokusobala, ukuze kuqedwe izinduku kuzo zonke izinhlobo zezimbazo futhi ngokushesha aphonse yonke into ndawonye. Ngizwe ukuthi uvuliwe php sebenza, ngakho-ke indlela elula ukuyibhala esipheqululini futhi izosebenza yonke indawo. Ayikho imibuzo ngalokhu. Umbuzo yilokhu. Kuningi ngempela okwenziwe lapho. Bangaki abantu abasebenze kulokhu? Futhi konke kwathatha isikhathi esingakanani? Ngoba amathuluzi wangokwezifiso ngokuvamile awanawo umsebenzi ongaka.
Umuntu oyedwa eqenjini lethu usebenze kusukela ehlobo kuya ekwindla. Lena kwakuyinguqulo yokuqala. Ngabe sengizibophezela ngama-330 ngedwa. Okubonayo, mina nozakwethu sikwenze phakathi. Ezinyangeni ezi-3, kusukela enguqulweni yokuqala kuya kweyokugcina, ngikwenze ngedwa kakhulu. Kodwa angiyazi kahle i-Javascript. Lona bekuwukuphela kwami futhi, ngithemba, iphrojekthi yami yokugcina ye-Javascript engasebenza nayo. Ngiyitholile, ngabheka - oh, ukwethuka. Kodwa ngangifuna ngempela ukuqeda umkhiqizo futhi yilokhu okwenzekile.
Siyabonga kakhulu ngombiko! Leli ithuluzi elikhulu. NGE ithebula Uqhathanise?
Ngiyabonga. Yingakho ngayiqamba ngokuthi i-Tabix, ngoba izinhlamvu zokuqala ziyefana.
Ngoba uyaqhudelana?
Kuzoba nokutshalwa kwezimali okuningi, sizoqhudelana.
Unganikela kanjani ngokuthengisa kubahlaziyi bangaphakathi ukuthi leli thuluzi lizongena esikhundleni ngokuphelele *Ithebula*? Ziyoba yini izingxabano?
Isebenza ngokomdabu nge-ClickHouse. Ngizamile i-Tableau, kodwa awukwazi ukubhala ukusekelwa kwezichazamazwi nokunye okunjalo. Ngiyazi ukuthi abantu basebenza kanjani neTabix. Babhala umbuzo, bawulayishe ku-CSV futhi bawulayishe ku-BI. Futhi kukhona kakade abakwenzayo lapho. Kodwa nginenkinga yokucabanga ukuthi bakwenza kanjani lokhu, ngoba iyithuluzi lokuqhafaza. Ingakwazi ukulayisha imigqa engu-5, ubukhulu bemigqa engu-000, kodwa ngeke kusaba khona, ngaphandle kwalokho isiphequluli ngeke sibhekane nakho.
Okungukuthi, kunemikhawulo engathi sína enanini ledatha, akunjalo?
Yebo. Angicabangi ukuthi ungafuna ukulayisha imigqa engu-10 etafuleni lakho esikrinini sakho sesiphequluli. Kwani?
Ingabe lokhu kusho ukuthi lesi isixhumi esibonakalayo sokubuka idatha ngokushesha? Isonte kancane, isonte?
Yebo, bheka ngokushesha ukuthi isebenza kanjani futhi uvele wakhe igrafu yesifinyezo. Bese unikela kwenye indawo. Sinesistimu yethu yokubika, lapho ngivele ngithathe khona lesi sicelo. Ngidweba i-Tabix futhi ngiyithumele ekubikeni kwethu.
Futhi omunye umbuzo. Ukuhlaziywa kweqembu?
Uma kukhona izicelo, sizozengeza.
Uqale nini ukuyisebenzisa? ClickHouse, ukuqaliswa kuthathe isikhathi esingakanani? ChofozaHouse nokuletha ku isimo sokukhiqiza?
Njengoba ngishilo, sisebenzise iqoqo lokuhlola ngesikhathi esifushane kakhulu. Siyifake ezinsukwini ezimbili. Futhi sayihlola amasonto ambalwa ngaphezulu. Futhi sifinyelele ekukhiqizeni ezinyangeni ezi-3. Kodwa sasine-ETL yethu, okungukuthi ithuluzi lokurekhoda idatha. Futhi wabhala kukho konke ayengakwenza. Angabhala ku-MongoDB, Cassandra, MySQL. Kwakulula ukumfundisa ukubhala ku-ClickHouse. Besinengqalasizinda eseyenziwe ngomumo ukuze isetshenziswe ngokushesha. Phakathi nezinyanga ezi-3 saqala ukulahla ingxenye yokuqala. Ezinyangeni eziyisi-6 sakushiya ngokuphelele konke okunye. Sisele ne-ClickHouse eyodwa kuphela.
U-Igor, ngiyabonga kakhulu ngombiko. Ngiwuthande kakhulu umsebenzi wokwakha izindlela kusetshenziswa amamephu. Ingabe zikhona izinhlelo zokuhlanganisa ne-Yandex.Maps futhi ikakhulukazi ne-Yandex.Maps yangokwezifiso?
Ngizamile ukuhlanganisa esikhundleni semephu ye-Google, kodwa angizange ngithole itimu emnyama ku-Yandex.Maps. Angikutshelanga nocezu olulodwa. Ngizohlehlisa ukuze ngingeze.
Isilayidi - imephu ye-Google. Kukhona umyalo “DRAW_GMAPS”, odweba imephu. Kukhona umyalo othi “DRAW_YMAPS”, okungukuthi ingadweba i-Yandex.Map. Kodwa empeleni, ngaphansi kwalo myalo kukhona i-Javascript, okungukuthi idatha oyithola ku-ClickHouse ingadluliselwa ku-Javascript, oyibhala lapha. Futhi unendawo yokukhipha lapho kufanele idwetshwe khona. Ungadweba noma iyiphi igrafu, okungukuthi noma iyiphi igrafu, imephu, ungadweba ingxenye yakho. Ngaphambi kwalokhu, benginomunye umtapo wolwazi wokudweba amagrafu ngokwawo.
Okusho ukuthi, ingabe likhona ithuluzi lokwenza ngokwezifiso ukusebenza kwesibonisi?
Noma yikuphi. Ungathatha futhi ufake umbala kabusha lawa machashazi, uwenze angabi bomvu, kodwa aluhlaza okwesibhakabhaka, aluhlaza.
Siyabonga ngombiko! Ubuneslayidi esethule amanye amathuluzi emibuzo ChofozaHouse ukwakha amadeshibhodi nemibiko yokuhlaziya. Ngiyakuqonda lokho okwamanje ngenkathi uqala ukusebenza nawe ClickHouse, awekho ama-adaptha abhalelwe la mathuluzi. Futhi ngiyazibuza ukuthi kungani unqume ukwenza ithuluzi lakho siqu, esikhundleni sokubhala i-adaptha yethuluzi elenziwe ngomumo? Ngicabanga ukuthi ukulungisa isihleli sokuhlola kuyashesha. Kungani unqume ukwenza umsebenzi ongaka?
Kukhona iphuzu elithakazelisayo lapha - iqiniso liwukuthi ngingumqondisi wezobuchwepheshe, hhayi usosayensi wedatha. Ngesikhathi siqala ukusebenzisa i-Druid, imephu yami yomgwaqo yayinemisebenzi engaba ngu-50% - ake sibale lokhu, noma asibale lokhu, noma sihlaziye lokhu. Futhi kwavela ukuthi sisebenzise i-ClickHouse. Futhi waqala ngokushesha ukwakha yonke into, ukubala, futhi ngokushesha wavala umgwaqo wakhe. Futhi ngaleso sikhathi ngabona ukuthi ngangintula ulwazi kwi-Data Science kanye nokubonwa kwedatha. I-Tabix iwuhlobo lomsebenzi wami wasekhaya wokufunda ukubona idatha. Bengibheka indlela yokugcwalisa i-Zeppelin. Angikuthandi kancane ukuhlela kwakhe. I-Redash ngibheke ukuthi ngiyengeza kanjani, kodwa umhleli ojwayelekile ubenganele. Futhi iSuperSet nayo ibhalwe ngolimi engingaluthandi ngempela. Futhi ngakho nganquma ukujikeleza, futhi yilokhu okwenzekile.
Igor, uyazamukela izicelo zokudonsa?
Yebo.
Siyabonga kakhulu ngombiko! Futhi imibuzo emibili. Okokuqala, awukhulumi ngokuthopha kakhulu I-Javascript. Ingabe ubhale nge-Javascript engenalutho noma uhlobo oluthile lohlaka?*
Kungcono kuJavascript engenalutho.
Ngakho yiluphi uhlaka?
I-Angular.
Kuyabonakala. Nombuzo wesibili. Uke wacabanga R и *Shiny**?*
Kucatshangelwe. Kudlaliwe.
Ungakwazi futhi ukubhala i-adaptha.
Ungu. Kubonakala sengathi umphakathi ukwenzile, kodwa, njengoba ngiphendule umbuzo odlule, ngangifuna ukuzizama mina ngokwami.
*Cha, mayelana nokubuka ngeso lengqondo, likhona futhi.
Uthi kukhona into enjalo futhi izokudwebela igrafu. Ngivule incwadi ekhuluma ngedatha. Futhi ngacabanga: “Ake ngizame ukubona le datha ngeso lengqondo. Ngizombhalela ukuze akwazi ukwakha kabusha ama-data.” Futhi ngaqala ukuqonda kangcono ubuchwepheshe bokuphakelwa kwedatha. Futhi ukube ngithathe ingxenye eseyenziwe ngomumo, mina ngokwami ngabe ngifunde okubi kakhulu ukuthi isetshenziswa kanjani, okungukuthi, ukubona ngeso lengqondo. Kodwa yebo, ngiwuthandile u-R, kodwa angikayifundi incwadi ethi “R for Dummies” okwamanje.
Siyabonga!
Umbuzo olula. Ingabe zikhona izindlela zokulayisha ngokushesha uphawu noma ishejuli?
Ingalayishwa ku-CSV noma ku-Excel.
Hhayi idatha, kodwa ipuleti esenziwe ngomumo, igrafu esenziwe ngomumo? Isibonelo, ukukhombisa umphathi.
Kukhona inkinobho "Layisha" futhi kukhona inkinobho "Layisha igrafu ku-png, ku-jpg".
Siyabonga!
I-PS Mini-imiyalelo yokufaka i-tabix
- Landa
- Khipha, kopisha umkhombandlela
buildku-nginx root_path - Lungiselela i-nginx
Source: www.habr.com