Sezvo ClickHouse iri yakasarudzika system, kana uchiishandisa zvakakosha kufunga nezve maumbirwo ayo. Mumushumo uyu, Alexey achataura nezvemienzaniso yezvikanganiso zvakajairika pakushandisa ClickHouse, izvo zvinogona kutungamirira kukusashanda basa. Mienzaniso inoshanda icharatidza kuti kusarudza imwe kana imwe dhata kugadzirisa chirongwa kunogona kuchinja kuita nemirairo yehukuru.
Mhoroi mose! Ini ndinonzi Alexey, ndinogadzira ClickHouse.
Chekutanga, ini ndinokurumidza kukufadza iwe ipapo, nhasi handisi kuzokuudza kuti ClickHouse chii. Kutaura chokwadi ndaneta nazvo. Pese pandinokuudza kuti chii. Uye zvichida munhu wose anotoziva.
Pane kudaro, ini ndichakuudza izvo zvinogona kukanganisa zviripo, ndiko kuti, mashandisiro aungaita ClickHouse zvisizvo. Muchokwadi, hapana chikonzero chekutya, nekuti isu tiri kugadzira ClickHouse sehurongwa huri nyore, huri nyore, uye hunoshanda kunze kwebhokisi. Ndakaiisa, hapana matambudziko.
Asi iwe uchiri kufanira kufunga kuti iyi sisitimu yakasarudzika uye iwe unogona nyore kusangana neyakajairwa yekushandisa kesi iyo inobvisa iyi system kubva munzvimbo yayo yekunyaradza.
Saka, rudzii rweraki iripo? Zvizhinji ndichataura pamusoro pezvinhu zviri pachena. Zvose zviri pachena kune vose, munhu wose anonzwisisa zvose uye anogona kufara kuti ivo vakachenjera, uye avo vasinganzwisisi vachadzidza chimwe chinhu chitsva.
Muenzaniso wekutanga uye wakapfava, uyo, zvinosuruvarisa, unowanzoitika, nhamba huru yekuisa ine mabheti maduku, i.e. nhamba huru yezvinyorwa zviduku.
Kana tikafunga kuti ClickHouse inoita sei kuisa, saka unogona kutumira ingangoita terabyte yedata muchikumbiro chimwe. Harisi dambudziko.
Uye ngationei kuti maitiro akajairika angave api. Semuenzaniso, tine tafura kubva kuYandex.Metrica data. Hits. 105 mamwe makoramu. 700 bytes isina kudzvanywa. Uye isu tichaisa nenzira yakanaka mumabheti emiriyoni imwe mitsara.
Isu tinopinza MergeTree mutafura, inobuda hafu yemiriyoni mitsara pasekondi. Hukuru. Mutafura yakadzokororwa ichava diki diki, ingangoita 400 mitsara pasekondi.
Uye kana iwe ukagonesa kuisa quorum, iwe unowana zvishoma zvishoma, asi zvakadaro zvine hunhu, 250 mazwi pasekondi. Kuiswa kweQuorum chinhu chisina kunyorwa muClickHouse*.
* kubva muna 2020, .
Chii chinoitika kana ukaita chimwe chinhu chakaipa? Isu tinoisa mutsara mumwe muMergeTree tafura uye tinowana mitsara makumi mashanu nepfumbamwe pasekondi. Ndizvo zvakapetwa ka59. MuReplicatedMergeTree - mitsara mitanhatu pasekondi. Uye kana quorum yakabatidzwa, zvino inobuda mitsetse miviri pasekondi. Mukuona kwangu, iyi imhando yehupenzi hwakakwana. Unganonoka sei kudaro? Ini ndakatonyora paT-shirt yangu kuti ClickHouse haifanirwe kunonoka. Asi zvakadaro zvinoitika dzimwe nguva.
Kutaura zvazviri, uku ndiko kukanganisa kwedu. Tingadai takaita kuti zvinhu zvese zvishande zviri nyore, asi hatina. Uye isu hatina kuzviita nekuti script yedu yaisada. Taive tatova nemabhucha. Isu takangogamuchira mabheti pamusuwo wedu, uye hapana matambudziko. Isu tinoiisa uye zvese zvinoshanda zvakanaka. Asi, chokwadi, marudzi ose ezviitiko zvinogoneka. Semuenzaniso, kana iwe uine boka remaseva pane iyo data inogadzirwa. Uye ivo havaisi data kazhinji, asi ivo vachiri kupedzisira nekuiswa kazhinji. Uye isu tinoda neimwe nzira kunzvenga izvi.
Kubva pane tekinoroji yekuona, poindi ndeyekuti kana iwe ukaita yekuisa muClickHouse, iyo data haipere mune chero inoyeuka. Isu hatina kana yerogi chaiyo MergeTree, asi ingori MergeTree, nekuti hapana danda kana memTable. Isu tinongokurumidza kunyora iyo data kune iyo faira system, yakatorongwa mumakoramu. Uye kana uine 100 columns, ipapo mafaera anopfuura mazana maviri anozoda kunyorerwa kune rakasiyana dhairekitori. Zvose izvi zvakaoma zvikuru.
Uye mubvunzo unomuka: "Ungazviita sei?" Kana mamiriro acho akaita zvekuti iwe uchiri kuda kurekodha data muClickHouse.
Nzira 1. Iyi ndiyo nzira iri nyore. Shandisa imwe mhando yemutsara wakagoverwa. Somuenzaniso, Kafka. Iwe unongotora data kubva kuKafka uye kuibata kamwe sekondi. Uye zvese zvichave zvakanaka, iwe unorekodha, zvese zvinoshanda zvakanaka.
Izvo zvisingabatsiri ndezvekuti Kafka imwe yakakura yakagoverwa system. Ini zvakare ndinonzwisisa kana iwe uchitova neKafka mukambani yako. Zvakanaka, zviri nyore. Asi kana isipo, saka iwe unofanirwa kufunga katatu usati wadhonza imwezve yakagoverwa system muprojekiti yako. Uye saka zvakakodzera kufunga dzimwe nzira.
Nzira 2. Iyi ndiyo imwe nzira yechikoro chekare uye panguva imwechete iri nyore kwazvo. Iwe une imwe mhando yeseva inogadzira matanda ako. Uye inongonyora matanda ako kufaira. Uye kamwe kwesekondi, semuenzaniso, tinopa zita iri faira uye tinobvarura imwe nyowani. Uye script yakaparadzana, ingave kuburikidza necron kana imwe daemon, inotora yekare faira uye inonyora kuna ClickHouse. Kana iwe ukanyora matanda kamwe chete sekondi, ipapo zvese zvichave zvakanaka.
Asi iyo yakashata yeiyi nzira ndeyekuti kana server yako iyo iyo matanda inogadzirwa inonyangarika pane imwe nzvimbo, ipapo iyo data ichanyangarika zvakare.
Nzira 3. Pane imwe nzira inofadza, iyo isingadi mafaira enguva pfupi zvachose. Semuenzaniso, une imwe mhando yekushambadzira spinner kana imwe inonakidza daemon inogadzira data. Uye iwe unogona kuunganidza boka re data zvakananga mu RAM, mu buffer. Uye kana nguva yakakwana yapfuura, unoisa iyi buffer parutivi, gadzira imwe nyowani, uye mune imwe tambo, isa izvo zvakatounganidzwa muClickHouse.
Kune rimwe divi, iyo data zvakare inonyangarika nekuuraya -9. Kana server yako ikaparara, iwe ucharasikirwa neiyi data. Uye rimwe dambudziko nderekuti kana usina kukwanisa kunyora kune dhatabhesi, ipapo data rako richaungana mu RAM. Uye ingave iyo RAM ichapera, kana iwe unongorasikirwa nedata.
Nzira 4. Imwe nzira inofadza. Iwe une imwe mhando ye server process. Uye inogona kutumira data kuClickHouse nekukasira, asi zviite mukubatana kumwe. Semuenzaniso, ndakatumira chikumbiro che http nekutumira-encoding: chunked nekuisa. Uye inogadzira chunks kwete kashoma, unogona kutumira mutsara wega wega, kunyangwe pachange paine pamusoro pekugadzira iyi data.
Nekudaro, mune iyi nyaya data ichatumirwa kuClickHouse nekukasira. Uye ClickHouse ichazvidzivirira pachayo.
Asi matambudziko anomukawo. Zvino iwe ucharasikirwa nedata, kusanganisira kana maitiro ako aurayiwa uye kana iyo ClickHouse maitiro akaurayiwa, nekuti ichave isina kukwana yekuisa. Uye muClickHouse inoisa maatomu kusvika kune imwe yakatsanangurwa chikumbaridzo muhukuru hwemitsara. Muchidimbu, iyi inzira inofadza. Inogonawo kushandiswa.
Nzira 5. Heino imwe nzira inofadza. Iyi ndiyo imwe mhando yenharaunda-yakagadziridzwa server ye data batching. Handina kuzvitarisa ini, saka hapana chandinogona kuvimbisa. Nekudaro, hapana vimbiso inopihwa ClickHouse pachayo. Iyi zvakare yakavhurika sosi, asi kune rimwe divi, iwe unogona kushandiswa kune imwe mhando chiyero chatinoedza kupa. Asi nekuda kwechinhu ichi - ini handizive, enda kuGitHub, tarisa iyo kodhi. Pamwe vakanyora zvakajairika.
* kubva muna 2020, inofanirawo kuwedzerwa mukufunga .
Nzira 6. Imwe nzira ndeyokushandisa Buffer tables. Kubatsira kweiyi nzira ndeyekuti iri nyore kwazvo kutanga kushandisa. Gadzira tafura yeBuffer uye uiise mairi.
Izvo zvisingabatsiri ndezvekuti dambudziko harina kugadziriswa zvachose. Kana, muchiyero chakaita seMergeTree, iwe unofanirwa kuunganidza data nebatch imwe pasekondi, ipapo muchiyero mutafura yebuffer, unofanirwa kuunganidza zvinosvika zviuru zvakati pasekondi. Kana inopfuura 10 000 pasekondi, inenge ichiri yakaipa. Uye kana iwe ukaiisa mumabhechi, iwe wakaona kuti inoshanduka kuita mazana ezviuru mitsara pasekondi. Uye izvi zvatove pa data rinorema.
Uye zvakare buffer matafura haana danda. Uye kana pane chimwe chinhu chakaipa nevhavha yako, ipapo data icharasika.
Uye sebhonasi, isu nguva pfupi yadarika takawana mukana paClickHouse wekutora data kubva kuKafka. Pane injini yetafura - Kafka. Iwe unongogadzira. Uye iwe unogona kusungirira mifananidzo yakagadzirwa pairi. Muchiitiko ichi, iyo pachayo inobvisa data kubva kuKafka uye kuiisa mumatafura aunoda.
Uye chinonyanya kufadza pamukana uyu ndechekuti hatisi isu takazviita. Ichi chinhu chenharaunda. Uye kana ndichiti "nharaunda yenharaunda," ndinoreva pasina kuzvidza. Takaverenga kodhi, takaita ongororo, inofanirwa kushanda zvakanaka.
* kubva muna 2020, rutsigiro rwakafanana rwakaonekwa .
Chii chimwe chingave chisina kunaka kana chisingatarisirwi kana uchiisa data? Kana iwe ukaita yekuisa kukosha chikumbiro uye nyora mamwe akaverengerwa mataurirwo mumitengo. Semuenzaniso, ikozvino () zvakare ishoko rakaverengerwa. Uye mune iyi kesi, ClickHouse inomanikidzwa kuburitsa muturikiri wemashoko aya pamutsetse wega wega, uye kuita kunodonha nemirairo yehukuru. Zviri nani kunzvenga izvi.
* parizvino, dambudziko ragadziriswa zvachose, hapasisina kudzoreredzwa kwekuita kana uchishandisa mataurirwo muZVINHU.
Mumwe muenzaniso ndeye panogona kunge paine mamwe matambudziko kana uine data pane rimwe batch iro riri reboka rezvikamu. Nekusagadzikana, ClickHouse zvikamu zvemwedzi. Uye kana iwe ukaisa batch yemiriyoni mitsara, uye paine data kwemakore akati wandei, ipapo iwe uchave neakati wandei mapartitions ipapo. Uye izvi zvakaenzana nenyaya yekuti pachava nemabheti akati wandei makumi mashoma muhukuru, nekuti mukati iwo anogara akapatsanurwa kuita zvikamu.
* Munguva pfupi yapfuura, mukuyedza mode, ClickHouse yakawedzera tsigiro yeiyo compact fomati ye chunks uye chunks mu RAM ine kunyora-mberi logi, iyo inopedza zvachose dambudziko.
Zvino ngatitarisei rudzi rwechipiri rwedambudziko - data typing.
Kunyora data kunogona kuve kwakasimba kana tambo. Tambo ndipo pawakangoitora ukataura kuti minda yako yese ndeyemhando yetambo. Izvi zvakaipa. Hapana chikonzero chekuita izvi.
Ngationei kuti tingazviita sei nenzira kwayo mune izvo zviitiko kana iwe uchida kutaura kuti isu tine imwe munda, tambo, uye rega ClickHouse izvione yega, uye ini handizvinetse. Asi zvichiri kukosha kuita nhamburiko yakati.
Semuenzaniso, tine IP kero. Mune imwe nguva, takaichengeta setambo. Somuenzaniso, 192.168.1.1. Uye mune imwe nyaya, ichave nhamba yemhando UInt32 *. 32 bits inokwana IPv4 kero.
Chekutanga, zvisingaite, iyo data ichamanikidzwa zvakaenzana. Pachava nemusiyano, hongu, asi kwete yakakura kudaro. Saka hapana akakosha matambudziko ne diski I / O.
Asi pane mutsauko wakakomba munguva yeprocessor uye nguva yekubvunza mibvunzo.
Ngativerengei huwandu hweakasarudzika IP kero kana akachengetwa senhamba. Izvi zvinoshanda kusvika 137 miriyoni mitsetse pasekondi. Kana zvakafanana zviri muchimiro chetambo, ipapo mitsara yemamiriyoni makumi matatu nemanomwe pasekondi. Handizivi kuti sei masangana aya akaitika. Ndakaita zvikumbiro izvi pachangu. Asi zvakadaro kanenge ka37 zvishoma zvishoma.
Uye kana iwe ukaverenga mutsauko mu disk space, ipapo panewo kusiyana. Uye mutsauko ungangoita kota, nekuti kune akawanda akasiyana IP kero. Uye dai paive nemitsara ine nhamba shoma yerevo dzakasiyana, zvino yaizotsikirirwa zviri nyore maererano neduramazwi kuita rinenge vhoriyamu imwe chete.
Uye musiyano wenguva wakapetwa mana haurambi munzira. Hameno kuti iwe hausi kutombozvipa mhosva, asi pandinoona mutsauko wakadaro, zvinondishungurudza.
Ngatitarisei nyaya dzakasiyana.
1. Imwe nyaya kana uine mashoma akasiyana akasiyana maitiro. Mune ino kesi, isu tinoshandisa yakapusa tsika yaungangoziva uye yaunogona kushandisa kune chero DBMS. Izvi zvese zvine musoro kwete zveClickHouse chete. Ingonyora zviziviso zvenhamba mudura re database. Uye iwe unogona kushandura kuita tambo uye kumashure kudivi rekushandisa kwako.
Semuenzaniso, une dunhu. Uye uri kuedza kuichengeta setambo. Uye ichanyorwa ipapo: Moscow uye Moscow Region. Uye pandinoona kuti inoti "Moscow", haisi chinhu, asi kana iri kuMoscow, neimwe nzira inosuruvara zvachose. Aya ndiwo ma byte mangani.
Pane kudaro, tinongonyora pasi nhamba Ulnt32 uye 250. Tine 250 muYandex, asi yako inogona kunge yakasiyana. Zvingoitika, ndichataura kuti ClickHouse ine yakavakirwa-mukati kugona kushanda ne geobase. Iwe unongonyora pasi dhairekitori rine matunhu, kusanganisira hierarchical, i.e. pachava neMoscow, Moscow Region, uye zvese zvaunoda. Uye iwe unogona kushandura padanho rekukumbira.
Yechipiri sarudzo inenge yakafanana, asi nerutsigiro mukati meClickHouse. Iyi ndiyo Enum data type. Iwe unongonyora zvese zvakakosha zvaunoda mukati meEnum. Semuenzaniso, rudzi rwechigadzirwa uye nyora ipapo: desktop, nhare, piritsi, TV. Pane 4 sarudzo pamwe chete.
Izvo zvakashata ndezvekuti iwe unofanirwa kuchinjeka nguva nenguva. Imwe sarudzo yakawedzerwa. Ngatiitei alter table. Muchokwadi, shandura tafura muClickHouse ndeyemahara. Kunyanya yemahara yeEnum nekuti iyo data pa diski haichinji. Asi zvakadaro, alter inowana kiyi* patafura uye inofanira kumirira kusvikira zvese zvasarudzwa zvaitwa. Uye chete mushure mekunge shanduko iyi yaitwa, i.e. pachine zvimwe zvinokanganisa.
* mushanduro dzichangoburwa dzeClickHouse, ALTER inogadzirwa isina-kuvharika zvachose.
Imwe sarudzo yakasarudzika yeClickHouse kubatanidza maduramazwi ekunze. Unogona kunyora manhamba muClickHouse, uye chengetedza madhairekitori ako mune chero system yakakunakira. Semuenzaniso, unogona kushandisa: MySQL, Mongo, Postgres. Iwe unogona kunyange kugadzira yako microservice iyo inotumira iyi data kuburikidza ne http. Uye paClickHouse level, iwe unonyora basa rinozoshandura iyi data kubva kunhamba kuenda kune tambo.
Iyi inzira yakasarudzika asi inoshanda kwazvo yekuita kujoinha patafura yekunze. Uye pane zvingasarudzwa zviviri. Mune imwe embodiment, iyi data ichave yakanyatso cached, iripo zvizere mu RAM uye yakagadziridzwa neimwe frequency. Uye mune imwe sarudzo, kana iyi data isingakwane mu RAM, saka unogona kuichengeta zvishoma.
Heino muenzaniso. Pane Yandex.Direct. Uye kune kambani yekushambadzira uye mabhena. Panogona kunge paine makumi emamiriyoni emakambani ekushambadza. Uye dzinokwana mu RAM. Uye kune mabhiriyoni emabhena, haakodzeri. Uye isu tinoshandisa duramazwi rakachengetwa kubva kuMySQL.
Dambudziko chete nderekuti duramazwi rakachengetwa rinoshanda zvakanaka kana iyo hit rate iri pedyo ne100%. Kana iri diki, saka kana uchigadzirisa mibvunzo yebatch yega yega data, iwe unotofanira kutora makiyi akashaikwa uye enda kunotora iyo data kubva kuMySQL. Nezve ClickHouse, ini ndinogona kuvimbisa kuti - hongu, hainonoke, handisi kuzotaura nezve mamwe masisitimu.
Uye sebhonasi, maduramazwi inzira iri nyore yekudzoreredza data muClickHouse. Ndiko kuti, iwe waive neshumo pamusoro pemakambani ekushambadzira, mushandisi akangochinja kambani yekushambadzira uye mune ese ekare data, mumishumo yese, iyi data yakachinjawo. Kana iwe ukanyora mitsara zvakananga patafura, hazvizogoneke kuivandudza.
Imwe nzira kana iwe usingazive kwaungawane zviziviso zvetambo dzako. unogona kungozvipedza. Uyezve, iyo yakapusa sarudzo ndeye kutora 64-bit hashi.
Dambudziko chete nderekuti kana hashi iri 64-bit, ipapo iwe unenge uine chokwadi nekudhumhana. Nekuti kana paine bhiriyoni mitsetse ipapo, saka mukana watove unooneka.
Uye hazvingave zvakanaka kwazvo kuhashi mazita emakambani ekushambadza nenzira iyi. Kana mishandirapamwe yekushambadzira yemakambani akasiyana-siyana yakasanganiswa, ipapo pachava nechinhu chisinganzwisisiki.
Uye pane nyore trick. Ichokwadi, haina kunyatsokodzera data rakakomba, asi kana chimwe chinhu chisina kunyanya kuoma, saka ingo wedzera chiziviso chemutengi kune kiyi yeduramazwi. Uye ipapo iwe uchave nekubonderana, asi mukati memutengi mumwe chete. Uye isu tinoshandisa nzira iyi yekubatanidza mepu muYandex.Metrica. Tine maURL ipapo, tinochengeta hashes. Uye tinoziva kuti, hongu, kune kudhumhana. Asi kana peji racho richiratidzwa, mukana wekuti pane imwe peji remumwe mushandisi mamwe ma URL akabatana uye izvi zvinozoonekwa zvinogona kuregeredzwa.
Sebhonasi, kune akawanda maoperation hashes chete akakwana uye tambo pachadzo hadzidi kuchengetwa chero kupi.
Mumwe muenzaniso ndewe kana tambo dziri pfupi, semuenzaniso, mawebhusaiti domains. Dzinogona kuchengetwa sezvazviri. Kana, semuenzaniso, mutauro webrowser ru ndeye 2 bytes. Zvechokwadi, ndinonyatsonzwira tsitsi mabheti, asi usanetseka, 2 bytes haisi tsitsi. Ndapota chengeta sezvazviri, usanetseka.
Imwe nyaya ndeye apo, pane zvinopesana, kune mitsetse yakawanda uye kune yakawanda yakasiyana-siyana mavari, uye kunyange iyo yakagadzirirwa inogona kusingagumi. Muenzaniso wakajairika mitsara yekutsvaga kana maURL. Tsvaga zvirevo, kusanganisira typos. Ngationei kuti mangani mazwi ekutsvaga akasiyana aripo pazuva. Uye zvinoitika kuti vanenge hafu yezviitiko zvose. Uye mune iyi kesi, iwe unogona kufunga kuti iwe unofanirwa kugadzirisa iyo data, kuverenga zviziviso, uye kuiisa mune yakaparadzana tafura. Asi iwe haufaniri kuita izvozvo. Ingochengeta mitsetse iyi sezvairi.
Zviri nani kusagadzira chero chinhu, nekuti kana ukachichengeta zvakasiyana, iwe unozofanirwa kujoina. Uye kujoinha uku, pane zvakanakisa, kungowana mukana kundangariro, kana ichiri kukwana mundangariro. Kana isingakodzeri, ipapo pachava nezvinetso.
Uye kana iyo data yakachengetwa munzvimbo, zvino inongoverengwa muhurongwa hunodiwa kubva kune faira system uye zvose zvakanaka.
Kana iwe uine maURL kana imwe yakaoma yakaoma tambo refu, saka zvakakodzera kuti ufunge kuti iwe unogona kuverenga imwe mhando yekubvisa pachine nguva uye kuinyora mune yakaparadzana koramu.
Kune ma URL, semuenzaniso, unogona kuchengeta iyo domain zvakasiyana. Uye kana iwe uchinyatsoda domain, saka ingo shandisa iyi koramu, uye maURL acharara ipapo, uye hautomboabata.
Ngationei kuti musiyano chii. ClickHouse ine basa rakasarudzika rinoverengera iyo domain. Inokurumidza zvikuru, isu takaigadzirisa. Uye, kutaura chokwadi, haina kana kuenderana neRFC, asi zvakadaro inofunga zvese zvatinoda.
Uye mune imwe nyaya isu tinongotora ma URL uye kuverenga iyo domain. Izvo zvinoshanda kusvika ku166 milliseconds. Uye kana iwe ukatora yakagadzirira-yakagadzirwa domain, zvino inozopedzisira yava 67 milliseconds, i.e. kanenge katatu nekukurumidza. Uye inokurumidza kwete nekuti isu tinoda kuita mamwe maverengero, asi nekuti isu tinoverenga data shoma.
Ndicho chikonzero chikumbiro chimwe, chinononoka, chine kukurumidza kwegigabytes pasekondi. Nekuti inoverenga mamwe gigabytes. Iyi idata isingakoshi zvachose. Chikumbiro chacho chinoita sechinomhanya, asi chinotora nguva yakareba kuti chipere.
Uye kana iwe ukatarisa huwandu hwe data pa diski, zvinosvika kuti iyo URL ndeye 126 megabytes, uye iyo domain inongova 5 megabytes. Inobuda ka25 zvishoma. Asi zvisinei, chikumbiro chinoitwa chete 4 nguva nekukurumidza. Asi imhaka yekuti data iri kupisa. Uye dai kwaitonhora, ingangoita 25 times nekukurumidza nekuda kwedisiki I/O.
Nenzira, kana iwe uchifungidzira kuti idiki diki diki pane URL, rinosvika riine madiki ka 4. Asi nekuda kwechimwe chikonzero, iyo data inotora 25 nguva shoma pa diski. Sei? Nekuda kwekumanikidza. Uye iyo URL inomanikidzwa, uye iyo domain inomanikidzwa. Asi kazhinji iyo URL ine boka remarara.
Uye, hongu, zvinobhadhara kushandisa iwo akakodzera data mhando dzakagadzirirwa zvakananga kune anodiwa kukosha kana akakodzera. Kana uri muIPv4, wobva wachengeta UInt32*. Kana IPv6, ipapo FixedString(16), nekuti IPv6 kero ndeye 128 bits, kureva yakachengetwa zvakananga mumabhinari fomati.
Asi ko kana iwe dzimwe nguva uine IPv4 kero uye dzimwe nguva IPv6? Hongu, unogona kuchengeta zvose. Imwe column yeIPv4, imwe yeIPv6. Ehe, pane sarudzo yekuratidza IPv4 muIPv6. Izvi zvichashanda zvakare, asi kana iwe uchiwanzoda IPv4 kero muzvikumbiro, zvino zvingave zvakanaka kuiisa muchikamu chakasiyana.
* ClickHouse ikozvino ine yakaparadzana IPv4, IPv6 dhata mhando dzinochengeta data nemazvo senhamba, asi inovamiririra zviri nyore setambo.
Izvo zvakakoshawo kuziva kuti zvakakosha kufanogadzirisa iyo data pachine nguva. Semuenzaniso, iwe unogamuchira mamwe matanda mbishi. Uye pamwe haufanirwe kungovaisa muClickHouse ipapo, kunyangwe iri kuyedza kusaita chinhu uye zvese zvinoshanda. Asi zvichiri kukosha kuita masvomhu anogoneka.
Semuenzaniso, browser version. Mune rimwe dhipatimendi riri pedyo, iro randisingadi kunongedza nemunwe, shanduro yebrowser inochengetwa seizvi, kureva, setambo: 12.3. Uye zvino, kuita rondedzero, vanotora tambo iyi voipatsanura kuita hurongwa, uyezve muchinhu chekutanga cheiyo array. Nomuzvarirwo, zvinhu zvose zvinononoka. Ndakabvunza kuti sei vachiita izvi. Vakandiudza kuti havadi kukurumidza kugadzirisa. Uye ini handifarire kukasira kutadza.
Saka muchiitiko ichi zvingava zvakanyanya kunaka kupatsanura kuita 4 columns. Usatye pano, nekuti iyi ClickHouse. ClickHouse ndeye columnar database. Uye zvakanyanya kutsvinda makoramu madiki, zviri nani. Pachave ne5 BrowserVersions, ita makoramu mashanu. Izvi zvakanaka.
Zvino ngatitarisei zvekuita kana uine tambo refu refu, dzakareba kwazvo. Ivo havadi kuchengetwa muClickHouse zvachose. Pane kudaro, unogona chete kuchengeta chiziviso muClickHouse. Uye isa iyi mitsetse mirefu mune imwe hurongwa.
Semuyenzaniso, imwe yedu yekuongorora masevhisi ine zvimwe zviitiko paramita. Uye kana paine maparamendi akawanda ezviitiko, tinongo chengetedza ekutanga 512 anouya nekuti 512 haisi tsitsi.
Uye kana iwe usingakwanise kusarudza pamhando dzako dzedata, saka iwe unogona zvakare kurekodha data muClickHouse, asi mune yechinguva tafura yeLog mhando, yakakosha yenguva pfupi data. Mushure meizvi, iwe unogona kuongorora kuti ndekupi kugovera kwehunhu kwaunayo ikoko, chii chiripo kazhinji, uye kugadzira iwo chaiwo marudzi.
*ClickHouse ikozvino ine data data iyo inokubvumira kuchengetedza tambo zvakanaka nekuedza kushoma.
Zvino ngatitarisei imwe nyaya inofadza. Dzimwe nguva zvinhu zvinoshanda zvisinganzwisisike kuvanhu. Ndichipinda ndichiona izvi. Uye zvinongoita sekunge izvi zvakaitwa nemumwe ane ruzivo, akangwara admin ane ruzivo rwakakura mukumisikidza MySQL vhezheni 3.23.
Pano tinoona matafura ane chiuru, rimwe nerimwe richinyora zvakasara zvekugovanisa ndiani anoziva chii nechuru.
Muchidimbu, ndinoremekedza ruzivo rwevamwe vanhu, kusanganisira kunzwisisa kwekutambura kunogona kuwanikwa kuburikidza nechiitiko ichi.
Uye zvikonzero zvakanyanya kana zvishoma zvakajeka. Aya ndiwo maitiro echinyakare anogona kunge akaungana achishanda nemamwe masisitimu. Semuenzaniso, matafura eMyISAM haana kiyi yakasanganiswa. Uye iyi nzira yekugovanisa data inogona kunge iri yekuda kuedza kuwana mashandiro akafanana.
Chimwe chikonzero ndechekuti zvakaoma kuita chero mashandiro ekuchinja pamatafura makuru. Zvese zvichavharwa. Kunyangwe mushanduro dzemazuva ano dzeMySQL dambudziko iri harichasiri rakanyanya.
Kana, semuenzaniso, microsharding, asi zvimwe pane izvo gare gare.
Iko hakuna chikonzero chekuita izvi muClickHouse, nekuti, chekutanga, kiyi yekutanga yakaunganidzwa, iyo data inorairwa nekiyi yekutanga.
Uye dzimwe nguva vanhu vanondibvunza: "Maitiro emibvunzo yakasiyana-siyana muClickHouse inosiyana sei zvichienderana nehukuru hwetafura?" Ndinoti hazvichinje zvachose. Semuenzaniso, une tafura ine bhiriyoni mitsara uye iwe unoverenga huwandu hwemiriyoni mitsara. Zvese zvakanaka. Kana paine mitsara yetiririyoni mutafura uye iwe ukaverenga miriyoni mitsetse, inenge yakafanana.
Uye, chechipiri, marudzi ese ezvinhu senge mawoko partitions hazvidiwi. Kana iwe ukapinda uye ukatarisa zviri pafaira system, uchaona kuti tafura ihombe hombe. Uye pane chimwe chinhu chakafanana nemapartitions mukati. Ndokunge, ClickHouse inokuitira zvese uye haufanirwe kutambura.
Alter muClickHouse ndeyemahara kana alter wedzera/donhedza mbiru.
Uye iwe haufanirwe kuita matafura madiki, nekuti kana uine mitsara gumi kana gumi mitsara mutafura, saka hazvina basa zvachose. ClickHouse isystem inokwirisa throughput, kwete latency, saka hazvina musoro kugadzirisa gumi mitsara.
Zvakarurama kushandisa tafura imwe huru. Bvisa mafungiro ekare, zvese zvichanaka.
Uye sebhonasi, mune yazvino vhezheni isu tave nekwaniso yekugadzira isingaite yekukamura kiyi kuitira kuti tiite ese marudzi ekugadzirisa mabasa pane ega ega.
Semuenzaniso, iwe unoda akawanda matafura madiki, semuenzaniso, kana paine kudikanwa kwekugadzirisa imwe data yepakati, unogamuchira chunks uye iwe unofanirwa kuita shanduko pazviri usati wanyorera kune yekupedzisira tafura. Kune iyi kesi, kune inoshamisa tafura injini - StripeLog. Izvo zvakaita seTinyLog, zvirinani chete.
* ikozvino ClickHouse zvakare ine .
Imwe antipattern ndeye microsharding. Semuenzaniso, iwe unofanirwa kushard data uye une maseva mashanu, uye mangwana pachava ne5 maseva. Uye iwe unofunga nezve nzira yekudzoreredza iyi data. Uye panzvimbo pacho iwe hausi kutyora mu6 shards, asi kuita 5 shards. Uye wobva watora mepu imwe neimwe yeiyi microshards kune yakaparadzana sevha. Uye iwe uchawana, semuenzaniso, 1 ClickHouses pane imwe server, semuenzaniso. Mamiriro akasiyana pane akasiyana madoko kana akaparadzana dhatabhesi.
Asi izvi hazvina kunyatsonaka muClickHouse. Nekuti kunyange imwe ClickHouse muenzaniso inoedza kushandisa ese aripo sevha zviwanikwa kugadzirisa chikumbiro chimwe. Ndiko kuti, une imwe sevha uye ine, semuenzaniso, 56 processor cores. Uri kumhanyisa mubvunzo unotora sekondi imwe uye ichashandisa 56 cores. Uye kana iwe ukaisa mazana maviri eClickHouses ipapo pane imwe sevha, zvino zvinozoitika kuti zviuru gumi zvichatanga. Kazhinji, zvinhu zvose zvichange zvakaipa zvikuru.
Chimwe chikonzero ndechekuti kugovaniswa kwebasa mukati mezviitiko izvi kuchave kusina kuenzana. Vamwe vachapedza kare, vamwe vanozopedzisa gare gare. Kana zvese izvi zvikaitika muchiitiko chimwe, ipapo ClickHouse pachayo yaizofunga nzira yekugovanisa data pakati petambo.
Uye chimwe chikonzero ndechekuti iwe uchave neinterprocessor kutaurirana kuburikidza neTCP. Iyo data ichafanirwa kuve seri, deserialized, uye iyi ihombe nhamba yemamicroshards. Hazvizongoshandi zvakanaka.
Imwe antipattern, kunyangwe isingakwanise kunzi antipattern. Ichi chitsama chikuru chepre-aggregation.
Kazhinji, pre-aggregation yakanaka. Wakanga uine bhiriyoni mitsara, wakaiunganidza uye ikava 1 mitsara, uye zvino mubvunzo unoitwa ipapo ipapo. Zvese zvakanaka. Unogona kuita izvi. Uye kune izvi, kunyange ClickHouse ine yakakosha tafura mhando, AggregatingMergeTree, iyo inoita yekuwedzera aggregation sezvo data rinoiswa.
Asi pane dzimwe nguva dzaunofunga kuti tichaunganidza data seizvi uye kuunganidza data seizvi. Uye mune rimwe dhipatimendi rakavakidzana, ini handidiwo kutaura kuti nderipi, vanoshandisa SummingMergeTree matafura kupfupikisa nekiyi yekutanga, uye anenge makumi maviri makoramu anoshandiswa sekiyi yekutanga. Zvingoitika, ndakachinja mazita emamwe makoramu ekuvanzika, asi ndizvo chaizvo.
Uye matambudziko akadaro anomuka. Chekutanga, vhoriyamu yako yedata haina kuderera zvakanyanya. Somuenzaniso, inoderera katatu. Katatu dzingave mutengo wakanaka kutenga iyo isina muganho analytics kugona kunomuka kana data rako risina kuunganidzwa. Kana iyo data yakaunganidzwa, saka pachinzvimbo cheanalytics unowana chete nhamba dzinosiririsa.
Uye chii chakakosha nezvazvo? Chokwadi ndechekuti vanhu ava vanobva kudhipatimendi rakavakidzana dzimwe nguva vanoenda kunokumbira kuwedzera imwe koramu kukiyi yekutanga. Ndiko kuti, isu takaunganidza data seizvi, asi ikozvino tinoda zvishoma. Asi ClickHouse haina alter primary kiyi. Naizvozvo, tinofanira kunyora mamwe magwaro muC ++. Uye ini handifarire zvinyorwa, kunyangwe zviri muC ++.
Uye kana iwe ukatarisa izvo ClickHouse yakasikirwa, saka isina-yakaunganidzwa data ndiyo chaiyo mamiriro ayo akazvarirwa. Kana uri kushandisa ClickHouse kune isina-akaunganidzwa data, saka uri kuzviita nemazvo. Kana iwe ukaunganidza, izvi dzimwe nguva zvinoregererwa.
Imwe nyaya inonakidza ndeye mibvunzo mune isingaperi loop. Dzimwe nguva ndinoenda kune imwe sevha yekugadzira ndotarisa show processlist ipapo. Uye pese pandinoona kuti pane chinhu chinotyisa chiri kuitika.
Somuenzaniso, zvakadai. Zviri pachena kuti zvese zvinogona kuitwa muchikumbiro chimwe chete. Ingonyora url mukati uye runyorwa ipapo.
Sei mibvunzo yakawanda yakadai mune isingaperi loop yakaipa? Kana index isingashandiswe, ipapo iwe uchave uine akawanda anopfuura pane imwechete data. Asi kana index ikashandiswa, semuenzaniso, une kiyi yekutanga ye ru uye unonyora url = chimwe chinhu ipapo. Uye iwe unofunga kuti kana URL imwe chete ikaverengwa kubva patafura, zvese zvichanaka. Asi chaizvoizvo kwete. Nekuti ClickHouse inoita zvese muma batches.
Paanenge achida kuverenga imwe dhata, anoverenga zvishoma, nekuti index iri muClickHouse ishoma. Iyi indekisi haibvumire kuti uwane mutsara mumwe chete patafura, chete mutsara weimwe mhando. Uye iyo data inomanikidzwa mumabhuraki. Kuti uverenge mutsara mumwe chete, unofanirwa kutora bhuroka rese uye woribvisa. Uye kana iwe uri kuita boka remibvunzo, iwe unenge uine yakawanda yekupindirana, uye iwe unenge uine basa rakawanda rekuita kakawanda.
Uye sebhonasi, unogona kuona kuti muClickHouse haufanirwe kutya kuendesa kunyangwe megabytes uye kunyange mazana emamegabytes kune IN chikamu. Ndinoyeuka kubva mukuita kwedu kuti kana muMySQL tikaendesa boka rehutano kuchikamu cheIN, semuenzaniso, tinoendesa 100 megabytes yedzimwe nhamba ipapo, ipapo MySQL inodya 10 gigabytes yekuyeuka uye hapana chimwe chinoitika kwairi, zvose. inoshanda zvisina kunaka.
Uye chechipiri ndechekuti muClickHouse, kana mibvunzo yako ikashandisa index, saka inogara isinganonoke kupfuura scan yakazara, kureva kuti kana iwe uchida kuverenga inenge tafura yese, inoenda zvakateerana uye kuverenga tafura yese. Muzhinji, iye achazvifunga ari oga.
Asi zvisinei pane zvimwe zvinonetsa. Semuenzaniso, chokwadi chekuti IN ine subquery haishandise index. Asi iri ndiro dambudziko redu uye tinofanira kurigadzirisa. Hapana chakakosha apa. Tichazvigadzirisa*.
Uye chimwe chinhu chinonakidza ndechekuti kana iwe uine chikumbiro chakareba uye nekugovera chikumbiro chiri kuenderera mberi, saka ichi chikumbiro chakareba kwazvo chinotumirwa kune yega yega server pasina kudzvanywa. Semuenzaniso, 100 megabytes uye 500 maseva. Uye, maererano, iwe uchave ne50 gigabytes inotamiswa pamusoro pe network. Ichafambiswa uye ipapo zvese zvinopedzwa zvinobudirira.
* yatove kushandisa; Zvose zvakagadziriswa sezvakavimbiswa.
Uye nyaya yakajairika ndeye apo zvikumbiro zvinobva kuAPI. Semuenzaniso, iwe wakagadzira imwe mhando yebasa rako pachako. Uye kana mumwe munhu achida sevhisi yako, saka unovhura iyo API uye chaizvo mazuva maviri gare gare unoona kuti chimwe chinhu chisinga nzwisisike chiri kuitika. Zvese zvakaremerwa uye zvimwe zvikumbiro zvinotyisa zviri kuuya izvo zvaifanira kunge zvisina kumboitika.
Uye pane mhinduro imwe chete. Kana iwe wakavhura iyo API, saka uchafanirwa kuicheka. Semuyenzaniso, suma imwe mhando yemakowata. Hapana dzimwe sarudzo dzakajairika. Kana zvisina kudaro, ivo vachakurumidza kunyora script uye pachava nezvinetso.
Uye ClickHouse ine yakakosha chimiro - quota calculation. Uyezve, iwe unogona kutamisa yako quota kiyi. Izvi, semuenzaniso, iyo yemukati mushandisi ID. Uye quotas ichaverengerwa yakazvimirira kune yega yega.
Zvino chimwe chinhu chinonakidza. Uku kudzokorodza kwemaoko.
Ndinoziva nezvezviitiko zvakawanda uko, kunyangwe ClickHouse ine yakavakirwa-mukati tsigiro yekudzokorora, vanhu vanodzokorora ClickHouse nemawoko.
Chii chiri musimboti? Une pombi yekugadzira data. Uye inoshanda yakazvimirira, semuenzaniso, munzvimbo dzakasiyana dze data. Iwe unonyora iyo data yakafanana nenzira imwechete muClickHouse. Ichokwadi, kudzidzira kunoratidza kuti data richaramba richisiyana nekuda kwezvimwe zvinhu mukodhi yako. Ndinovimba yave mamuri.
Uye nguva nenguva iwe uchafanirwa kuwiriranisa nemaoko. Semuenzaniso, kamwe chete pamwedzi admins vanoita rsync.
Muchokwadi, zviri nyore kushandisa kudzokorodza kwakavakirwa muClickHouse. Asi panogona kunge paine zvimwe zvinopesana, nekuti nekuda kweizvi iwe unofanirwa kushandisa ZooKeeper. Ini handisi kuzotaura chero chinhu chakaipa nezveZooKeeper, in musimboti, iyo system inoshanda, asi zvinoitika kuti vanhu havaishandise nekuda kwejava-phobia, nekuti ClickHouse ndiyo yakanaka system, yakanyorwa muC ++, yaunogona kushandisa uye zvese zvichanaka . Uye ZooKeeper iri mujava. Uye neimwe nzira hautomboda kutarisa, asi ipapo unogona kushandisa bhuku rekudzokorora.
ClickHouse inzira inoshanda. Anorangarira zvaunoda. Kana iwe uine bhuku rekudzokorora, saka iwe unogona kugadzira tafura Yakagoverwa inotarisa yako replicas yemanyorero uye inoita faindi pakati pavo. Uye kune kana yakasarudzika sarudzo inobvumidza iwe kuti udzivise flops, kunyangwe kana mitsetse yako ichisiyana.
Mamwe matambudziko anogona kumuka kana ukashandisa primitive tafura injini. ClickHouse muvaki ane boka remainjini ematafura akasiyana. Panyaya dzese dzakakomba, sezvakanyorwa muzvinyorwa, shandisa matafura kubva kumhuri yeMergeTree. Uye zvimwe zvese - izvi zvakadaro, kumatare ega ega kana bvunzo.
Mune MergeTree tafura, haufanire kuve uine chero zuva uye nguva. Unogona kuishandisa. Kana pasina zuva uye nguva, nyora kuti default ndeye 2000. Izvi zvichashanda uye hazvizodi zviwanikwa.
Uye mune iyo nyowani vhezheni yeseva, unogona kutotsanangura kuti une tsika yekugovera pasina kiyi yekuparadzanisa. Zvichava zvimwe chetezvo.
Kune rimwe divi, unogona kushandisa primitive tafura injini. Semuenzaniso, zadza iyo data kamwe uye tarisa, shandura uye bvisa. Unogona kushandisa Log.
Kana kuchengetedza mavhoriyamu madiki ekugadziriswa kwepakati ndeye StripeLog kana TinyLog.
Memory inogona kushandiswa kana huwandu hwe data hudiki uye iwe unogona kungo twiddle chimwe chinhu mu RAM.
ClickHouse hainyatsoda renormalized data.
Heino muenzaniso chaiwo. Iyi nhamba huru ye URLs. Unoaisa mutafura inotevera. Uye vakabva vafunga kuita JOIN navo, asi izvi hazvishande, semutemo, nekuti ClickHouse inotsigira Hash JOIN chete. Kana pasina RAM yakakwana kune yakawanda data inoda kubatana, saka JOIN haishande*.
Kana iyo data iri yekadhiini yepamusoro, saka usazvidya moyo, ichengetedze mune denormalized fomu, ma URL ari zvakananga panzvimbo mutafura huru.
* uye ikozvino ClickHouse zvakare ine yekubatanidza kujoinha, uye inoshanda mumamiriro ezvinhu apo data repakati risingakwane mu RAM. Asi izvi hazvishande uye kurudziro inoramba ichishanda.
Mimwe mienzaniso yakati wandei, asi ini ndinotokahadzika kuti inopesana nemuenzaniso here kana kuti kwete.
ClickHouse ine chimwe chikanganiso chinozivikanwa. Iyo haizive nzira yekuvandudza*. Mune dzimwe nzira, izvi zvakatonaka. Kana iwe uine data yakakosha, semuenzaniso, accounting, saka hapana achakwanisa kuitumira, nekuti hapana zvigadziriso.
* Tsigiro yekuvandudza uye kudzima mubatch modhi yakawedzerwa kare kare.
Asi kune dzimwe nzira dzakakosha dzinobvumira zvigadziriso sekunge zviri kumashure. Semuenzaniso, matafura senge ReplaceMergeTree. Ivo vanogadziridza panguva yekubatanidzwa kwemashure. Unogona kumanikidza izvi uchishandisa optimize tafura. Asi usaite izvi kakawanda, nekuti zvinozopenengura chikamu.
Yakagoverwa JOINs muClickHouse zvakare haina kubatwa zvakanaka nemubvunzo kuronga.
Zvakaipa, asi dzimwe nguva Ok.
Uchishandisa ClickHouse chete kuverenga data kumashure uchishandisa select*.
Ini handingakurudzire kushandisa ClickHouse kune zvinorema kuverenga. Asi ichi hachisi chokwadi zvachose, nekuti isu tave kutobva kure nekurudziro iyi. Uye isu munguva pfupi yapfuura takawedzera kugona kushandisa michina yekudzidza modhi muClickHouse - Catboost. Uye zvinondinetsa nekuti ndinofunga, βZvinotyisa. Aya ndiwo maitiro mangani pabyte inobuda! Ndinovenga kutambisa wachi pamabhaiti.
Asi usatya, isa ClickHouse, zvese zvichanaka. Kana paine, tine nharaunda. Nenzira, nharaunda ndiwe. Uye kana uine chero matambudziko, unogona kuenda kune yedu chat, uye tinovimba ivo vachakubatsira.
Mibvunzo yako
Ndatenda nemushumo! Ndekupi kwandinganyunyuta nezve ClickHouse kuparara?
Unogona kunyunyuta kwandiri pachangu izvozvi.
Ini nguva pfupi yadarika ndakatanga kushandisa ClickHouse. Ndakabva ndangodonhedza cli interface.
Chii chibodzwa.
Kwapera chinguva ndakapwanya server nediki yakasarudzwa.
Une tarenda.
Ndakavhura GitHub bug, asi haina kufuratirwa.
Ngationei.
Alexey akandinyengedza kuti ndiende kumushumo, achivimbisa kundiudza kuti unowana sei data mukati.
Very simple.
Izvi ndakazviona nezuro. Zvimwe zvakanangana.
Hakuna manomano anotyisa ipapo. Pane chete block-by-block compression. Iyo default ndeye LZ4, unogona kugonesa ZSTD*. Zvivharo kubva pa64 kilobytes kusvika 1 megabyte.
* kune zvakare tsigiro yehunyanzvi compression codecs inogona kushandiswa mucheni nemamwe maalgorithms.
Mabhuroko acho angori data raw?
Kwete zvachose mbishi. Kune arrays. Kana uine koramu yenhamba, ipapo manhamba mumutsara anoiswa muhurongwa.
Zviri pachena.
Alexey, muenzaniso wanga uine uniqExact pamusoro peIPs, i.e. chokwadi chekuti uniqExact inotora nguva yakareba kuverenga nemitsara kupfuura nenhamba, zvichingodaro. Zvakadini kana tikashandisa feint nenzeve dzedu uye kukanda panguva yekuongorora? Ndiko kuti, iwe unoratidzika kunge wakataura kuti pa diski yedu haina kunyanya kusiyana. Kana tikaverenga mitsara kubva kudhisiki uye kukanda, maunganidzwa edu achakurumidza here kana kuti kwete? Kana kuti tichawana zvishoma pano? Zvinoita sekuti wakaedza izvi, asi nekuda kwechimwe chikonzero hauna kuzviratidza mubhenji.
Ndinofunga zvichanonoka pane pasina kukanda. Muchiitiko ichi, kero ye IP inofanira kuparadzaniswa kubva patambo. Ehe, paClickHouse, yedu IP kero parsing zvakare yakagadziridzwa. Takaedza nesimba, asi ipapo une nhamba dzakanyorwa muchimiro chezviuru gumi. Kusagadzikana zvakanyanya. Kune rimwe divi, iyo uniqExact basa rinoshanda zvishoma nezvishoma patambo, kwete chete nekuti idzi tambo, asiwo nekuti imwe nyanzvi yegorgorithm inosarudzwa. Tambo dzinongogadziriswa zvakasiyana.
Ko kana tikatora imwe primitive data type? Semuenzaniso, isu takanyora pasi mushandisi id, iyo yatinayo mairi, takainyora pasi semutsara, uye ndokuikwenya, ichanyanya kunakidza kana kwete?
Ndinokahadzika. Ini ndinofunga zvichatonyanya kusuruvara, nekuti mushure mezvose, kupatsanura nhamba idambudziko rakakura. Zvinoratidzika kwandiri kuti uyu waanoshanda naye akatopa mushumo wekuti zvakaoma sei kupatsanura nhamba mune gumi rezviuru fomu, asi pamwe kwete.
Alexey, ndinokutendai zvikuru nemushumo! Uye ndinokutendai zvikuru nokuda kweClickHouse! Ndine mubvunzo pamusoro pezvirongwa. Pane here zvirongwa zvechinhu chekuvandudza maduramazwi zvisina kukwana?
Ndiko kuti, kutangazve zvishoma?
Hongu Hongu. Kufanana nekugona kuseta MySQL munda ipapo, i.e. gadziridza mushure kuitira kuti iyi data chete inotakurwa kana duramazwi rakakura kwazvo.
Chinhu chinonakidza kwazvo. Uye ndinofunga mumwe munhu akazvikurudzira mukutaura kwedu. Pamwe waitova iwewe.
Handifunge kudaro.
Hukuru, zvino zvinozoitika kuti pane zvikumbiro zviviri. Uye iwe unogona kutanga kuita zvishoma nezvishoma. Asi ini ndinoda kukuyambira iwe ipapo kuti chimiro ichi chiri nyore kuita. Ndiko kuti, muchirevo, iwe unongoda kunyora nhamba yeshanduro mutafura uye wozonyora: shanduro shoma pane yakadaro uye yakadaro. Izvi zvinoreva kuti, kazhinji, tichapa izvi kune vanofarira. Uri munhu anofarira here?
Hongu, asi, zvinosuruvarisa, kwete muC ++.
Vaunoshanda navo vanoziva kunyora muC ++ here?
Ndichawana munhu.
Hukuru*.
* chimiro chakawedzerwa mwedzi miviri mushure meshumo - munyori wemubvunzo akaugadzira ndokutumira yake .
Ndinokutendai!
Mhoro! Ndatenda nemushumo! Iwe wakataura kuti ClickHouse yakanaka kwazvo pakushandisa zvese zviwanikwa zviripo kwairi. Uye mutauri padivi paLuxoft akataura nezve mhinduro yake yeRussia Post. Akataura kuti vainyanya kufarira ClickHouse, asi havana kuishandisa pachinzvimbo chemukwikwidzi wavo mukuru chaizvo nekuti yaidya CPU yese. Uye ivo havana kukwanisa kuibaya mukuvaka kwavo, muZooKeeper yavo ine dockers. Zvinogoneka here kudzikamisa ClickHouse kuitira kuti isadye zvese zvinowanikwa kwairi?
Hongu, zvinogoneka uye zviri nyore kwazvo. Kana iwe uchida kushandisa mashoma macores, saka ingonyora set max_threads = 1. Uye ndizvozvo, ichaita chikumbiro mune imwe musimboti. Uyezve, iwe unogona kutsanangura marongero akasiyana evashandisi vakasiyana. Saka hapana dambudziko. Uye udza vamwe vako kubva kuLuxoft kuti hazvina kunaka kuti havana kuwana iyi marongero muzvinyorwa.
Alexey, mhoro! Ndinoda kubvunza pamusoro pemubvunzo uyu. Aka hakasi kekutanga kunzwa kuti vanhu vazhinji vari kutanga kushandisa ClickHouse sekuchengetedza matanda. Pamushumo wati usaite izvi, i.e. haufanirwe kuchengeta tambo refu. Unofungei nezvazvo?
Kutanga, matanda, sekutonga, kwete tambo refu. Kune, hongu, kunze. Semuenzaniso, imwe sevhisi yakanyorwa mujava inokanda yakasarudzika, inoiswa. Uye zvichingodaro mune isingaperi loop, uye nzvimbo pane hard drive inopera. Mhinduro yacho iri nyore kwazvo. Kana mitsetse yakareba kwazvo, woicheka. Kureba kunorevei? Makumi emakirobhayiti akaipa*.
* mushanduro dzichangoburwa dzeClickHouse, "adaptive index granularity" inogoneswa, izvo zvinobvisa dambudziko rekuchengetedza mitsetse mirefu kwechikamu chikuru.
Ko kilobyte yakajairika?
Okay
Mhoro! Ndatenda neshumo! Ndakatobvunza nezve izvi mukutaura, asi handiyeuke kana ndakagamuchira mhinduro. Pane zvirongwa here zvekuwedzera iyo WITH chikamu nenzira yeCTE?
Kwete izvozvi. Yedu WITH chikamu chinoti frivolous. Zvakafanana nechinhu chidiki kwatiri.
Ndinonzwisisa. Ndatenda!
Ndatenda neshumo! Zvinonakidza kwazvo! Mubvunzo wepasi rose. Pane here zvirongwa zvekugadzirisa kudzima data, pamwe muchimiro cheimwe rudzi rwe stubs?
Zvakakodzera. Iri ndiro basa redu rekutanga mumutsetse wedu. Iye zvino tiri kushingaira kufunga nezvekuti tingaita sei zvose nenzira yakarurama. Uye unofanira kutanga kudzvanya keyboard*.
* akadzvanya mabhatani ekhibhodi uye akaita zvese.
Izvi zvichakanganisa mashandiro ehurongwa here kana kuti kwete? Ko kuiswa kwacho kuchakurumidza here sezvazviri iye zvino?
Zvichida kuzvibvisa pachako uye zvigadziriso pachazvo zvichange zvichirema, asi izvi hazvizokanganisa mashandiro ezvisarudzo kana kuita kwekuisa.
Uye mumwe mubvunzo mudiki. Pamharidzo wakataura nezve primary key. Saizvozvo, isu tine partitioning, inova mwedzi wega wega, handizvo? Uye kana isu tikaisa zuva renji rinokwana mumwedzi, zvino chikamu ichi chete chinoverengwa, handiti?
Hongu.
Mubvunzo. Kana isu tisingakwanise kusarudza chero kiyi yekutanga, saka zvakaringana here kuzviita zvakananga zvinoenderana ne "Date" ndima kuitira kuti kumashure kune kushoma kurongeka kweiyo data kuitira kuti ikwane nenzira yakarongeka? Kana iwe usina mibvunzo yemhando uye haugone kana kusarudza chero kiyi yekutanga, zvakakodzera here kuisa zuva mukiyi yekutanga?
Hongu.
Zvimwe zvine musoro kuisa munda mukiyi yekutanga iyo inomanikidza iyo data zvirinani kana yakarongedzwa nendima iyi. Semuenzaniso, mushandisi ID. Mushandisi, semuenzaniso, anoenda kune imwechete saiti. Mune ino kesi, isa mushandisi id uye nguva. Uye ipapo data rako richave rakamanikidzwa zvirinani. Kana riri iro zuva, kana iwe usina uye usingambove nemhando yemibvunzo pamazuva, saka haufanirwe kuisa zuva racho mukiyi yekutanga.
OK ndinokutendai zvikuru!
Source: www.habr.com