Operator muKubernetes yekugadzirisa masumbu e database. Vladislav Klimenko (Altinity, 2019)

Chirevo ichi chakazvipira kune dzinoshanda nyaya dzekugadzira opareta muKubernetes, kugadzira dhizaini yayo uye nheyo dzekutanga dzekushandisa.

Muchikamu chekutanga chemushumo tichatarisa:

• chii chinonzi opareta muKubernetes uye nei ichidikanwa;
• sei chaizvo mushandisi anorerutsa manejimendi eakaoma masisitimu;
• izvo mushandisi anogona uye zvaasingakwanise kuita.

Tevere, ngatienderere mberi nekukurukura chimiro chemukati chemushandisi. Ngatitarisei kuvakwa uye kushanda kweanoshanda nhanho nhanho. Ngatitarisei zvakadzama:

• kudyidzana pakati peanoshanda naKubernetes;
• mabasa api anotorwa nemushandi uye mabasa api aanopa kuKubernetes.

Ngatitarisei kubata shards uye dhatabhesi replicas muKubernetes.
Zvadaro, tichakurukura nyaya dzekuchengetedza data:

• maitiro ekushanda nePersistent Storage kubva pakuona kwemushandisi;
• matambudziko ekushandisa Local Storage.

Muchikamu chekupedzisira chemushumo, tichaona mienzaniso inoshanda yekushandisa clickhouse-operator kubva kuAmazon kana Google Cloud Service. Chirevo ichi chakavakirwa pamuenzaniso wekuvandudza uye ruzivo rwekushanda kweanoshanda weClickHouse.

Vhidhiyo:

Ini ndinonzi Vladislav Klimenko. Nhasi ndanga ndichida kutaura nezve chiitiko chedu mukugadzira nekushanda mushandisi, uye uyu inyanzvi inoshanda yekugadzirisa masumbu e database. Semuyenzaniso ClickHouse-operator kubata iyo ClickHouse cluster.

Sei isu tine mukana wekutaura nezve opareta uye ClickHouse?

• Isu tinotsigira nekuvandudza ClickHouse.
• Parizvino, tiri kuyedza kuita zvishoma nezvishoma mupiro wedu mukuvandudza kweClickHouse. Uye isu tiri wechipiri mushure meYandex maererano nehuwandu hwekuchinja kwakaitwa kuClickHouse.
• Tiri kuedza kuita mamwe mapurojekiti eClickHouse ecosystem.

Ndinoda kukuudza nezvechimwe chezvirongwa izvi. Izvi ndezveClickHouse-operator yeKubernetes.

Mumushumo wangu ndinoda kubata pamisoro miviri:

• Nyaya yekutanga ndeyekuti yedu ClickHouse dhatabhesi manejimendi anoshanda sei muKubernetes.
• Chinyorwa chechipiri ndechekuti chero mushandisi anoshanda sei, kureva kuti anodyidzana sei neKubernetes.

Nekudaro, iyi mibvunzo miviri ichapindirana mushumo yangu yese.

Ndiani angafarira kuteerera zvandiri kuedza kutaura?

• Ichave inonyanya kufarira kune avo vanoshandisa maopareta.
• Kana kune avo vanoda kugadzira yavo kuitira kuti vanzwisise kuti inoshanda sei mukati, kuti mushandisi anodyidzana sei naKubernetes, uye ndeapi makomba angaonekwa.

Kuti unzwisise zvakanyanya zvatichakurukura nhasi, ipfungwa yakanaka kuziva mashandiro anoita Kubernetes uye kuve nekudzidziswa kwegore.

Chii chinonzi ClickHouse? Iyi dhatabhesi yekoronari ine chaiwo maficha epamhepo kugadzirisa emibvunzo yekuongorora. Uye yakavhurika zvachose sosi.

Uye zvakakosha kuti tizive zvinhu zviviri chete. Iwe unofanirwa kuziva kuti iyi dhatabhesi, saka zvandichakuudza zvichashanda kune chero dhatabhesi. Uye chokwadi chekuti ClickHouse DBMS inoyera zvakanyanya, inopa ingangoita mutsara scalability. Uye saka, iyo cluster state inzvimbo yakasikwa yeClickHouse. Uye isu tinonyanya kufarira kukurukura maitiro ekushandira iyo ClickHouse cluster muKubernetes.

Nei achidiwa ikoko? Sei isu tisingakwanisi kuramba tichiishandisa isu pachedu? Uye mhinduro dzine chikamu chehunyanzvi uye chikamu chesangano.

• Mukuita, isu tiri kuwedzera kusangana nemamiriro ezvinhu apo mumakambani makuru zvinenge zvese zvikamu zvatova muKubernetes. Databases inoramba iri kunze.
• Uye mubvunzo uri kuramba uchibvunzwa: "Izvi zvinogona kuiswa mukati?" Naizvozvo, makambani makuru ari kuyedza kuwana yakanyanya kubatana kwemanejimendi kuitira kuti ikurumidze kubata matura avo edata.
• Uye izvi zvinonyanya kubatsira kana iwe uchida mukana wakakura wekudzokorora chinhu chimwe chete munzvimbo itsva, i.e. kutakurika.

Zviri nyore kana kuti zvakaoma sei? Izvi, zvechokwadi, zvinogona kuitwa nemaoko. Asi hazvisi nyore, nekuti isu tine yakawedzera kuomarara kwekutonga Kubernetes pachayo, asi panguva imwe chete iyo chaiyo yeClickHouse yakasimudzwa. Uye kuunganidza kwakadaro kunoguma.

Uye zvese pamwe chete izvi zvinopa yakakura kwazvo seti yetekinoroji, inova yakaoma kwazvo kubata, nekuti Kubernetes inounza yayo yega nyaya dzemazuva ese kuti ishande, uye ClickHouse inounza zvayo nyaya pakushanda kwemazuva ese. Kunyanya kana isu tine akati wandei ClickHouses, uye isu tinofanirwa kugara tichiita chimwe chinhu navo.

Iine dhizaini yekumisikidza, ClickHouse ine huwandu hwakaringana huwandu hwezvinhu zvinogadzira unogara uchiremerwa paDevOps:

• Kana isu tichida kushandura chimwe chinhu muClickHouse, semuenzaniso, wedzera replica kana shard, saka isu tinoda kugadzirisa iyo gadziriso.
• Wobva wachinja iyo data schema, nekuti ClickHouse ine chaiyo sharding nzira. Ikoko iwe unofanirwa kuisa dhayagiramu yedata, kuisa zvigadziriso.
• Unofanira kuseta monitoring.
• Kuunganidza matanda emashadhi matsva, ezvinyorwa zvitsva.
• Chengetedza kudzorerwa.
• Uye restart.

Aya mabasa enguva dzose andinoda chaizvo kuita kuti zvive nyore kushandisa.

Kubernetes pachayo inobatsira mukushanda zvakanaka, asi pane zvekutanga system zvinhu.

Kubernetes yakanaka pakufambisa uye otomatiki zvinhu senge:

• Kudzorera.
• Restart.
• Storage system management.

Izvo zvakanaka, ndiyo nzira chaiyo, asi haana chaanoziva nezve mashandisiro anoita dhatabhesi cluster.

Tinoda zvimwe, tinoda kuti dhatabhesi rese rishande muKubernetes.

Ndinoda kuwana chimwe chinhu senge hombe bhatani dzvuku bhatani raunodzvanya uye sumbu rine mabasa emazuva ese anoda kugadziriswa anoiswa uye anochengetwa mukati mehupenyu hwayo hwese. ClickHouse cluster muKubernetes.

Uye takaedza kuita mhinduro yaizobatsira kuti basa rive nyore. Iyi iClickHouse-operator yeKubernetes kubva kuAltinity.

An opareta chirongwa chine basa guru kutonga mamwe mapurogiramu, kureva maneja.

Uye ine maitiro emaitiro. Iwe unogona kudaidza iyi kodhi ruzivo nezvenyaya yenyaya.

Uye basa rake guru nderekuita kuti hupenyu hweDevOps huve nyore uye kuderedza micromanagement, kuitira kuti iye (DevOps) atofunga mumashoko epamusoro-soro, kureva, kuitira kuti iye (DevOps) arege kuita micromanagement, kuitira kuti arege kugadzirisa. zvese zvese nemaoko.

Uye mushandisi chete ndiye robhoti mubatsiri anobata ne microtasks uye anobatsira DevOps.

Sei uchida mushandisi? Anoita zvakanyanya muzvikamu zviviri:

• Kana nyanzvi inobata neClickHouse isina ruzivo rwakakwana, asi yatoda kushanda ClickHouse, mutyairi anofambisa basa uye anobvumidza iwe kuti ushandise ClickHouse cluster ine yakaomesesa gadziriso, pasina kupinda mune yakawandisa dhata pamusoro pekuti zvese zvinoshanda sei. mukati. Iwe unongomupa mabasa epamusoro-soro, uye zvinoshanda.
• Uye iro rechipiri basa rarinoita zvakanyanya ndipo pazvinenge zvichidikanwa kuita otomatiki nhamba yakakura yeakajairika mabasa. Inobvisa microtasks kubva kune system administrator.

Izvi zvinonyanya kudiwa kungave neavo vachangotanga rwendo rwavo, kana neavo vanoda kuita akawanda otomatiki.

Ko iyo opareta-based approach inosiyana sei nemamwe masisitimu? Kune Helm. Izvo zvinobatsirawo kuisa ClickHouse; unogona kudhirowa machati ehelm, ayo anotoisa iyo yese ClickHouse cluster. Ndeupi zvino musiyano uripo pakati peanoshanda uye akafanana, semuenzaniso, Helm?

Musiyano mukuru wakakosha ndewekuti Helm is package manejimendi uye Operator anoenda nhanho mberi. Uku ndiko kutsigirwa kwehupenyu hwese. Uku hakusi kungoisa chete, aya mabasa emazuva ese anosanganisira scaling, sharding, i.e. zvese zvinoda kuitwa panguva yehupenyu kutenderera (kana zvichidikanwa, wozodzima zvakare) - izvi zvese zvinosarudzwa nemushandisi. Inoedza kuita otomatiki uye kuchengetedza iyo yese software lifecycle. Uyu ndiwo mutsauko wayo mukuru kubva kune dzimwe mhinduro dzinounzwa.

Ndiyo yaive introduction part, ngatienderere mberi.

Tinovaka sei mushandisi wedu? Isu tiri kuyedza kusvika kune iyo nyaya kubata iyo ClickHouse cluster sechinhu chimwe chete.

Pano isu tine data yekupinda kuruboshwe rwemufananidzo. Iyi iYAML ine cluster yakatarwa, iyo inopfuudzwa kuKubernetes nenzira yekare kuburikidza ne kubectl. Ikoko opareta wedu anoitora oita mashiripiti ake. Uye pakubuda tinowana chirongwa chinotevera. Uku kuita kweClickHouse muKubernetes.

Uye ipapo isu tichatarisa zvishoma nezvishoma kuti chaizvo mushandisi anoshanda sei, ndeapi akajairwa mabasa anogona kugadziriswa. Tichangofunga mabasa akajairika nekuti tine nguva shoma. Uye hazvisi zvese zvingasarudzwa nemushandisi zvichakurukurwa.

Ngatitange kubva pakudzidzira. Yedu purojekiti yakavhurika sosi, saka iwe unogona kuona kuti inoshanda sei paGitHub. Uye iwe unogona kuenderera kubva mukufunga kuti kana iwe uchingoda kuivhura, saka unogona kutanga neKukurumidza Kutanga Gwaro.

Kana iwe uchida kunzwisisa zvakadzama, saka tinoedza kuchengetedza zvinyorwa mune imwe kana yakaderera fomu.

Ngatitange nedambudziko rinoshanda. Basa rekutanga, apo isu tese tinoda kutanga, nderekumhanyisa muenzaniso wekutanga neimwe nzira. Ndingaite sei kuti nditange ClickHouse ndichishandisa mushandisi, kunyangwe ndisinganyatso kunyatsoziva kuti inoshanda sei? Tiri kunyora manifesto, nekuti... Kwese kutaurirana nek8s kutaurirana kuburikidza nemamanifesiti.

Iyi manifesto yakaoma kudaro. Zvatakasimbisa mutsvuku ndizvo zvatinofanira kutarisa pazviri. Isu tinokumbira mushandisi kuti agadzire cluster inonzi demo.

Iyi ndiyo mienzaniso yakakosha ikozvino. Kuchengetera hakusati kwatsanangurwa, asi isu tichadzokera kunzvimbo yekuchengetera gare gare. Parizvino, isu tichaona dynamics yekukura kwe cluster.

Isu takagadzira manifesto iyi. Tinozvidyisa kumushandisi wedu. Akashanda, akaita mashiripiti.

Isu tinotarisa pane console. Zvikamu zvitatu zvinofadza: Pod, maviri masevhisi, uye StatefulSet.

Mushandi akashanda, uye isu tinogona kuona izvo chaizvo zvaakagadzira.

Anogadzira chinhu chakadai. Tine StatefulSet, Pod, ConfigMap yega yega replica, ConfigMap yeboka rese. Masevhisi anodiwa semapoinzi ekupinda musumbu.

Masevhisi ndiwo epakati Load Balancer Service uye anogona zvakare kushandiswa kune yega replica, kune yega yega shard.

Basa redu rekutanga rinotaridzika seizvi. Inobva kune imwe node.

Ngatienderere mberi tichiomesa zvinhu. Tinofanira kupatsanura cluster.

Mabasa edu ari kukura, masimba ari kutanga. Tinoda kuwedzera chidimbu. Isu tinotevera budiriro. Isu tiri kushandura maitiro edu. Isu tinoratidza kuti tinoda zvidimbu zviviri.

Iri ndiro faira rimwe chete rinokura zvine simba nekukura kwehurongwa. Chengetedzo kwete, chengetedzo ichakurukurwa mberi, iyi inyaya yakasiyana.

Isu tinodyisa iyo YAML opareta uye toona zvinoitika.

Mushandi akafunga uye akaita zvinhu zvinotevera. Isu tatova nemaPods maviri, matatu masevhisi uye, kamwe kamwe, 2 StatefulSets. Sei 2 StatefulSets?

Pamufananidzo waive seizvi - iyi ndiyo mamiriro edu ekutanga, pataive nepodhi imwe.

Zvakava seizvi. Parizvino zvese zviri nyore, zvave zvakapetwa.

Uye nei pakava maviri StatefulSets? Pano isu tinofanirwa kudzika uye kukurukura mubvunzo wekuti Pods inotungamirwa sei muKubernetes.

Pane chinhu chinonzi StatefulSet chinokutendera kuti ugadzire seti yePods kubva patemplate. Chinhu chakakosha apa iTemplate. Uye iwe unogona kuvhura akawanda maPods uchishandisa imwe template mune imwe StatefulSet. Uye mutsara wakakosha pano ndewe "mapodhi mazhinji etemplate imwe."

Uye paive nemuyedzo mukuru wekugadzira cluster yese, kuirongedza muStatefulSet imwe. Ichashanda, hapana dambudziko nazvo. Asi pane imwe caveat. Kana isu tichida kuunganidza heterogeneous cluster, kureva, kubva akati wandei shanduro dzeClickHouse, ipapo mibvunzo inotanga kumuka. Ehe, StatefulSet inokwanisa kuita dhizaini, uye ipapo iwe unogona kuburitsa vhezheni itsva, tsanangura kuti haufanirwe kuyedza kupfuura akawanda manode panguva imwe chete.

Asi kana isu tikawedzera basa racho uye tichiti tinoda kugadzira sumbu rakasiyana zvachose uye hatidi kuchinja kubva kushanduro yekare kuenda kune itsva tichishandisa rolling update, asi isu tinongoda kugadzira sumbu rakasiyana-siyana zvese maererano. yemhando dzakasiyana dzeClickHouse uye maererano nekuchengetedza kwakasiyana. Tinoda, semuenzaniso, kugadzira mamwe mareplicas pamadhisiki akaparadzana, pane anononoka, kazhinji, kuvaka zvachose chisumbu chakasiyana. Uye nekuda kwekuti StatefulSet inogadzira yakamisikidzwa mhinduro kubva kune imwe template, hapana nzira yekuita izvi.

Mushure mekufunga, zvakasarudzwa kuti taizozviita nenzira iyi. Isu tine replica yega yega mune yayo StatefulSet. Pane zvimwe zvipingamupinyi kune iyi mhinduro, asi mukuita zvese zvakaputirwa zvachose nemushandisi. Uye kune zvakawanda zvakanakira. Isu tinogona kuvaka iyo chaiyo cluster yatinoda, semuenzaniso, iyo yakanyatso siyana. Naizvozvo, musumbu umo mune mashadhi maviri ane replica imwe, tichava ne2 StatefulSets uye 2 Pods chaizvo nekuti isu takasarudza nzira iyi nekuda kwezvikonzero zvataurwa pamusoro kuti tikwanise kuvaka sumbu rakasiyana.

Ngatidzokere kumatambudziko anoshanda. Muchikwata chedu tinoda kugadzirisa vashandisi, i.e. iwe unofanirwa kuita imwe gadziriso yeClickHouse muKubernetes. Mushandi anopa zvose zvingaita izvi.

Tinogona kunyora zvatinoda zvakananga muYAML. Ese sarudzo dzekugadzirisa dzakaiswa mepu yakananga kubva kune iyi YAML kuenda kuClickHouse configs, iyo inobva yagovaniswa musumbu rese.

Unogona kuzvinyora sezvizvi. Izvi semuenzaniso. Iyo password inogona kuvharwa. Chokwadi zvese zveClickHouse zvigadziriso zvinotsigirwa. Heino muenzaniso chete.

Iyo cluster configuration inogoverwa seConfigMap. Mukuita, iyo ConfigMap yekuvandudza haiitike ipapo ipapo, saka kana iyo cluster yakakura, ipapo maitiro ekusundidzira gadziriso anotora imwe nguva. Asi izvi zvese zviri nyore kushandisa.

Ngatiomese basa. Chisumbu chiri kukura. Tinoda kudzokorora data. Ndokunge, isu tatova nemashadhi maviri, imwe replica imwe neimwe, uye vashandisi vakagadzirirwa. Tiri kukura uye tinoda kuita kudzokorora.

Chii chatinoda pakudzokorora?

Tinoda ZooKeeper. MuClickHouse, kudzokorora kunovakwa uchishandisa ZooKeeper. ZooKeeper inodiwa kuitira kuti dzakasiyana ClickHouse replicas dzive nekubvumirana maererano nemabhuraki edata ari paClickHouse.

ZooKeeper inogona kushandiswa nemunhu wese. Kana iyo bhizinesi iine yekunze ZooKeeper, saka inogona kushandiswa. Kana zvisiri, unogona kuimisa kubva kune yedu repository. Pane installer inoita kuti chinhu ichi chive nyore.

Uye iyo yekudyidzana dhizaini yehurongwa hwese inobuda seizvi. Isu tine Kubernetes sepuratifomu. Inoita iyo ClickHouse opareta. Ndakafananidza ZooKeeper pano. Uye mushandisi anodyidzana neClickHouse uye ZooKeeper. Ndiko kuti, kudyidzana kunoguma.

Uye izvi zvese zvinodikanwa kuti ClickHouse ibudirire kudzokorora data muk8s.

Ngatitarisei basa racho pacharo, kuti manifest yekudzokorora ichataridzika sei.

Tiri kuwedzera zvikamu zviviri kune yedu manifest. Yekutanga ndiko kuwana ZooKeeper, inogona kunge iri mukati meKubernetes kana yekunze. Iyi ingori tsananguro. Uye isu tinoodha replicas. Avo. tinoda replicas mbiri. Pakazara, isu tinofanirwa kuve nemapodhi mana pakubuda. Tinoyeuka nezvekuchengetedza, ichadzoka zvishoma gare gare. Kuchengeta inyaya yakasiyana.

Zvakanga zvakaita seizvi.

Zvinoita sezvizvi. Replicas inowedzerwa. Yechina haina kukwana, isu tinotenda kuti panogona kunge paine vazhinji vavo ipapo. Uye ZooKeeper inowedzerwa kudivi. Zvirongwa zviri kuwedzera kuoma.

Uye inguva yekuwedzera basa rinotevera. Isu tichawedzera Kuramba Kuchengetedza.

Kune Inopfuurira Kuchengetedza tine sarudzo dzakasiyana siyana.

Kana isu tichimhanya mune cloud provider, semuenzaniso, kushandisa Amazon, Google, saka pane muedzo mukuru wekushandisa cloud storage. Zviri nyore chaizvo, zvakanaka.

Uye pane yechipiri sarudzo. Izvi ndezvekuchengetedza kwenzvimbo, kana tine madhisiki emunharaunda pane imwe neimwe node. Iyi sarudzo yakanyanya kuoma kuita, asi panguva imwechete inobereka zvakanyanya.

Ngationei zvatinazvo maererano nekuchengetwa kwegore.

Pane zvakanakira. Zviri nyore chaizvo kugadzirisa. Isu tinongoodha kubva kune wegore mupi uyo ndapota atipe chengetedzo yezvakadaro uye zvakati, zvekirasi yakadai. Makirasi anorongwa nevanopa vakazvimirira.

Uye pane drawback. Kune vamwe, iyi isiri-yakakosha drawback. Zvechokwadi, pachava nedzimwe nyaya dzekuita. Iri nyore kushandisa uye yakavimbika, asi pane zvimwe zvinogona kuita zvinokanganisa.

Uye nokuti ClickHouse inotarisa zvakanyanya pakubudirira, munhu anogona kutotaura kuti inosvina zvese zvazvinogona, ndosaka vatengi vazhinji vachiedza kudzvanya chibereko chakanyanya.

Uye kuti tiwane zvakanyanya kubva pazviri, tinoda kuchengetedza munharaunda.

Kubernetes inopa zvibodzwa zvitatu zvekushandisa chengetedzo yemuno muKubernetes. Izvi:

• EmptyDir
• HostPath.
• Local

Ngationei kuti akasiyana sei uye akafanana sei.

Chekutanga, mune ese matatu maitiro tine chengetedzo - aya madhisiki emunharaunda anowanikwa pane imwechete yemuviri k8s node. Asi vane zvimwe zvakasiyana.

Ngatitangei neicho chakareruka, i.e. emptyDir. Chii ichi mukuita? Mukutaura kwedu, tinokumbira iyo containerization system (kazhinji Docker) kuti itipe mukana kune folda pane yemuno dhisiki.

Mukuita, Docker inogadzira folda yenguva pfupi pane imwe nzvimbo munzira dzayo uye inoidaidza kuti hashi refu. Uye inopa interface yekuiwana.

Iri basa richaita sei nekuchenjera? Izvi zvichashanda panharaunda disk speed, i.e. Uku ndiko kuwana kuzere kune sikuruu yako.

Asi nyaya iyi ine drawback yayo. Kushingirira kunokahadzika panyaya iyi. Kekutanga Docker inofamba nemidziyo, Persistent inorasika. Kana Kubernetes achida kufambisa iyi Pod kune imwe dhisiki nekuda kwechimwe chikonzero, iyo data icharasika.

Iyi nzira yakanakira bvunzo, nekuti inotoratidza kukurumidza kukurumidza, asi kune chimwe chinhu chakakomba iyi sarudzo haina kukodzera.

Naizvozvo pane nzira yechipiri. Iyi ndiyo hostPath. Kana iwe ukatarisa kune yakapfuura slide neiyi, unogona kuona musiyano mumwe chete. Folda yedu yakasimuka kubva kuDocker yakananga kuKubernetes node. Zviri nyore pano. Isu tinotsanangura zvakananga nzira pane yemuno faira system kwatinoda kuchengetedza data redu.

Iyi nzira ine zvakanakira. Izvi zvatova Kushingirira chaiko, uye yemhando yepamusoro pane izvozvo. Tichava ne data yakanyorwa pane diski pane imwe kero.

Panewo zvazvakaipira. Uku ndiko kuoma kwehutungamiri. Yedu Kubernetes ingangoda kufambisa iyo Podhi kune imwe yemuviri node. Uye apa ndipo panopinda DevOps. Anofanira kutsanangura nenzira kwayo kuhurongwa hwese kuti mapodhi aya anogona kungotamiswa kune iwo manodhi pauri une chimwe chinhu chakaiswa munzira idzi, uye kwete inopfuura node imwe panguva. Zvakaoma hazvo.

Kunyanya nekuda kweizvi zvinangwa, takagadzira matemplate mune yedu opareta kuitira kuti tivanze zvese izvi zvakaomarara. Uye iwe unogona kungoti: "Ini ndinoda kuva nemuenzaniso weClickHouse yega yega node yemuviri uye nenzira yakadai neyakadaro."

Asi isu hatisi isu chete vanoda chido ichi, saka vanyoro veKubernetes pachavo vanonzwisisawo kuti vanhu vanoda kuwana madhisiki emuviri, saka vanopa chikamu chechitatu.

Inonzi yemuno. Ikoko hapana mutsauko kubva kune yapfuura siraidhi. Izvo zvisati zvave zvakakosha kuti tisimbise nemaoko kuti isu hatigone kutamisa mapodhi aya kubva kune node kuenda kune node, nekuti iwo anofanirwa kusungirirwa pane imwe nzira kune yemunharaunda dhisiki yenyama, asi ikozvino ruzivo urwu rwese rwakavharirwa muKubernetes pachayo. Uye zvinoita kuti zvive nyore kugadzirisa.

Ngatidzokere kudambudziko redu rinoshanda. Ngatidzokere kune YAML template. Pano tine chengetedzo chaiyo. Tadzoka pazviri. Isu tinoseta yemhando yepamusoro VolumeClaim template semu k8s. Uye isu tinotsanangura rudzi rwekuchengetedza rwatinoda.

Mushure meizvi, k8s ichakumbira kuchengetedza. Tichazvipa isu muStatefulSet. Uye pakupedzisira ichave pakushandisa ClickHouse.

Taive nechirongwa ichi. Kuchengeta Kwedu Kunoramba Kuripo kwakanga kwakatsvuka, izvo zvaiita sokunge kuti zvaifanira kuitwa.

Uye inoshanduka yakasvibira. Iye zvino iyo ClickHouse pane k8s cluster scheme inopedzwa zvachose. Isu tine shards, replicas, ZooKeeper, isu tine chaiyo Inoshingirira, iyo inoshandiswa neimwe nzira kana imwe. Chirongwa ichi chatova kushanda zvizere.

Tinoramba tichirarama. Chikwata chedu chiri kukura. Uye Alexey anoedza, uye anoburitsa vhezheni itsva yeClickHouse.

Basa rinoshanda rinomuka - kuyedza iyo itsva vhezheni yeClickHouse pane yedu cluster. Uye, sezvazviri, haudi kuiburitsa kunze; iwe unoda kuisa vhezheni nyowani mune imwe replica kumwe kukona iri kure, uye pamwe kwete imwe vhezheni itsva, asi mbiri kamwechete, nekuti dzinobuda kazhinji.

Tingati kudini pamusoro peizvi?

Pano tine mukana wakadaro. Aya ndiwo pod templates. Iwe unogona kunyora kuti opareta yedu inobvumidza zvachose iwe kuvaka heterogeneous cluster. Avo. gadzirisa, kutanga kubva kune ese mareplicas muboka, achipera nega yega replica, iyo vhezheni yatinoda ClickHouse, iyo vhezheni yatinoda kuchengetedza. Tinogona kugadzirisa zvizere cluster nekugadzirisa kwatinoda.

Ngatipindei mukati zvishoma. Izvi zvisati zvaitika, takataura nezve mashandiro anoita ClickHouse-operator zvine chekuita neiyo ClickHouse.

Iye zvino ndinoda kutaura mazwi mashoma pamusoro pekuti chero mushandisi anoshanda sei mune zvese, pamwe nemabatiro aanoita neK8s.

Ngatitarisei kudyidzana nemaK8 kutanga. Chii chinoitika kana isu tikaita kubectl kunyorera? Zvinhu zvedu zvinoonekwa mune etcd kuburikidza neAPI.

Semuenzaniso, zvakakosha Kubernetes zvinhu: pod, StatefulSet, sevhisi, zvichingodaro pasi pasi runyorwa.

Panguva imwecheteyo, hapana chenyama chinoitika. Zviro izvi zvinofanirwa kuitwa muchikwata.

Nechinangwa ichi, mutongi anooneka. Iyo controller chinhu chakakosha k8s chikamu chinogona kuita izvi tsananguro. Anoziva maitiro uye zvokuita mumuviri. Anoziva kumhanyisa midziyo, izvo zvinoda kugadziriswa ipapo kuitira kuti server ishande.

Uye inoita zvinhu zvedu muK8s.

Asi isu tinoda kushanda kwete chete nemapods uye StatefulSets, isu tinoda kugadzira ClickHouseInstallation, i.e. chinhu cheClickHouse mhando, kuitira kuti ishande nayo seyakakwana. Kusvika pari zvino hakuna mukana wakadaro.

Asi maK8 ane chinhu chinotevera chakanaka. Tinoda kuti tive nenzvimbo yakaita sechinhu chakaoma ichi umo sumbu redu raizounganidzwa kubva kumapods uye StatefulSet.

Uye chii chinofanira kuitwa nokuda kweizvi? Kutanga, Tsika Resource Tsanangudzo inouya mumufananidzo. Chii? Iyi ndiyo tsananguro yeK8s, kuti iwe uchave neimwe mhando yedata, iyo yatinoda kuwedzera tsika sosi kune pod, StatefulSet, iyo ichave yakaoma mukati. Iyi ndiyo tsananguro yechimiro che data.

Isu tinotumirawo ikoko kuburikidza ne kubectl shandisa. Kubernetes akazvitora nemufaro.

Uye ikozvino mukuchengetedza kwedu, chinhu chiri mu etcd chine mukana wekurekodha tsika sosi inonzi ClickHouseInstallation.

Asi parizvino hapana chimwe chichaitika. Ndokunge, kana isu tagadzira iyo YAML faira yatakatarisa ichitsanangura shards uye replicas toti "kubectl apply," ipapo Kubernetes anoigamuchira, oiisa etcd oti: "Zvakanaka, asi handizive zvekuita. nayo. Handizive kuchengetedza ClickHouseInstalation. "

Nekudaro, isu tinoda mumwe munhu wekubatsira Kubernetes kushandira iyo itsva data mhando. Kuruboshwe isu tine chizvarwa Kubernetes controller inoshanda nemhando dzemhando dzedata. Uye kurudyi isu tinofanirwa kuve netsika controller iyo inogona kushanda netsika data mhando.

Uye neimwe nzira inonzi mushandisi. Ini ndakaisanganisira pano seKubernetes, nekuti inogona zvakare kuurayiwa kunze kweK8s. Kazhinji kazhinji, hongu, vese vashandisi vanourayiwa muKubernetes, asi hapana chinoitadzisa kumira kunze, saka pano inofambiswa zvakanyanya kunze.

Uye zvakare, mutongi wetsika, anozivikanwawo seanoshanda, anodyidzana naKubernetes kuburikidza neAPI. Inotoziva nzira yekudyidzana neAPI. Uye anotoziva maitiro ekugadzirisa dunhu rakaoma iro isu tinoda kugadzira kubva kune tsika sosi. Izvi ndizvo chaizvo zvinoitwa nemushandi.

Mushandi anoshanda sei? Ngatitarisei kudivi rekurudyi tione kuti anozviita sei. Ngationei kuti mushandisi anopfeka sei zvese izvi uye kuti kuwedzeredza kupindirana neK8s kunoitika sei.

Mushandisi chirongwa. Iye akanangana nezviitiko. Mushandi anonyoresa kuzviitiko achishandisa Kubernetes API. Iyo Kubernetes API ine nzvimbo dzekupinda kwaunogona kunyorera kune zviitiko. Uye kana chimwe chinhu chichichinja muK8s, ipapo Kubernetes inotumira zviitiko kumunhu wose, i.e. ani nani akanyoresa kune ino API poindi anogashira zviziviso.

Mushandi anonyoresa kune zviitiko uye anofanira kuita imwe mhando yemhinduro. Basa rayo nderekupindura kune zviitiko zvinobuda.

Zviitiko zvinogadzirwa nezvimwe zvigadziriso. Yedu YAML faira ine tsananguro yeClickHouseInstallation inosvika. Akaenda ku etcd kuburikidza ne kubectl apply. Chiitiko chakatangwa ipapo, uye nekudaro chiitiko ichi chakauya kune ClickHouse-operator. Mushandi akagamuchira tsananguro iyi. Uye anofanira kuita chimwe chinhu. Kana iyo yekuvandudza yasvika yeClickHouseInstallation chinhu, saka iwe unofanirwa kugadzirisa iyo cluster. Uye basa remushandisi nderekuvandudza sumbu.

Arikuitei? Chekutanga, isu tinofanirwa kugadzira hurongwa hwekuita pane zvatichaita nekuvandudzwa uku. Zvigadziriso zvinogona kunge zvishoma, i.e. diki mukuitwa kweYAML, asi inogona kusanganisira shanduko hombe pane sumbu. Nokudaro, mushandisi anogadzira chirongwa, uye ipapo anonamatira pairi.

Maererano nechirongwa ichi, anotanga kubika chimiro ichi mukati kuitira kuti aite mapodhi, mabasa, i.e. kuita basa rake guru. Iyi ndiyo nzira yekuvaka yeClickHouse cluster muKubernetes.

Zvino ngatibatei chinhu chinofadza chakadaro. Uku kupatsanurwa kwemutoro pakati peKubernetes nemushandisi, i.e. zvinoitwa naKubernetes, izvo mushandisi anoita, uye mabatiro avanoita kune mumwe nemumwe.

Kubernetes inotarisira zvinhu zvehurongwa, i.e. kune yakakosha seti yezvinhu zvinogona kududzirwa sesystem-scope. Kubernetes anoziva nzira yekuvhura mapods, kutangazve midziyo, kukwidza mavhoriyamu, kushanda neConfigMap, i.e. zvese zvinogona kunzi system.

Vashandi vanoshanda mumadomasi. Mumwe nemumwe anoshanda akagadzirirwa nzvimbo yake yezvidzidzo. Isu takazviitira ClickHouse.

Uye mushandisi anopindirana chaizvo maererano nenzvimbo yenyaya, sekuwedzera replica, kugadzira dhayagiramu, kumisikidza kutarisa. Izvi zvinoguma nekuparadzana.

Ngatitarisei muenzaniso unoshanda wekuti kupatsanura uku kwemutoro kunoitika sei kana tichiita kuwedzera replica chiito.

Mushandi anogamuchira basa - kuwedzera replica. Mushandisi anoita sei? Mushandi anoverengera kuti itsva StatefulSet inoda kugadzirwa, umo matemplate akadaro uye akadai, vhoriyamu yekuda, inofanirwa kutsanangurwa.

Akaigadzirira yese ndokuipfuudza kumaK8. Anoti anoda ConfigMap, StatefulSet, Vhoriyamu. Kubernetes ari kushanda. Anogadzira zvikamu zvekutanga zvaanoshanda nazvo.

Uye ipapo ClickHouse-operator inouya mukutamba zvakare. Atova nepodhi yenyama yaanogona kutoita chimwe chinhu. Uye ClickHouse-operator zvakare inoshanda mune domain mazwi. Avo. Kunyanya ClickHouse, kuti ubatanidze replica musumbu, iwe unofanirwa, kutanga, kugadzirisa iyo data schema iripo musumbu iri. Uye, chechipiri, iyi replica inofanirwa kuverengerwa mukutarisisa kuitira kuti inyatso kurondwa. Mushandisi anotogadzirisa izvi.

Uye chete mushure meiyo ClickHouse pachayo inouya mukutamba, i.e. chimwe chikamu chepamusoro. Iyi yatova dhatabhesi. Iyo ine yayo muenzaniso, imwe yakagadziridzwa replica yakagadzirira kujoina sumbu.

Zvinoitika kuti ketani yekuuraya uye kupatsanurwa kwemutoro kana uchiwedzera replica yakareba.

Tinoenderera mberi nemabasa edu anoshanda. Kana iwe uchitova nesumbu, unogona kutamisa gadziriso.

Isu takazviita kuti iwe ugone kunamira mukati me xml iripo, iyo ClickHouse inonzwisisa.

Unogona kunyatsoita ClickHouse. Kungotumirwa kwenzvimbo ndiko kwandakataura nezvazvo pakutsanangura hostPath, kuchengetedza kwenzvimbo. Aya ndiwo maitiro ekuita zoned deployment nenzira kwayo.

Basa rinotevera rinoshanda nderekutarisa.

Kana sumbu redu richichinja, saka isu tinoda nguva nenguva kugadzirisa kutarisa.

Ngatitarisei dhayagiramu. Tatotarisa miseve yakasvibira pano. Zvino ngatitarisei miseve mitsvuku. Aya ndiwo maitiro atinoda kutarisa masumbu edu. Mametrics kubva kuClickHouse cluster anopinda sei muPrometheus, obva apinda muGrafana.

Chii chinonetsa nekutarisa? Neiko ikoku kuchiratidzirwa sorudzi rwakati rwebudiriro? Kuoma kuri mumasimba. Kana isu tiine cluster imwe uye yakamira, saka tinogona kumisikidza yekutarisa kamwe chete uye tisanetseke zvakare.

Asi kana tine masumbu akawanda, kana chimwe chinhu chiri kuramba chichichinja, ipapo maitiro ane simba. Uye kugara uchigadzirisa kutarisa kutambisa zviwanikwa uye nguva, i.e. kunyange husimbe chete. Izvi zvinoda otomatiki. Kuomerwa kuri musimba rekuita. Uye mushandisi anogadzirisa izvi zvakanyanya.

Chikwata chedu chakakura sei? Pakutanga akanga akadaro.

Akabva adai.

Pakupedzisira, akazova seizvi.

Uye kuongorora kunoitwa otomatiki nemushandisi. Nzvimbo imwe chete yekupinda.

Uye tichingobuda tinotarisa kuGrafana dashboard kuti tione kuti hupenyu hweboka redu huri kufashaira sei mukati.

Nenzira, Grafana dashboard zvakare yakagoverwa nemushandi wedu zvakananga mune kodhi kodhi. Unogona kubatanidza uye kushandisa. DevOps yedu yakandipa iyi skrini.

Taizoda kuenda kupi? Izvi:

• Gadzira test automation. Basa guru nderokuongorora otomatiki kweshanduro itsva.
• Isu zvakare tinoda kunyatso kusanganisa kusanganisa neZooKeeper. Uye kune zvirongwa zvekubatanidza neZooKeeper-operator. Avo. Mushandi akanyorerwa ZooKeeper uye zvine musoro kuti vaviri vanoshanda vanotanga kubatanidza kuvaka mhinduro iri nyore.
• Tinoda kuita zviratidzo zvakakosha zvakaoma.
• Ini ndakasimbisa mugirini kuti tave kusvika kunhaka yeMatemplate - ZVAKAITWA, kureva kuti nekuburitswa kunotevera kweanoshanda isu tinenge tatova nenhaka yematemplate. Ichi chishandiso chine simba chinokutendera iwe kuti uvake magadzirirwo akaomarara kubva kuzvidimbu.
• Uye isu tinoda otomatiki yemabasa akaomarara. Iyo huru ndeyeKugovanazve.

Ngatitore zvimwe zvepakati.

Chii chatinowana semhedzisiro? Uye zvakakodzera here kuita kana kuti kwete? Izvo zvinotofanira here kuedza kudhonza dhatabhesi muKubernetes uye kushandisa opareta mune zvese uye iyo Alitnity opareta kunyanya?

Pakubuda tinowana:

• Yakakosha kurerutsa uye otomatiki yekumisikidza, kutumira, uye kugadzirisa.
• Pakarepo yakavakirwa-mukati monitoring.
• Uye yakagadzirira-kushandisa-codified matemplate emamiriro akaoma. Chiito chakafanana nekuwedzera replica hachidi kuitwa nemaoko. Mushandi anoita izvi.

Kwasara mubvunzo mumwe chete wekupedzisira. Isu tatova nedhatabhesi muKubernetes, virtualization. Zvakadini nekuita kwemhinduro yakadai, kunyanya sezvo ClickHouse yakagadziridzwa kuita?

Mhinduro ndeyekuti zvese zvakanaka! Ini handisi kuzoenda mune zvakadzama; iyi ndiyo musoro weimwe rondedzero.

Asi pane chirongwa chakadai seTSBS. Basa rayo guru nderei? Iyi idhizaini yekuita bvunzo. Uku kuedza kuenzanisa kudziya nekudziya, kupfava nekupfava.

Anoshanda sei? Imwe data set inogadzirwa. Ipapo iyi seti yedata inoitiswa pane akasiyana dhatabhesi uchishandisa imwechete seti yebvunzo. Uye yega yega dhatabhesi inogadzirisa dambudziko rimwe nenzira yainoziva sei. Uye ipapo unogona kuenzanisa mhedzisiro.

Inototsigira boka guru rezvinyorwa. Ndakaratidza zvitatu zvikuru. Izvi:

• TimescaleDB.
• InfluxDB.
• ClickHouse.

Kuenzanisa kwakaitwawo neimwe mhinduro yakafanana. Kuenzanisa neRedShift. Kuenzanisa kwakaitwa paAmazon. ClickHouse zvakare iri pamberi pemunhu wese munyaya iyi.

Ndedzipi mhedziso dzinogona kuwanikwa kubva pane zvandakataura?

• DB muKubernetes inogoneka. Pamwe chero zvinogoneka, asi zvakazara zvinoita sekunge zvinogoneka. ClickHouse muKubernetes zvinogoneka nerubatsiro rwemushandisi wedu.
• Mushandi anobatsira otomatiki maitiro uye anonyatsoita kuti hupenyu huve nyore.
• Kuita kwakajairika.
• Uye zvinoratidzika kwatiri kuti izvi zvinogona uye zvinofanira kushandiswa.

Open source - join us!

Sezvandambotaura, mushandisi ndeye yakavhurika sosi chigadzirwa, saka zvingave zvakanaka kwazvo kana huwandu hwepamusoro hwevanhu vakaishandisa. Join us! Takakumirirai mose!

Ndinotenda kune vese!

Mibvunzo yako

Ndatenda nemushumo! Ini ndinonzi Anton. Ini ndinobva kuSEMrush. Ndiri kunetseka kuti chii chiri kuitika nekutema matanda. Isu tinonzwa nezvekutarisa, asi hapana nezve kutema miti, kana tikataura nezve cluster yese. Semuenzaniso, isu takasimudza cluster pane Hardware. Uye isu tinoshandisa matanda epakati, tichiaunganidza kuita murwi wakafanana tichishandisa nzira dzakajairwa. Uye ipapo kubva ipapo tinowana iyo data inotifarira.

Mubvunzo wakanaka, i.e. kupinda mune todo rondedzero. Mushandi wedu haasati aita izvi. Ichiri kusimukira, chirongwa ichi chichiri chidiki. Tinonzwisisa kukosha kwekutema miti. Iyi inyaya yakakoshawo. Uye pamwe haina kukosha kudarika kuongorora. Asi chekutanga pachirongwa chekushandiswa kwaive kutarisa. Pachava nekutema miti. Sezvingatarisirwa, tinoedza kuita otomatiki zvikamu zvese zvehupenyu hwesumbu. Nokudaro, mhinduro ndeyokuti panguva ino mufambisi, zvinosuruvarisa, haazivi kuita izvi, asi zviri muzvirongwa, tichazviita. Kana iwe uchida kujoina, wobva wadhonza chikumbiro, ndapota.

Mhoro! Ndatenda nemushumo! Ndine mubvunzo wakajairwa une chekuita nePersistent Volumes. Kana isu tichigadzira gadziriso neiyi opareta, mushandisi anoona sei pane imwe node yatinayo imwe dhisiki kana folda yakasungirirwa? Tinofanira kutanga kumutsanangurira kuti ndapota isa ClickHouse yedu pane idzi node dzine dhisiki?

Sekunzwisisa kwandinoita, mubvunzo uyu kuenderera kwekuchengetedza kwenzvimbo, kunyanya iyo hostPath chikamu chayo. Izvi zvakafanana nekutsanangurira hurongwa hwese kuti pod inoda kutangwa pane yakadaro uye node, iyo yatine dhisiki yakabatanidzwa mumuviri, iyo inosungirirwa nenzira yakadaro uye yakadaro. Ichi chikamu chese chandakabata pamusoro nekuti mhinduro iripo yakakura.

Muchidimbu zvinoita seizvi. Sezvingatarisirwa, tinofanira kugovera aya mavhoriyamu. Parizvino, hapana gadziriro ine simba mukuchengetedza kwenzvimbo, saka DevOps inofanirwa kucheka madhisiki pachawo, mavhoriyamu aya. Uye ivo vanofanirwa kutsanangura Kubernetes kupa kuti iwe uchave neInoshingirira mavhoriyamu eakadaro uye akadai kirasi, ayo ari pane akadai neakadaro. Ipapo iwe unozofanirwa kutsanangurira Kubernetes kuti mapodhi anoda akadai uye akadai enzvimbo yekuchengetedza kirasi anofanirwa kunangana chete kune akadaro uye akadaro node uchishandisa mavara. Nekuda kwezvinangwa izvi, mushandisi ane kugona kugovera imwe mhando yezita uye imwe chete pamuenzaniso wekugamuchira. Uye zvinozoitika kuti mapodhi achafambiswa neKubernetes kuti amhanye chete pane node dzinosangana nezvinodiwa, zvinyorwa, zviri nyore. Vatungamiri vanopa mavara uye madhisiki ekupa nemaoko. Uye zvino inoyera.

Uye isarudzo yechitatu, yemuno, inobatsira kuita kuti izvi zvive nyore. Sezvandambosimbisa, iri ibasa rinorwadza rekugadzirisa, izvo zvinozobatsira kuwana yakanyanya kuita.

Ndine mubvunzo wechipiri unechekuita neizvi. Kubernetes yakagadzirwa nenzira yekuti hazvina basa kwatiri kana tikarasikirwa node kana kwete. Chii chatinofanira kuita munyaya iyi kana tarasikirwa node apo shard yedu yakaturika?

Hongu, Kubernetes pakutanga akaiswa kuti hukama hwedu nemapods edu hwakafanana nemombe, asi pano nesu dhisiki imwe neimwe inova chimwe chinhu sechipfuwo. Pane dambudziko rekuti hatigoni kungozvirasa. Uye kuvandudzwa kweKubernetes kuri kuenda kune iyo nzira yekuti hazvibviri kuibata zvachose nehuzivi, sekunge iri chinhu chakaraswa zvachose.

Zvino kune mubvunzo unoshanda. Chii chaunofanira kuita kana wakarasa node paive ne diski? Apa dambudziko riri kugadziriswa padanho repamusoro. Panyaya yeClickHouse, isu tine replicas inoshanda padanho repamusoro, i.e. paClickHouse level.

Chii chinoguma nemafungiro? DevOps ine basa rekuona kuti data haina kurasika. Anofanira kumisa kudzokorora nenzira kwayo uye anofanira kuona kuti kudzokorora kuri kushanda. Iyo replica paClickHouse level inofanirwa kunge iine data rakapetwa. Iri harisi dambudziko rinogadziriswa nemushandi. Uye kwete dambudziko iro Kubernetes pacharo rinogadzirisa. Iyi iri paClickHouse level.

Chii chaunofanira kuita kana iron node yako ikadonha? Uye zvinozoitika kuti iwe uchafanirwa kuisa yechipiri, kunyatso govera disc pairi, uye shandisa mavara. Uye mushure meizvozvo, inosangana nezvinodiwa izvo Kubernetes anogona kuvhura muenzaniso pod pairi. Kubernetes achaivhura. Nhamba yako yemapods haina kukwana kuti isangane nenhamba yakataurwa. Ichapfuura nemukutenderera kwandakaratidza. Uye padanho repamusoro, ClickHouse inonzwisisa kuti isu tapinda mukopi, ichiri isina chinhu uye isu tinofanirwa kutanga kuendesa data kwairi. Avo. Iyi nzira haisati yanyatso shanda.

Ndatenda nemushumo! Kana marudzi ese ezvinhu akashata aitika, mushandisi anorovera uye otangazve, uye panguva iyoyo zviitiko zvasvika, unozvibata here izvi?

Chii chinoitika kana mushandisi akadonha uye akatangazve, handiti?

Ehe. Uye panguva iyoyo zviitiko zvakasvika.

Basa rekuita mune iyi nyaya rakagovaniswa pakati peanoshanda naKubernetes. Kubernetes ane kugona kudzokorodza chiitiko chakaitika. Anodzokorora. Uye basa remushandisi ndere kuona kuti kana irogi yechiitiko ichidzokororwa paari, zviitiko izvi hazvina simba. Uye kuitira kuti kudzokororwa kwechiitiko chimwe chete kurege kuputsa hurongwa hwedu. Uye mushandisi wedu anorarama nebasa iri.

Mhoro! Ndatenda nemushumo! Dmitry Zavyalov, kambani Smedova. Pane zvirongwa zvekuwedzera kugona kugadzirisa ne haproxy kune anoshanda? Ndingafarire imwe imwe balancer kunze kweiyo yakajairwa, kuti ive nehungwaru uye inonzwisisa kuti ClickHouse iriko chaizvo.

Uri kutaura nezveIngress here?

Ehe, tsiva Ingress ne haproxy. Mune haproxy unogona kutsanangura iyo topology ye cluster iyo ine replicas.

Hatisati tazvifunga. Kana uchiida uye uchigona kutsanangura kuti sei ichidikanwa, zvino zvinozogoneka kuishandisa, kunyanya kana iwe uchida kutora chikamu. Tichafara kufunga nezvechisarudzo. Mhinduro pfupi ndeye kwete, isu parizvino hatina basa rakadaro. Tinotenda nerubatsiro, tichatarisa nyaya iyi. Uye kana iwe ukatsanangurawo nyaya yekushandisa uye nei ichidikanwa mukuita, semuenzaniso, gadzira nyaya paGitHub, zvino zvichave zvakanaka.

Atove.

Fine. Isu takavhurika kune chero mazano. Uye haproxy inowedzerwa kune todo runyorwa. Rondedzero yetodo iri kukura, haisati ichiderera. Asi izvi zvakanaka, zvinoreva kuti chigadzirwa chiri kudiwa.

Source: www.habr.com