Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Rahoton ya keɓe ga al'amura masu amfani na haɓaka mai aiki a Kubernetes, tsara gine-ginensa da ƙa'idodin aiki na yau da kullun.

A kashi na farko na rahoton za mu yi la'akari da haka:

  • menene ma'aikaci a Kubernetes kuma me yasa ake buƙata;
  • yadda ainihin ma'aikaci ya sauƙaƙa sarrafa tsarin hadaddun;
  • abin da ma'aikaci zai iya kuma ba zai iya yi ba.

Na gaba, bari mu ci gaba zuwa tattauna tsarin ciki na ma'aikacin. Bari mu dubi gine-gine da kuma aiki na ma'aikaci mataki-mataki. Bari mu duba dalla-dalla:

  • hulɗa tsakanin mai aiki da Kubernetes;
  • menene ayyuka da ma'aikacin ke ɗauka da kuma waɗanne ayyuka yake wakilta zuwa Kubernetes.

Bari mu kalli sarrafa shards da kwafin bayanai a cikin Kubernetes.
Na gaba, za mu tattauna batutuwan ajiyar bayanai:

  • yadda za a yi aiki tare da Ma'ajiya na dindindin daga ra'ayi na ma'aikaci;
  • Matsalolin amfani da Ma'ajiyar Gida.

A ɓangaren ƙarshe na rahoton, za mu yi la'akari da misalai masu amfani na aikace-aikace clickhouse-operator tare da Amazon ko Google Cloud Service. Rahoton ya dogara ne akan misalin haɓakawa da ƙwarewar aiki na ma'aikaci don ClickHouse.

Video:

Sunana Vladislav Klimenko. A yau ina so in yi magana game da ƙwarewarmu wajen haɓakawa da sarrafa ma'aikaci, kuma wannan ƙwararren mai aiki ne don sarrafa tarin bayanai. Misali ClickHouse-operator don sarrafa gungun ClickHouse.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Me yasa muke da damar yin magana game da mai aiki da ClickHouse?

  • Muna tallafawa da haɓaka ClickHouse.
  • A halin yanzu, muna ƙoƙarin ba da gudummawa a hankali don ci gaban ClickHouse. Kuma mun kasance na biyu bayan Yandex dangane da girman canje-canjen da aka yi zuwa ClickHouse.
  • Muna ƙoƙarin yin ƙarin ayyuka don yanayin muhalli na ClickHouse.

Ina so in gaya muku ɗaya daga cikin waɗannan ayyukan. Wannan game da ClickHouse-operator don Kubernetes ne.

A cikin rahotona ina so in tabo batutuwa guda biyu:

  • Maudu'in farko shine yadda ma'aikacin sarrafa bayanai na ClickHouse ke aiki a Kubernetes.
  • Maudu'i na biyu shine yadda kowane ma'aikaci ke aiki, watau yadda yake mu'amala da Kubernetes.

Koyaya, waɗannan tambayoyin biyu za su haɗu a cikin rahoton nawa.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Wanene zai yi sha'awar sauraron abin da nake ƙoƙarin faɗa?

  • Zai fi sha'awa ga waɗanda ke aiki da ma'aikata.
  • Ko kuma ga waɗanda suke so su yi nasu don fahimtar yadda yake aiki a ciki, yadda ma'aikacin ke hulɗa tare da Kubernetes, da kuma waɗanne matsaloli na iya bayyana.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Don mafi kyawun fahimtar abin da za mu tattauna a yau, yana da kyau a san yadda Kubernetes ke aiki da samun horo na girgije.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Menene ClickHouse? Wannan ma'ajin bayanai ne na columnar tare da takamaiman fasali don sarrafa tambayoyin nazari akan layi. Kuma shi ne gaba daya bude tushen.

Kuma yana da mahimmanci a gare mu mu san abubuwa biyu kawai. Kuna buƙatar sanin cewa wannan ma'adanin bayanai ne, don haka abin da zan gaya muku zai yi amfani da kusan kowace rumbun adana bayanai. Kuma gaskiyar cewa ClickHouse DBMS yana da ma'auni sosai, yana ba da ma'auni na kusan madaidaiciya. Sabili da haka, jihar tari yanayi ne na halitta don ClickHouse. Kuma mun fi sha'awar tattauna yadda ake hidimar gungu na ClickHouse a Kubernetes.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Me yasa ake bukata a can? Me ya sa ba za mu iya ci gaba da sarrafa shi da kanmu ba? Kuma amsoshi wani bangare ne na fasaha da wani bangare na kungiyance.

  • A aikace, muna ƙara fuskantar halin da ake ciki inda a cikin manyan kamfanoni kusan dukkanin abubuwan da aka gyara sun riga sun kasance a Kubernetes. Rukunin bayanai sun kasance a waje.
  • Kuma ana ƙara yin tambayar: "Za a iya sanya wannan a ciki?" Don haka, manyan kamfanoni suna ƙoƙarin cimma iyakar haɗin kai na gudanarwa don su sami damar sarrafa ma'ajiyar bayanan su cikin sauri.
  • Kuma wannan yana taimakawa musamman idan kuna buƙatar mafi girman damar don maimaita abu iri ɗaya a cikin sabon wuri, watau matsakaicin ɗaukar hoto.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Yaya sauki ko wahala? Wannan, ba shakka, ana iya yin shi da hannu. Amma ba haka ba ne mai sauƙi, saboda muna da ƙarin rikitarwa na sarrafa Kubernetes kanta, amma a lokaci guda ƙayyadaddun ƙayyadaddun ClickHouse suna da yawa. Kuma irin wannan sakamakon tarawa.

Kuma duk tare wannan yana ba da babbar fasahar fasaha, wanda ya zama da wahala a sarrafa shi, saboda Kubernetes yana kawo nasa al'amuran yau da kullun don aiki, kuma ClickHouse yana kawo nasa al'amurran yau da kullun. Musamman idan muna da ClickHouses da yawa, kuma muna buƙatar yin wani abu koyaushe tare da su.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Tare da tsayayyen tsari, ClickHouse yana da adadi mai yawa na batutuwa waɗanda ke haifar da kullun nauyi akan DevOps:

  • Lokacin da muke son canza wani abu a cikin ClickHouse, alal misali, ƙara kwafi ko shard, to muna buƙatar sarrafa tsarin.
  • Sannan canza tsarin bayanan, saboda ClickHouse yana da takamaiman hanyar sharding. A can kuna buƙatar tsara zane-zanen bayanai, tsara tsarin daidaitawa.
  • Kuna buƙatar saita sa ido.
  • Tattara rajistan ayyukan don sabbin shards, don sabbin kwafi.
  • Kula da maidowa.
  • Kuma zata sake farawa.

Waɗannan ayyuka ne na yau da kullun waɗanda da gaske nake so in sauƙaƙe amfani da su.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kubernetes kanta yana taimakawa sosai a cikin aiki, amma akan abubuwan tsarin asali.

Kubernetes yana da kyau a sauƙaƙe da sarrafa abubuwa kamar:

  • Farfadowa da na'ura.
  • Sake kunnawa
  • Gudanar da tsarin ajiya.

Wannan yana da kyau, wannan shine madaidaicin alkibla, amma gaba ɗaya bai san yadda ake tafiyar da gungun bayanai ba.

Muna son ƙarin, muna son duk bayanan su yi aiki a Kubernetes.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ina so in sami wani abu kamar babban maɓallin jan sihiri ɗaya wanda kuke latsawa kuma gungu tare da ayyukan yau da kullun waɗanda ke buƙatar warwarewa ana tura su kuma ana kiyaye su gabaɗayan rayuwarta. ClickHouse cluster a cikin Kubernetes.

Kuma mun yi kokarin samar da mafita da za ta taimaka wajen saukaka aikin. Wannan ma'aikacin ClickHouse ne don Kubernetes daga Altinity.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Operator shiri ne wanda babban aikinsa shi ne sarrafa wasu shirye-shirye, watau Manager.

Kuma ya ƙunshi sifofi na ɗabi'a. Kuna iya kiran wannan ingantaccen ilimin game da yankin batun.

Kuma babban aikinsa shi ne ya sauƙaƙa rayuwar DevOps tare da rage yawan sarrafa micromanagement, ta yadda shi (DevOps) ya riga ya yi tunani a cikin matakai masu girma, watau, don kada ya shiga micromanagement, don kada ya daidaita. duk cikakkun bayanai da hannu.

Kuma kawai ma'aikacin shine mataimaki na mutum-mutumi wanda ke hulɗa da microtasks kuma yana taimakawa DevOps.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Me yasa kuke buƙatar afareta? Yana aiki da kyau a cikin nau'i biyu:

  • Lokacin da ƙwararren da ke hulɗa da ClickHouse ba shi da isasshen ƙwarewa, amma ya riga ya buƙaci yin aiki da ClickHouse, mai aiki yana sauƙaƙe aikin kuma yana ba ku damar yin aiki da gungu na ClickHouse tare da tsari mai rikitarwa, ba tare da yin cikakken bayani game da yadda duk yake aiki ba. ciki. Kuna kawai ba shi ayyuka masu girma, kuma yana aiki.
  • Kuma aiki na biyu da ya fi dacewa a cikinsa shine lokacin da ya zama dole don sarrafa manyan ayyuka na yau da kullun. Yana cire microtasks daga masu gudanar da tsarin.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Wannan ya fi buƙata ko dai ta waɗanda ke fara tafiya, ko kuma waɗanda ke buƙatar yin aiki da yawa.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ta yaya tsarin tushen aiki ya bambanta da sauran tsarin? Akwai Helm. Hakanan yana taimakawa shigar da ClickHouse; zaku iya zana taswirar helm, wanda har ma zai shigar da gungu na ClickHouse gaba daya. Mene ne bambanci tsakanin mai aiki da guda ɗaya, misali, Helm?

Babban bambanci na asali shine Helm shine sarrafa kunshin kuma Mai aiki ya ci gaba da gaba. Wannan tallafi ne ga dukan tsarin rayuwa. Wannan ba kawai shigarwa ba ne, waɗannan ayyuka ne na yau da kullum waɗanda suka haɗa da sikelin, sharding, i.e. duk abin da ya kamata a yi a lokacin zagayowar rayuwa (idan ya cancanta, sannan kuma sharewa) - duk abin da mai aiki ya yanke shawarar. Yana ƙoƙarin sarrafa sarrafa kansa da kula da duk tsawon rayuwar software. Wannan shi ne ainihin bambancinsa da sauran hanyoyin da aka gabatar.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Wannan shi ne bangaren gabatarwa, mu ci gaba.

Ta yaya za mu gina ma'aikacin mu? Muna ƙoƙarin tunkarar batun don sarrafa gungu na ClickHouse a matsayin hanya ɗaya.

Anan muna da bayanan shigarwa a gefen hagu na hoton. Wannan YAML ne tare da ƙayyadaddun tari, wanda aka wuce zuwa Kubernetes ta hanyar gargajiya ta kubectl. Can ma’aikacin mu ya dauko shi ya yi sihirinsa. Kuma a fitarwa muna samun makirci mai zuwa. Wannan aiwatarwa ne na ClickHouse a cikin Kubernetes.

Sannan a hankali za mu kalli yadda ainihin ma'aikacin ke aiki, waɗanne ayyuka na yau da kullun za a iya warware su. Za mu yi la'akari da ayyuka na yau da kullun saboda muna da iyakacin lokaci. Kuma ba duk abin da mai aiki zai iya yanke shawara ba ne za a tattauna.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari mu fara daga aiki. Aikinmu gaba daya bude tushe ne, saboda haka zaku iya ganin yadda yake aiki akan GitHub. Kuma zaku iya ci gaba daga la'akari da cewa idan kuna son ƙaddamar da shi kawai, to zaku iya farawa tare da Jagoran Farawa Mai Sauri.

Idan kuna son fahimta daki-daki, to muna ƙoƙarin kiyaye takaddun a cikin tsari mai kyau ko žasa.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari mu fara da matsala mai amfani. Aiki na farko, inda duk muke so mu fara, shine gudanar da misali na farko ko ta yaya. Ta yaya zan iya ƙaddamar da ClickHouse ta amfani da mai aiki, koda kuwa ban san ainihin yadda yake aiki ba? Muna rubuta takarda, saboda ... Duk sadarwa tare da k8s sadarwa ne ta hanyar bayyanawa.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Wannan shi ne irin wannan hadadden bayani. Abin da muka haskaka a ja shine abin da ya kamata mu mai da hankali akai. Muna tambayar ma'aikacin ya ƙirƙiri gungu mai suna demo.

Waɗannan misalai ne na asali a yanzu. Har yanzu ba a bayyana ma'ajiya ba, amma za mu koma wurin ajiya kaɗan nan gaba. A yanzu, za mu lura da yanayin ci gaban tari.

Mun ƙirƙiri wannan ma'anar. Muna ciyar da shi ga ma'aikacin mu. Ya yi aiki, ya yi sihiri.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Muna kallon wasan bidiyo. Abubuwa uku suna da ban sha'awa: Pod, Sabis guda biyu, da StatefulSet.

Mai aiki ya yi aiki, kuma muna iya ganin ainihin abin da ya halitta.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ya halitta wani abu kamar wannan. Muna da StatefulSet, Pod, ConfigMap don kowane kwafi, ConfigMap ga duka tari. Ana buƙatar sabis azaman wuraren shiga cikin tari.

Sabis ɗin su ne Sabis na Ma'auni na Load na tsakiya kuma ana iya amfani da su don kowane kwafi, ga kowane shard.

Tarin mu na asali yayi kama da wannan. Yana daga kumburi guda ɗaya.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Mu ci gaba da dagula abubuwa. Muna buƙatar share gungu.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ayyukanmu suna girma, haɓakawa sun fara. Muna so mu ƙara shard. Muna bin ci gaban. Muna canza ƙayyadaddun ƙayyadaddun mu. Mun nuna cewa muna son shards biyu.

Wannan fayil iri ɗaya ne wanda ke haɓakawa tare da haɓakar tsarin. Adana a'a, za a ƙara yin magana akan ajiya, wannan batu ne daban.

Muna ciyar da ma'aikacin YAML kuma mu ga abin da ya faru.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ma'aikacin yayi tunani kuma ya sanya abubuwa masu zuwa. Mun riga muna da Pods guda biyu, Sabis uku da, ba zato ba tsammani, 2 StatefulSets. Me yasa 2 StatefulSets?

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

A kan zane ya kasance kamar haka - wannan shine yanayin mu na farko, lokacin da muke da kwasfa daya.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ya zama haka. Ya zuwa yanzu komai yana da sauki, an kwafi shi.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma me yasa aka zama StatefulSets guda biyu? Anan muna buƙatar ƙaddamarwa kuma mu tattauna tambayar yadda ake sarrafa Pods a Kubernetes.

Akwai wani abu da ake kira StatefulSet wanda ke ba ka damar ƙirƙirar saitin Pods daga samfuri. Maɓalli mai mahimmanci anan shine Samfura. Kuma zaku iya ƙaddamar da Pods da yawa ta amfani da samfuri ɗaya a cikin StatefulSet ɗaya. Kuma mabuɗin kalmar anan shine "yawan Pods don samfuri ɗaya."

Kuma akwai babban jaraba don yin gabaɗayan tari, tare da tattara shi cikin StatefulSet ɗaya. Zai yi aiki, babu matsala tare da shi. Amma akwai gargadi guda ɗaya. Idan muna son tara gungu iri-iri, watau, daga nau'ikan ClickHouse da yawa, to tambayoyinmu sun fara. Ee, StatefulSet na iya yin sabuntawar birgima, kuma a can za ku iya fitar da sabon sigar, bayyana cewa kuna buƙatar gwada nodes da yawa a lokaci guda.

Amma idan muka fitar da aikin kuma muka ce muna son yin gungu mai ban sha'awa gaba ɗaya kuma ba ma son canzawa daga tsohuwar sigar zuwa sabon ta amfani da sabuntawar birgima, amma kawai muna son ƙirƙirar tari iri-iri duka biyu cikin sharuddan. na daban-daban iri na ClickHouse da kuma cikin sharuddan ajiya daban-daban. Muna so, alal misali, yin wasu kwafi akan faifai daban-daban, akan masu jinkirin, gabaɗaya, don gina gungu iri-iri. Kuma saboda gaskiyar cewa StatefulSet yana yin daidaitaccen bayani daga samfuri ɗaya, babu wata hanyar yin wannan.

Bayan wani tunani, sai aka yanke shawarar cewa za mu yi haka. Muna da kowane kwafi a cikin nasa StatefulSet. Akwai wasu kura-kurai ga wannan maganin, amma a aikace duk mai aiki ne ya tattara shi gaba ɗaya. Kuma akwai fa'idodi da yawa. Za mu iya gina ainihin gungu da muke so, alal misali, cikakken nau'in nau'i. Saboda haka, a cikin gungu wanda muke da shards guda biyu tare da kwafi ɗaya, za mu sami 2 StatefulSets da 2 Pods daidai saboda mun zaɓi wannan hanyar saboda dalilan da aka ambata a sama don samun damar gina gungu iri-iri.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Mu koma ga matsalolin aiki. A cikin gungun mu muna buƙatar daidaita masu amfani, watau. kuna buƙatar yin wasu sanyi na ClickHouse a cikin Kubernetes. Mai aiki yana ba da duk damar yin hakan.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Za mu iya rubuta abin da muke so kai tsaye a cikin YAML. Ana tsara duk zaɓuɓɓukan daidaitawa kai tsaye daga wannan YAML zuwa cikin saitunan ClickHouse, waɗanda kuma ana rarraba su cikin gungu.

Kuna iya rubuta shi kamar haka. Wannan misali ne. Ana iya rufaffen kalmar sirri. Lallai ana tallafawa duk zaɓuɓɓukan sanyi na ClickHouse. Ga misali kawai.

Ana rarraba saitin tari azaman ConfigMap. A aikace, sabuntawar ConfigMap baya faruwa nan take, don haka idan tarin yana da girma, to tsarin tura tsarin yana ɗaukar ɗan lokaci. Amma duk wannan ya dace sosai don amfani.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari mu dagula aikin. Tarin yana tasowa. Muna son yin kwafin bayanai. Wato mun riga mun sami shards guda biyu, kwafi ɗaya kowanne, kuma an daidaita masu amfani. Muna girma kuma muna son yin kwafi.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Me muke bukata don kwafi?

Muna buƙatar ZooKeeper. A cikin ClickHouse, ana gina kwafi ta amfani da ZooKeeper. Ana buƙatar ZooKeeper don kwafin ClickHouse daban-daban su sami yarjejeniya game da waɗanne tubalan bayanai ke kan abin ClickHouse.

ZooKeeper na iya amfani da kowa. Idan kamfani yana da ZooKeeper na waje, to ana iya amfani dashi. Idan ba haka ba, zaku iya shigar da shi daga ma'ajiyar mu. Akwai mai sakawa wanda ke sauƙaƙa wannan duka.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma zane-zanen hulɗar tsarin gaba ɗaya ya juya kamar haka. Muna da Kubernetes azaman dandamali. Yana aiwatar da ma'aikacin ClickHouse. Na dauki hoton ZooKeeper anan. Kuma mai aiki yana hulɗa tare da ClickHouse da ZooKeeper. Wato sakamakon mu'amala.

Kuma duk wannan ya zama dole don ClickHouse don samun nasarar kwafin bayanai a cikin k8s.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari yanzu mu kalli aikin da kansa, yadda bayyanar da kwafi zai yi kama.

Muna ƙara sassa biyu zuwa bayanin mu. Na farko shine inda za'a sami ZooKeeper, wanda zai iya kasancewa ko dai cikin Kubernetes ko na waje. Wannan siffa ce kawai. Kuma muna yin odar kwafi. Wadancan. muna son kwafi biyu. Gabaɗaya, yakamata mu sami kwasfa 4 a fitarwa. Mun tuna game da ajiya, zai dawo kadan daga baya. Adana labari ne daban.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Haka ya kasance.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ya zama haka. Ana ƙara kwafi. Na 4th bai dace ba, mun yi imanin cewa za a iya samun yawancin su a can. Kuma ana ƙara ZooKeeper zuwa gefe. Shirye-shiryen suna zama masu rikitarwa.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma lokaci yayi da za a ƙara aiki na gaba. Za mu ƙara ma'ajiya na dindindin.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)Don Ma'ajiyar Dagewa muna da zaɓuɓɓuka daban-daban.

Idan muna gudana a cikin mai ba da girgije, alal misali, ta amfani da Amazon, Google, to akwai babban jaraba don amfani da ajiyar girgije. Yana da matukar dacewa, yana da kyau.

Kuma akwai zaɓi na biyu. Wannan don ajiyar gida ne, lokacin da muke da fayafai na gida akan kowane kumburi. Wannan zaɓin ya fi wahalar aiwatarwa, amma a lokaci guda yana da amfani.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari mu ga abin da muke da shi game da ajiyar girgije.

Akwai fa'idodi. Yana da sauƙin daidaitawa. Muna yin oda kawai daga mai samar da gajimare wanda da fatan za a ba mu ajiyar irin wannan ƙarfin, irin wannan da irin wannan aji. Ana tsara azuzuwan ta masu samarwa da kansu.

Kuma akwai koma baya. Ga wasu, wannan baya da mahimmanci. Tabbas, za a sami wasu batutuwan aiki. Yana da matukar dacewa don amfani kuma abin dogara, amma akwai wasu yuwuwar rashin aiki.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma saboda ClickHouse yana mai da hankali musamman kan yawan aiki, mutum na iya cewa yana matse duk abin da zai iya, wanda shine dalilin da ya sa yawancin abokan ciniki ke ƙoƙarin fitar da mafi girman yawan aiki.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma don samun fa'ida daga gare ta, muna buƙatar ma'ajiyar gida.

Kubernetes yana ba da abstractions guda uku don amfani da ma'ajiyar gida a Kubernetes. Wannan:

  • EmptyDir
  • HostPath.
  • Na gida

Bari mu dubi yadda suka bambanta da kuma yadda suke kama.

Da fari dai, a cikin dukkan hanyoyin guda uku muna da ajiya - waɗannan fayafai ne na gida waɗanda ke kan kullin k8s na zahiri ɗaya. Amma suna da wasu bambance-bambance.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari mu fara da mafi sauƙi, watau emptyDir. Menene wannan a aikace? A cikin ƙayyadaddun mu, muna tambayar tsarin kwantena (mafi yawancin Docker) don ba mu damar shiga babban fayil akan faifan gida.

A aikace, Docker yana ƙirƙirar babban fayil na wucin gadi a wani wuri tare da nasa hanyoyin kuma ya kira shi dogon zanta. Kuma yana ba da hanyar sadarwa don samun dama gare shi.

Ta yaya wannan zai yi aiki da hikima? Wannan zai yi aiki a saurin faifai na gida, watau. Wannan shine cikakken damar zuwa dunƙule ku.

Amma wannan harka yana da nasa drawback. Dagewa yana da shakka a cikin wannan al'amari. A karon farko Docker yana motsawa tare da kwantena, Mai dagewa ya ɓace. Idan Kubernetes yana so ya motsa wannan Pod zuwa wani faifai don wasu dalilai, bayanan za su ɓace.

Wannan tsarin yana da kyau ga gwaje-gwaje, saboda ya riga ya nuna saurin al'ada, amma ga wani abu mai tsanani wannan zaɓi bai dace ba.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Don haka akwai hanya ta biyu. Wannan shine hostPath. Idan ka kalli faifan da ya gabata da kuma wannan, za ka iya ganin bambanci ɗaya kawai. Babban fayil ɗinmu ya matsa daga Docker kai tsaye zuwa kumburin Kubernetes. Yana da ɗan sauki a nan. Muna ƙayyade hanyar kai tsaye akan tsarin fayil ɗin gida inda muke son adana bayanan mu.

Wannan hanya tana da amfani. Wannan ya riga ya zama na gaske na dindindin, kuma na al'ada a wancan. Za mu sami bayanan da aka rubuta akan faifai a wasu adireshi.

Akwai kuma rashin amfani. Wannan shi ne sarkar gudanarwa. Kubernetes ɗin mu na iya son matsar da Pod zuwa wani kumburin jiki. Kuma wannan shine inda DevOps ya shigo cikin wasa. Dole ne ya bayyana daidai ga tsarin gabaɗayan cewa waɗannan kwas ɗin za a iya matsar da su zuwa ƙofofin da kuke da wani abu da aka ɗora tare da waɗannan hanyoyin, kuma ba fiye da kumburi ɗaya a lokaci ɗaya ba. Yana da matukar wahala.

Musamman don waɗannan dalilai, mun yi samfura a cikin ma'aikacin mu don ɓoye duk wannan rikitarwa. Kuma zaka iya cewa kawai: "Ina so in sami misalin ClickHouse guda ɗaya don kowane kumburi na jiki da irin wannan kuma irin wannan hanya."

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Amma ba mu kaɗai ne muke buƙatar wannan buƙata ba, don haka ma'aikatan Kubernetes su ma sun fahimci cewa mutane suna son samun damar yin amfani da faifai na zahiri, don haka suna samar da Layer na uku.

Ana kiran shi gida. A zahiri babu bambanci daga zamewar da ta gabata. Sai kawai kafin ya zama dole don tabbatar da cewa ba za mu iya canja wurin waɗannan kwasfan fayiloli daga kumburi zuwa kumburi ba, saboda dole ne a haɗa su tare da wasu hanyoyi zuwa faifan jiki na gida, amma yanzu duk wannan ilimin yana cikin Kubernetes kanta. Kuma ya juya ya zama mafi sauƙi don daidaitawa.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Mu koma kan matsalarmu ta aikace. Mu koma ga tsarin YAML. Anan muna da ajiyar gaske. Mun dawo gare shi. Mun saita samfurin ƙira mai ƙima kamar a cikin k8s. Kuma mun bayyana irin nau'in ajiyar da muke so.

Bayan wannan, k8s zai buƙaci ajiya. Zai raba mana shi a cikin StatefulSet. Kuma a ƙarshe zai kasance a zubar da ClickHouse.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Muna da wannan tsarin. Ma'ajiyar mu na dagewa yayi ja, wanda da alama yana nuna cewa ana buƙatar yin hakan.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma ya zama kore. Yanzu ClickHouse akan tsarin gungu na k8s an kammala shi gaba daya. Muna da shards, kwafi, ZooKeeper, muna da na gaske na dindindin, wanda aka aiwatar ta hanya ɗaya ko wata. Tsarin ya riga ya fara aiki sosai.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Mu ci gaba da rayuwa. Tarin mu yana tasowa. Kuma Alexey yayi ƙoƙari, kuma ya fitar da sabon sigar ClickHouse.

Wani aiki mai amfani ya taso - don gwada sabon sigar ClickHouse akan tarin mu. Kuma, a zahiri, ba kwa son mirgine shi duka; kuna son sanya sabon sigar a cikin kwafi ɗaya a wani wuri a cikin kusurwa mai nisa, kuma wataƙila ba sabon sigar ɗaya bane, amma biyu a lokaci ɗaya, saboda suna fitowa sau da yawa.

Me za mu iya cewa game da wannan?

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

A nan muna da irin wannan damar. Waɗannan samfuran kwas ɗin ne. Kuna iya rubuta cewa ma'aikacin mu gaba ɗaya yana ba ku damar gina gungu iri-iri. Wadancan. saita, farawa daga duk kwafi a cikin bunch, yana ƙarewa tare da kowane kwafi na sirri, wanda sigar da muke son ClickHouse, wane nau'in da muke son ajiya. Za mu iya daidaita tari tare da tsarin da muke buƙata.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Mu dan zurfafa a ciki. Kafin wannan, mun yi magana game da yadda ClickHouse-operator ke aiki dangane da ƙayyadaddun ClickHouse.

Yanzu zan so in faɗi 'yan kalmomi game da yadda kowane ma'aikaci ke aiki gaba ɗaya, da kuma yadda yake mu'amala da K8s.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari mu fara fara yin hulɗa tare da K8s. Me zai faru idan muka yi amfani da kubectl? Abubuwanmu suna bayyana a etcd ta API.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Misali, ainihin abubuwan Kubernetes: kwafsa, StatefulSet, sabis, da sauransu akan lissafin.

A lokaci guda, babu wani abu na zahiri da ke faruwa tukuna. Dole ne waɗannan abubuwa su kasance a cikin tari.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Don wannan dalili, mai sarrafawa yana bayyana. Mai sarrafawa wani yanki ne na k8s na musamman wanda zai iya samar da waɗannan kwatancen. Ya san yadda da abin da zai yi a jiki. Ya san yadda ake gudanar da kwantena, abin da ake buƙatar saita shi a can domin uwar garke ta yi aiki.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma yana fitar da abubuwan mu a cikin K8s.

Amma muna son yin aiki ba kawai tare da pods da StatefulSets ba, muna son ƙirƙirar ClickHouseInstallation, watau wani abu na nau'in ClickHouse, don yin aiki da shi gaba ɗaya. Ya zuwa yanzu babu yiwuwar haka.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Amma K8s yana da abubuwa masu kyau masu zuwa. Muna son mu sami wani wuri kamar wannan hadadden mahallin inda za a haɗa tarin mu daga kwas ɗin da StatefulSet.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma me ya kamata a yi don wannan? Na farko, Ma'anar Albarkatun Al'ada ta shigo cikin hoton. Menene shi? Wannan siffa ce ta K8s, cewa za ku sami ƙarin nau'in bayanai guda ɗaya, cewa muna son ƙara kayan aiki na al'ada a cikin kwafsa, StatefulSet, wanda zai kasance mai rikitarwa a ciki. Wannan shi ne bayanin tsarin bayanai.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Muna kuma aika shi can ta hanyar kubectl apply. Kubernetes da farin ciki ya dauka.

Kuma yanzu a cikin ajiyarmu, abin da ke cikin etcd yana da damar yin rikodin kayan aiki na al'ada da ake kira ClickHouseInstallation.

Amma a yanzu babu abin da zai kara faruwa. Wato idan yanzu mun ƙirƙiri fayil ɗin YAML wanda muka duba yana kwatanta shards da replicas kuma mu ce “kubectl apply,” to Kubernetes zai karɓa, ya saka shi cikin etcd ya ce: “Mai girma, amma ban san abin da zan yi ba. da shi. Ban san yadda ake kula da ClickHouseInstallation ba."

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Saboda haka, muna buƙatar wanda zai taimaka Kubernetes yayi hidimar sabon nau'in bayanai. A gefen hagu muna da mai sarrafa Kubernetes na asali wanda ke aiki tare da nau'ikan bayanan asali. Kuma a dama ya kamata mu sami mai kula da al'ada wanda zai iya aiki tare da nau'in bayanan al'ada.

Kuma a wata hanya ana kiran shi mai aiki. Na haɗa shi musamman a nan kamar Kubernetes, saboda kuma ana iya kashe shi a wajen K8s. Mafi sau da yawa, ba shakka, an kashe duk masu aiki a Kubernetes, amma babu abin da ya hana shi tsayawa a waje, don haka a nan an motsa shi musamman a waje.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma bi da bi, mai sarrafa al'ada, wanda kuma aka sani da mai aiki, yana hulɗa tare da Kubernetes ta API. Ya riga ya san yadda ake hulɗa da API. Kuma ya riga ya san yadda za a yi hadadden da'ira da muke so mu yi daga kayan aiki na musamman. Wannan shi ne ainihin abin da ma'aikacin ke yi.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ta yaya ma'aikacin ke aiki? Mu kalli bangaren dama mu ga yadda yake yi. Bari mu gano yadda ma'aikacin ke yin duk wannan da yadda ƙarin hulɗa tare da K8s ke faruwa.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Mai aiki shiri ne. Ta kasance mai tsarin taron. Mai aiki yana biyan kuɗi zuwa abubuwan da suka faru ta amfani da Kubernetes API. API ɗin Kubernetes yana da wuraren shigarwa inda zaku iya biyan kuɗi zuwa abubuwan da suka faru. Kuma idan wani abu ya canza a cikin K8s, to Kubernetes yana aika abubuwan ga kowa da kowa, watau. duk wanda ya yi rajista zuwa wannan batu na API zai sami sanarwa.

Mai aiki yana biyan kuɗi zuwa abubuwan da suka faru kuma dole ne ya yi wani irin martani. Ayyukansa shine mayar da martani ga abubuwan da suka faru.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ana haifar da abubuwan ta wasu sabuntawa. Fayil ɗin mu na YAML tare da bayanin ClickHouseInstallation ya isa. Ya je zuwa etcd ta kubectl apply. An haifar da wani taron a can, kuma a sakamakon haka wannan taron ya zo wurin ClickHouse-operator. Ma'aikacin ya sami wannan bayanin. Kuma dole ne ya yi wani abu. Idan sabuntawa ya zo don abin ClickHouseInstallation, to kuna buƙatar sabunta tarin. Kuma aikin ma'aikaci shine sabunta tari.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Me yake yi? Da farko, muna buƙatar tsara tsarin aiki don abin da za mu yi da wannan sabuntawar. Sabuntawa na iya zama ƙanana sosai, watau. ƙanana a cikin aiwatar da YAML, amma yana iya haifar da manyan canje-canje akan gungu. Saboda haka, ma'aikacin ya ƙirƙiri tsari, sa'an nan kuma ya manne da shi.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bisa ga wannan tsari, ya fara dafa wannan tsari a ciki don ya zama nau'i, ayyuka, watau. yi abin da babban aikinsa yake. Wannan shine yadda ake gina gungu na ClickHouse a cikin Kubernetes.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Yanzu bari mu taɓa irin wannan abu mai ban sha'awa. Wannan rarrabuwar alhaki ne tsakanin Kubernetes da mai aiki, watau. abin da Kubernetes ke yi, abin da ma'aikacin ke yi, da yadda suke hulɗa da juna.

Kubernetes yana da alhakin abubuwan tsarin, watau. don ainihin saitin abubuwa waɗanda za a iya fassara su azaman tsarin-tsari. Kubernetes ya san yadda ake ƙaddamar da kwasfan fayiloli, yadda ake sake kunna kwantena, yadda ake hawan girma, yadda ake aiki da ConfigMap, i.e. duk abin da za a iya kira tsarin.

Masu aiki suna aiki a cikin yankuna. An yi kowane ma'aikaci don yankin abin da ya shafi kansa. Mun yi shi don ClickHouse.

Kuma ma'aikaci yana hulɗa daidai da yanayin yanki, kamar ƙara kwafi, yin zane, saita sa ido. Wannan yana haifar da rarrabuwa.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari mu kalli misali mai amfani na yadda wannan rabon alhakin ke faruwa lokacin da muka yi ƙara aikin kwafi.

Mai aiki yana karɓar ɗawainiya - don ƙara kwafi. Menene ma'aikacin ke yi? Mai aiki zai lissafta cewa sabon StatefulSet yana buƙatar ƙirƙirar, wanda a ciki dole ne a bayyana irin waɗannan samfuran samfuran, da'awar ƙara.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ya shirya duka ya mika shi ga K8s. Ya ce yana buƙatar ConfigMap, StatefulSet, Volume. Kubernetes yana aiki. Yana samar da ainihin raka'o'in da yake aiki da su.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Sannan ClickHouse-operator ya sake shiga wasa. Ya riga yana da kwandon jiki wanda zai iya yin wani abu a kai. Kuma ClickHouse-operator yana sake aiki cikin sharuddan yanki. Wadancan. Musamman ClickHouse, don haɗa kwafi a cikin gungu, dole ne, da farko, saita tsarin bayanan da ke cikin wannan gungu. Kuma, na biyu, wannan kwafin dole ne a saka shi a cikin sa ido don a iya gano shi a fili. Mai aiki ya riga ya tsara wannan.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma sai bayan wannan ClickHouse kanta ta shigo cikin wasa, watau. wani babban matakin mahaluži. Wannan tuni ma'adanin bayanai ne. Yana da nasa misali, wani kwafi da aka tsara wanda ke shirye don shiga tari.

Ya bayyana cewa jerin aiwatarwa da rabon alhaki lokacin ƙara kwafi yana da tsayi sosai.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Muna ci gaba da ayyukanmu masu amfani. Idan kun riga kuna da tari, zaku iya ƙaura tsarin.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Mun sanya shi don ku iya liƙa kai tsaye cikin xml data kasance, wanda ClickHouse ya fahimta.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuna iya daidaita ClickHouse. Aiwatar da yankin kawai shine abin da na yi magana game da shi lokacin da ke bayanin hostPath, ajiyar gida. Wannan shine yadda za'a yi jigilar zoned daidai.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Aiki na gaba mai amfani shine sa ido.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Idan tarin mu ya canza, to muna buƙatar saita sa ido lokaci-lokaci.

Bari mu dubi zane. Mun riga mun kalli korayen kiban nan. Yanzu bari mu kalli jajayen kiban. Wannan shine yadda muke so mu sanya ido kan gungun mu. Yadda ma'auni daga gunkin ClickHouse ke shiga cikin Prometheus, sannan zuwa Grafana.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Menene wahalar sa ido? Me yasa aka gabatar da wannan a matsayin wani nau'in nasara? Wahalar ta ta'allaka ne a cikin motsi. Lokacin da muke da gungu ɗaya kuma yana tsaye, za mu iya saita sa ido sau ɗaya kuma ba za mu ƙara damuwa ba.

Amma idan muna da gungu da yawa, ko kuma wani abu yana canzawa akai-akai, to tsarin yana da ƙarfi. Kuma sake fasalin sa ido akai-akai shine ɓata dukiya da lokaci, watau. ko da kasala kawai. Wannan yana buƙatar sarrafa kansa. Wahalar ta ta'allaka ne a cikin sauye-sauyen tsarin. Kuma ma'aikaci yana sarrafa wannan da kyau sosai.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ta yaya gungun mu ya bunkasa? Tun farko haka yake.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Sai ya kasance kamar haka.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

A ƙarshe ya zama haka.

Kuma mai aiki yana yin sa ido ta atomatik. Wurin shiga guda ɗaya.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Kuma kawai a wurin fita muna duba dashboard ɗin Grafana don ganin yadda rayuwar gungun mu ke tafasa a ciki.

Af, Grafana dashboard kuma ana rarraba shi tare da ma'aikacin mu kai tsaye a cikin lambar tushe. Kuna iya haɗawa da amfani. DevOps ɗin mu ya ba ni wannan hoton.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Ina zamu so muje gaba? Wannan:

  • Ƙirƙirar gwajin sarrafa kansa. Babban aikin shine gwaji ta atomatik na sabbin nau'ikan.
  • Muna kuma da gaske muna son sarrafa haɗin kai tare da ZooKeeper. Kuma akwai shirye-shiryen haɗawa da ZooKeeper-operator. Wadancan. An rubuta ma'aikaci don ZooKeeper kuma yana da ma'ana cewa masu aiki biyu sun fara haɗawa don gina mafita mafi dacewa.
  • Muna son yin ƙarin hadaddun alamomi masu mahimmanci.
  • Na yi haske da kore cewa muna gabatowa gadon Samfura - AIKATA, watau tare da sakin ma'aikaci na gaba za mu riga mu sami gadon samfuri. Wannan kayan aiki ne mai ƙarfi wanda ke ba ku damar gina hadaddun jeri daga guda.
  • Kuma muna son sarrafa ayyuka masu rikitarwa. Babban shine Re-sharding.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Bari mu ɗauki wasu matsakaicin sakamako.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Menene muke samu a sakamakon haka? Kuma yana da daraja a yi ko a'a? Shin yana da mahimmanci don ƙoƙarin jawo bayanan zuwa Kubernetes kuma amfani da mai aiki gabaɗaya da ma'aikacin Alitnity musamman?

A fitarwa muna samun:

  • Mahimman sauƙaƙewa da sarrafa kansa na daidaitawa, ƙaddamarwa, da kiyayewa.
  • Saka idanu da aka gina nan da nan.
  • Kuma shirye-shiryen yin amfani da ƙididdiga masu ƙima don yanayi masu rikitarwa. Wani aiki kamar ƙara kwafi baya buƙatar a yi da hannu. Mai aiki yana yin wannan.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Tambaya guda ɗaya kawai ta rage. Mun riga mun sami bayanai a cikin Kubernetes, haɓakawa. Menene game da aikin irin wannan mafita, musamman tunda ClickHouse an inganta shi don aiki?

Amsar ita ce komai yana da kyau! Ba zan yi cikakken bayani ba, wannan shi ne batun wani rahoto na daban.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Amma akwai irin wannan aikin kamar TSBS. Menene babban aikinsa? Wannan gwajin aikin bayanai ne. Wannan ƙoƙari ne don kwatanta dumi tare da dumi, mai laushi da laushi.

Ta yaya yake aiki? Ana samar da saitin bayanai ɗaya. Sannan ana gudanar da wannan saitin bayanai akan rumbun adana bayanai daban-daban ta hanyar amfani da saitin gwaje-gwaje iri daya. Kuma kowace rumbun adana bayanai tana warware matsala guda ta yadda ta san yadda. Sannan zaku iya kwatanta sakamakon.

Ya riga ya goyi bayan ɗimbin tarin bayanai. Na gano manyan guda uku. Wannan:

  • TimecaleDB.
  • InfluxDB.
  • ClickHouse.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

An kuma yi kwatancen da wani bayani makamancin haka. Kwatanta da RedShift. An yi kwatancen akan Amazon. ClickHouse shima yana gaban kowa akan wannan lamarin.

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Wace matsaya za a iya cimma daga abin da na ce?

  • DB a cikin Kubernetes yana yiwuwa. Wataƙila kowane abu yana yiwuwa, amma gabaɗaya yana kama da zai yiwu. ClickHouse a cikin Kubernetes tabbas yana yiwuwa tare da taimakon ma'aikacin mu.
  • Mai aiki yana taimakawa aiwatar da ayyuka ta atomatik kuma da gaske yana sauƙaƙa rayuwa.
  • Ayyukan al'ada ne.
  • Kuma ga alama a gare mu za a iya kuma ya kamata a yi amfani da wannan.

Bude tushen - shiga mu!

Kamar yadda na fada a baya, ma'aikacin samfurin buɗaɗɗe ne gaba ɗaya, don haka zai yi kyau sosai idan matsakaicin adadin mutane ya yi amfani da shi. Shiga mu! Muna jiran ku duka!

Godiya ga kowa!

Tambayoyi

Mai gudanarwa a Kubernetes don sarrafa tarin bayanai. Vladislav Klimenko (Altinity, 2019)

Na gode da rahoton! Sunana Anton. Na fito daga SEMrush Ina mamakin me ke faruwa game da shiga. Muna jin labarin sa ido, amma ba komai game da shiga, idan muka yi magana game da dukan gungu. Misali, mun ɗaga gungu akan kayan masarufi. Kuma muna amfani da gungumen azaba, muna tattara su cikin tudun gama gari ta amfani da daidaitattun hanyoyi. Sannan daga nan ne muke samun bayanan da ke sha'awar mu.

Tambaya mai kyau, watau shiga cikin jerin abubuwan da za a yi. Ma'aikacinmu bai sarrafa wannan ta atomatik ba tukuna. Har yanzu yana ci gaba, aikin har yanzu matashi ne. Mun fahimci buƙatar shiga. Wannan kuma batu ne mai matukar muhimmanci. Kuma tabbas ba shi da mahimmanci fiye da saka idanu. Amma na farko a cikin jerin don aiwatarwa shine sa ido. Za a yi katako. A zahiri, muna ƙoƙarin sarrafa kowane fanni na rayuwar tari. Saboda haka, amsar ita ce, a halin yanzu mai aiki, da rashin alheri, bai san yadda za a yi wannan ba, amma yana cikin tsare-tsaren, za mu yi shi. Idan kana son shiga, sai a ja request, don Allah.

Sannu! Na gode da rahoton! Ina da madaidaicin tambaya mai alaƙa da Juzu'i masu dagewa. Lokacin da muka ƙirƙiri saiti tare da wannan afaretan, ta yaya mai aiki zai tantance ko wane kumburi muke da wani faifai ko babban fayil ɗin da aka makala? Dole ne mu fara bayyana masa cewa don Allah sanya ClickHouse a kan waɗannan nodes ɗin da ke da faifai?

Kamar yadda na fahimta, wannan tambaya ci gaba ce ta ajiya na gida, musamman madaidaicin hanyar masauki. Wannan yana kama da bayyana wa dukkan tsarin cewa ana buƙatar ƙaddamar da kwasfa akan irin wannan kumburin, wanda muke da faifai mai haɗawa ta jiki, wanda aka ɗora akan irin wannan hanyar. Wannan sashe ne gaba daya da na tabo a kai a kai domin amsar da ake samu tana da girma sosai.

A takaice dai yana kama da haka. A zahiri, muna buƙatar tanadin waɗannan kundin. A halin yanzu, babu wani tanadi mai ƙarfi a cikin ajiyar gida, don haka DevOps dole ne ya yanke fayafai da kansu, waɗannan kundin. Kuma dole ne su bayyana KubberneTes na samar da cewa zaku sami dagewa irin wannan da kuma irin wannan aji, waɗanda, waɗanda suke akan irin waɗannan nodes. Sannan kuna buƙatar bayyana wa Kubernetes cewa kwas ɗin da ke buƙatar irin wannan kuma irin wannan aji na ajiya na gida yana buƙatar jagora kawai zuwa irin waɗannan nodes ta amfani da labels. Don waɗannan dalilai, mai aiki yana da ikon sanya wani nau'in lakabi da ɗaya kowane misali mai masaukin baki. Kuma ya bayyana cewa Kubernetes za su yi amfani da kwas ɗin don yin aiki kawai a kan nodes waɗanda suka dace da buƙatun, alamomi, cikin sauƙi. Masu gudanarwa suna ba da lakabi da kayan aiki da hannu. Sannan ya yi mizani.

Kuma zaɓi na uku ne, na gida, wanda ke taimaka wa wannan ɗan sauƙi. Kamar yadda na riga na nanata, wannan aiki ne mai ɗorewa akan daidaitawa, wanda a ƙarshe yana taimakawa wajen samun mafi girman aiki.

Ina da tambaya ta biyu dangane da wannan. An tsara Kubernetes ta hanyar da ba ruwanmu ko mun rasa kumburi ko a'a. Menene ya kamata mu yi a wannan yanayin idan mun rasa kumburin da ke ratayewa?

Ee, an fara sanya Kubernetes cewa dangantakarmu da kwas ɗinmu kamar shanu ne, amma a nan tare da mu kowane faifai ya zama wani abu kamar dabba. Akwai irin wannan matsalar da ba za mu iya jefar da su kawai ba. Kuma ci gaban Kubernetes yana tafiya a cikin hanyar da ba zai yuwu a bi da shi gabaɗaya ta hanyar falsafa ba, kamar dai ita ce hanyar da aka watsar gabaɗaya.

Yanzu ga tambaya mai amfani. Me za ku yi idan kun rasa kumburin da faifan ke ciki? Anan ana magance matsalar a matakin mafi girma. A cikin yanayin ClickHouse, muna da kwafi waɗanda ke aiki a matsayi mafi girma, watau. a matakin ClickHouse.

Menene sakamakon halin da ake ciki? DevOps ne ke da alhakin tabbatar da cewa bayanai ba su ɓace ba. Dole ne ya saita kwafi daidai kuma dole ne ya tabbatar cewa kwafi yana gudana. Kwafi a matakin ClickHouse dole ne ya sami kwafin bayanai. Wannan ba shine matsalar da mai aiki ke warwarewa ba. Kuma ba matsalar da Kubernetes kanta ke warwarewa ba. Wannan yana a matakin ClickHouse.

Me za ku yi idan kumburin ƙarfe naku ya faɗi? Kuma ya zama cewa kuna buƙatar shigar da na biyu, samar da diski daidai da shi, kuma kuyi amfani da lakabi. Kuma bayan haka, zai cika ka'idodin da Kubernetes zai iya ƙaddamar da kwasfan misali akansa. Kubernetes zai kaddamar da shi. Adadin kwas ɗinku bai isa ya dace da ƙayyadadden lamba ba. Zai bi ta zagayowar da na nuna. Kuma a matakin mafi girma, ClickHouse zai fahimci cewa mun shigar da kwafi, har yanzu babu komai kuma muna buƙatar fara canja wurin bayanai zuwa gare shi. Wadancan. Har yanzu wannan tsari bai kasance mai sarrafa kansa ba tukuna.

Na gode da rahoton! Lokacin da kowane nau'in abubuwa masu banƙyama suka faru, ma'aikacin ya yi karo kuma ya sake farawa, kuma a wannan lokacin abubuwan sun zo, kuna ko ta yaya kuke ɗaukar wannan?

Me zai faru idan mai aiki ya yi karo kuma ya sake farawa, daidai?

Ee. Kuma a lokacin ne abubuwa suka zo.

Ayyukan abin da za a yi a cikin wannan yanayin an raba wani bangare tsakanin mai aiki da Kubernetes. Kubernetes yana da ikon sake kunna wani lamari da ya faru. Ya sake kunnawa. Kuma aikin ma'aikacin shine tabbatar da cewa lokacin da aka sake kunna tarihin taron a kansa, waɗannan abubuwan suna da ƙarfi. Kuma don kada faruwar irin wannan lamari ya sake tabarbarewa tsarinmu. Kuma ma'aikacin mu yana jure wa wannan aikin.

Sannu! Na gode da rahoton! Dmitry Zavyalov, kamfanin Smedova. Shin akwai shirye-shiryen ƙara ikon daidaitawa tare da haproxy ga mai aiki? Zan yi sha'awar wasu ma'auni ban da daidaitaccen ɗaya, don yana da wayo kuma ya fahimci cewa ClickHouse yana nan da gaske.

Kuna maganar Ingress?

Ee, maye gurbin Ingress tare da haproxy. A cikin haproxy za ka iya ƙayyade yanayin topology na gungu inda yake da kwafi.

Ba mu yi tunani game da shi ba tukuna. Idan kuna buƙatar shi kuma kuna iya bayyana dalilin da yasa ake buƙata, to za'a iya aiwatar da shi, musamman idan kuna son shiga. Za mu yi farin cikin yin la'akari da zaɓin. Amsar a takaice ita ce a'a, a halin yanzu ba mu da irin wannan aikin. Godiya ga tip, za mu duba wannan al'amari. Kuma idan kun kuma bayyana yanayin amfani da dalilin da yasa ake buƙata a aikace, alal misali, ƙirƙirar batutuwa akan GitHub, to hakan zai yi kyau.

Ya riga.

Lafiya. Muna budewa ga kowace shawara. Kuma an ƙara haproxy zuwa jerin abubuwan todo. Jerin todo yana girma, bai ragu ba tukuna. Amma wannan yana da kyau, yana nufin cewa samfurin yana buƙatar.

source: www.habr.com

Add a comment