Hai Habr!
Muna tunatar da ku cewa mun fito da wani mai ban sha'awa kuma mai amfani game da tsarin Kubernetes. Abin ya fara da "" Brendan Burns, kuma, a hanya, muna da aiki a cikin wannan sashin . A yau muna gayyatar ku don karanta wani labarin daga shafin yanar gizon MiniIO wanda ke ba da taƙaitaccen bayani game da halaye da ƙayyadaddun tsarin ajiyar bayanai a Kubernetes.
Kubernetes ya canza fasalin haɓaka aikace-aikacen gargajiya da tsarin turawa. Yanzu ƙungiya za ta iya haɓakawa, gwadawa, da tura aikace-aikacen cikin al'amuran kwanaki-a cikin mahalli da yawa, duk a cikin gungu na Kubernetes. Irin wannan aiki tare da al'ummomin da suka gabata na fasaha yakan ɗauki makonni, idan ba watanni ba.
Wannan haɓaka yana yiwuwa ta hanyar abstraction da Kubernetes ya bayar - wato, saboda Kubernetes da kansa yana hulɗa tare da ƙananan bayanai na na'urori na jiki ko na kayan aiki, yana bawa masu amfani damar bayyana na'urar da ake so, adadin ƙwaƙwalwar da ake so, da kuma adadin akwati. misalai, a tsakanin sauran sigogi. Tare da ɗimbin al'umma da ke tallafawa Kubernetes da karɓuwarta ta ci gaba da faɗaɗa, Kubernetes shine jagora a tsakanin duk dandamalin kide-kide na kwantena ta gefe mai faɗi.
Yayinda amfani da Kubernetes ke girma, haka rudani game da tsarin ajiyar sa..
Tare da kowa da kowa yana fafatawa don wani yanki na Kubernetes kek (watau ajiyar bayanai), idan ana magana game da ajiyar bayanai, siginar yana nutsewa cikin hayaniya mai yawa.
Kubernetes ya ƙunshi ƙirar zamani don haɓaka aikace-aikace, turawa, da gudanarwa. Wannan samfurin zamani yana cire ma'ajin bayanai daga ƙididdiga. Don cikakken fahimtar detachment a cikin mahallin Kubernetes, kuna buƙatar fahimtar menene aikace-aikacen hukuma da marasa jiha, da kuma yadda ma'ajin bayanai suka dace da hakan. Wannan shine inda tsarin REST API da S3 ke amfani da shi yana da fa'ida bayyananne akan tsarin POSIX/CSI na sauran mafita.
A cikin wannan labarin, za mu yi magana game da tsarin adana bayanai a cikin Kubernetes kuma mu taɓa takamaiman muhawara tsakanin aikace-aikacen jihohi da marasa ƙasa don ƙarin fahimtar menene bambanci tsakanin su da dalilin da yasa yake da mahimmanci. Sauran rubutun za su dubi aikace-aikace da tsarin ajiyar bayanan su bisa ga mafi kyawun ayyuka don aiki tare da kwantena da Kubernetes.
Kwantena marasa Jiha
Kwantenan bisa ga dabi'a masu nauyi ne kuma marasa nauyi. Ana iya dakatar da su cikin sauƙi, cire su, ko tura su zuwa wani kumburi - duk cikin daƙiƙa. A cikin babban tsarin kaɗe-kaɗe na kwantena, irin waɗannan ayyuka suna faruwa koyaushe, kuma masu amfani ba sa ma lura da irin waɗannan canje-canje. Koyaya, motsi yana yiwuwa ne kawai idan kwandon ba shi da wani dogaro akan kumburin da yake a kai. Irin waɗannan kwantena an ce suna aiki marasa jiha.
Akwatunan Jiha
Idan kwantena ya adana bayanai akan na'urorin da aka makala a cikin gida (ko na'urar toshewa), to sai an matsar da ma'ajin bayanan da yake cikin su zuwa wani sabon kulli, tare da kwandon da kanta, idan ya gaza. Wannan yana da mahimmanci saboda in ba haka ba aikace-aikacen da ke gudana a cikin akwati ba zai iya yin aiki yadda ya kamata ba saboda yana buƙatar samun damar bayanan da aka adana akan kafofin watsa labarai na gida. Irin waɗannan kwantena an ce suna aiki m.
Daga mahangar fasaha zalla, ana iya matsar da kwantena na jihohi zuwa wasu nodes. Ana samun wannan yawanci ta amfani da tsarin fayil da aka rarraba ko toshe ma'ajiyar cibiyar sadarwa da ke haɗe zuwa duk nodes masu gudana kwantena. Ta wannan hanyar, kwantena suna samun damar juzu'i don adana bayanai na dindindin, kuma ana adana bayanai akan fayafai da ke cikin cibiyar sadarwa. Zan kira wannan hanyar "m kwantena tsarin kula", kuma a cikin sauran labarin zan kira shi don kare daidaito.
A cikin tsari na zahiri na zahiri, duk fastocin aikace-aikacen suna haɗe zuwa tsarin fayil ɗin da aka rarraba - nau'in ma'ajin da aka raba inda duk bayanan aikace-aikacen ke zama. Yayin da wasu bambance-bambancen suna yiwuwa, wannan babbar hanya ce.
Yanzu bari mu ga dalilin da ya sa tsarin tsarin kwantena ya zama abin ƙyama a cikin duniyar da ke tsakiyar girgije.
Ƙirar aikace-aikacen asali na Cloud
A al'adance, aikace-aikacen suna amfani da ma'ajin bayanai don tsararrun bayanai da faifai na gida ko tsarin fayil da aka rarraba inda aka zubar da duk bayanan da ba a tsara su ba ko ma da aka yi. Yayin da ɗimbin bayanan da ba a tsara su ba suna girma, masu haɓakawa sun fahimci cewa POSIX ya kasance mai yawan magana, yana da babban abin sama da ƙasa, kuma a ƙarshe ya hana aiwatar da aikace-aikacen yayin motsawa zuwa babban ma'auni.
Wannan ya fi ba da gudummawa ga fitowar sabon ma'auni don adana bayanai, wato, tushen girgije, aiki da farko bisa REST API da 'yantar da aikace-aikacen daga kulawa mai nauyi na ajiyar bayanan gida. A wannan yanayin, aikace-aikacen yana tafiya yadda ya kamata zuwa yanayin rashin jiha (tunda jihar tana cikin ma'ajiyar nesa). An gina aikace-aikacen zamani daga karce tare da wannan dalili. A ka'ida, duk wani aikace-aikacen zamani da ke sarrafa bayanai iri ɗaya ko wani (Logs, metadata, blobs, da dai sauransu) an gina su ne bisa ga tsarin girgije, inda ake mayar da jihar zuwa tsarin software da aka keɓe musamman don adanawa.
Hanyar da ta dace da kwantena ta tilasta duk wannan yanayin komawa daidai inda ya fara!
Ta hanyar amfani da musaya na POSIX don adana bayanai, aikace-aikacen suna aiki kamar suna da ƙima, kuma saboda wannan, sun fita daga mafi mahimmancin ka'idoji na ƙira-centric na girgije, wato, ikon bambanta girman zaren ma'aikacin aikace-aikacen dangane da mai shigowa. shigar da kaya, matsawa zuwa sabon kumburi da zarar kumburin halin yanzu ya gaza, da sauransu.
Idan muka yi la'akari da wannan halin da ake ciki, mun gano cewa lokacin zabar kantin sayar da bayanai, muna sake fuskantar matsalar POSIX vs. REST API, AMMA tare da ƙarin haɓaka matsalolin POSIX saboda yanayin rarraba yanayi na Kubernetes. Musamman,
- POSIX yana magana ne: POSIX Semantics yana buƙatar kowane aiki ya haɗa shi da metadata da masu bayanin fayil waɗanda ke taimakawa kula da yanayin aikin. Wannan yana haifar da ƙima mai mahimmanci waɗanda ba su da ƙimar gaske. API ɗin ajiyar abubuwa, musamman S3 API, kawar da waɗannan buƙatun, barin aikace-aikacen ya kunna wuta sannan "manta" game da kiran. Amsar tsarin ajiya yana nuna ko aikin ya yi nasara ko a'a. Idan ya gaza, aikace-aikacen na iya sake gwadawa.
- Ƙuntataccen hanyar sadarwa: A cikin tsarin da aka rarraba, ana nuna cewa za a iya samun aikace-aikacen da yawa da ke ƙoƙarin rubuta bayanai zuwa kafofin watsa labaru iri ɗaya. Sabili da haka, ba kawai aikace-aikacen za su yi gogayya da juna don canja wurin bayanai ba (don aika bayanai zuwa kafofin watsa labaru), amma tsarin ajiya da kansa zai yi gasa don wannan bandwidth ta hanyar aikawa da bayanai a cikin diski na jiki. Saboda yanayin POSIX, adadin kiran cibiyar sadarwa yana ƙaruwa sau da yawa. A gefe guda, S3 API yana ba da bambanci tsakanin kiran cibiyar sadarwa tsakanin waɗanda suka samo asali daga abokin ciniki zuwa uwar garken da waɗanda ke faruwa a cikin uwar garke.
- Tsaro: An tsara tsarin tsaro na POSIX don sa hannu na ɗan adam mai aiki: masu gudanarwa suna tsara matakan samun dama ga kowane mai amfani ko rukuni. Wannan yanayin yana da wuyar daidaitawa zuwa duniyar mai tsakiyar girgije. Aikace-aikace na zamani sun dogara da tsarin tsaro na tushen API, inda aka keɓe haƙƙin samun dama azaman saitin manufofi, asusun sabis, takaddun shaida na ɗan lokaci, da sauransu.
- Gudanarwa: Manyan kwantena na da wasu abubuwan gudanarwa. Muna magana ne game da daidaita daidaitattun damar yin amfani da bayanai, tabbatar da daidaiton bayanai, duk wannan yana buƙatar yin la'akari da hankali kan waɗanne tsarin samun damar bayanai don amfani. Dole ne a shigar da ƙarin software, kulawa, da kuma daidaita su, ban da ƙarin ƙoƙarin haɓakawa.
Interface Ma'ajiya Data Kwantena
Duk da yake Interface Storage Interface (CSI) ya kasance babban taimako tare da yaduwa na Kubernetes girma Layer, wani ɓangare na fitar da shi zuwa ga masu siyar da ajiya na ɓangare na uku, kuma ba da gangan ba ya ba da gudummawa ga imani cewa tsarin tsarin gandun daji shine hanyar da aka ba da shawarar don adana bayanai a cikin Kubernetes.
An haɓaka CSI azaman ma'auni don samar da shinge na sabani da tsarin adana fayil zuwa aikace-aikacen gado lokacin da ke gudana akan Kubernetes. Kuma, kamar yadda wannan labarin ya nuna, kawai halin da ake ciki wanda tsarin tsarin kwantena (da CSI a cikin halin yanzu) yana da ma'ana shine lokacin da aikace-aikacen kanta tsarin gado ne wanda ba zai yiwu ba don ƙara goyon baya ga API ajiya na abu. .
Yana da mahimmanci a fahimci cewa yin amfani da CSI a cikin nau'i na yanzu, wato, hawan girma lokacin aiki tare da aikace-aikacen zamani, za mu fuskanci kusan irin matsalolin da suka taso a cikin tsarin da aka tsara ma'ajin bayanai a cikin salon POSIX.
Hanya mafi Kyau
A wannan yanayin, yana da mahimmanci a fahimci cewa yawancin aikace-aikacen ba a tsara su ta asali ba musamman don aikin jiha ko na ƙasa. Wannan halin ya dogara da tsarin gine-ginen tsarin gabaɗaya da takamaiman zaɓin ƙira da aka yi. Bari mu ɗan yi magana game da aikace-aikacen hukuma.
A ka'ida, duk bayanan aikace-aikacen za a iya raba su zuwa manyan nau'ikan iri da yawa:
- Shiga bayanan
- Tambarin lokaci
- Bayanan ciniki
- Metadata
- Hotunan kwantena
- Blob (binary babban abu) bayanai
Duk waɗannan nau'ikan bayanan suna da tallafi sosai akan dandamali na ajiya na zamani, kuma akwai dandamali da yawa na tushen girgije waɗanda aka keɓance don isar da bayanai a cikin kowane takamaiman tsari. Misali, bayanan ma'amala da metadata na iya zama a cikin bayanan asalin girgije na zamani kamar CockroachDB, YugaByte, da sauransu. Ana iya adana hotunan kwantena ko bayanan toshewa a cikin rajistar docker dangane da MiniIO. Ana iya adana bayanan tambarin lokaci a cikin jerin bayanai na lokaci kamar InfluxDB, da sauransu. Ba za mu yi cikakken bayani game da kowane nau'in bayanai da aikace-aikacen sa a nan ba, amma babban ra'ayin shi ne a guji adana bayanan da aka dage da su wanda ya dogara da hawan faifai na gida.
Bugu da ƙari, sau da yawa yana da tasiri don samar da Layer caching na wucin gadi wanda ke aiki azaman wurin ajiyar fayil na wucin gadi don aikace-aikace, amma aikace-aikacen bai kamata ya dogara da wannan Layer a matsayin tushen gaskiya ba.
Ma'ajiyar aikace-aikace na jaha
Duk da yake a mafi yawan lokuta yana da amfani a kiyaye aikace-aikacen ba su da ƙasa, waɗannan aikace-aikacen da aka ƙera don adana bayanai - kamar rumbun adana bayanai, ma'ajiyar abubuwa, ma'ajiyar ƙima - dole ne su kasance masu ƙima. Bari mu ga dalilin da yasa aka tura waɗannan aikace-aikacen akan Kubernetes. Bari mu ɗauki MinIO a matsayin misali, amma irin waɗannan ƙa'idodin sun shafi kowane babban tsarin ajiya na asali na girgije.
An ƙirƙira aikace-aikacen asali na Cloud don cin gajiyar sassaucin da ke cikin kwantena. Wannan yana nufin cewa ba su yi wani zato game da yanayin da za a tura su ba. Misali, MinIO tana amfani da injin gogewa na ciki don samar da tsarin da isasshen juriya don ci gaba da aiki koda rabin fayafai sun gaza. MinIO kuma tana sarrafa amincin bayanai da tsaro ta amfani da hashing-gefen uwar garken mallakar mallaka da ɓoyewa.
Don irin waɗannan aikace-aikacen-tsakiyar gajimare, ƙididdiga na gida (PVs) sun fi dacewa azaman ajiyar ajiya. PV na gida yana ba da damar adana ɗanyen bayanai, yayin da aikace-aikacen da ke gudana akan waɗannan PVs suna tattara bayanai da kansu don auna bayanai da sarrafa buƙatun bayanai.
Wannan hanya ta fi sauƙi kuma mafi mahimmanci fiye da PVs na tushen CSI, waɗanda ke gabatar da nasu yadudduka na sarrafa bayanai da sakewa a cikin tsarin; Ma'anar ita ce, waɗannan yadudduka yawanci suna cin karo da aikace-aikacen da aka ƙera don zama mai faɗi.
Ƙaƙƙarfan motsi don ƙaddamar da bayanai daga ƙididdiga
A cikin wannan labarin, mun yi magana game da yadda ake mayar da aikace-aikacen don yin aiki ba tare da adana jihar ba, ko, a wasu kalmomi, an cire bayanan adanawa daga yin lissafi akan shi. A ƙarshe, bari mu kalli wasu misalai na gaske na wannan yanayin.
, sanannen dandamali na nazarin bayanai, ya kasance a al'adance kuma an tura shi akan HDFS. Duk da haka, yayin da Spark ke motsawa zuwa duniyar mai-tsakiyar gajimare, ana ƙara yin amfani da dandalin mara ƙasa ta amfani da `s3a`. Spark yana amfani da s3a don canja wurin jihar zuwa wasu tsarin, yayin da Spark kwantena da kansu suna aiki gaba ɗaya maras ƙasa. Sauran manyan ƴan kasuwa a fagen nazarin manyan bayanai, musamman, , , Har ila yau, suna motsawa don yin aiki tare da rarraba bayanan ajiya da lissafin akan su.
Hakanan ana iya ganin irin waɗannan samfuran akan wasu manyan dandamali na nazari, gami da Presto, Tensorflow zuwa R, Jupyter. Ta hanyar sauke yanayin zuwa tsarin ajiyar girgije mai nisa, zai zama mafi sauƙi don sarrafawa da haɓaka aikace-aikacen ku. Bugu da ƙari, yana sauƙaƙe ɗaukar aikace-aikacen zuwa wurare daban-daban.
source: www.habr.com