Kulawa ya zama muhimmin mahimmanci na haɓaka hanyoyin samar da girgije yayin da rikitaccen tsarin rarraba ya karu. Wajibi ne a fahimci halayensu. Muna buƙatar kayan aiki masu ƙima waɗanda za su iya tattara bayanai daga duk sabis - da kuma samar da ƙwararrun ƙwararrun ma'amala guda ɗaya tare da nazarin aikin, nunin kuskure, samuwa da rajistan ayyukan.
Waɗannan kayan aikin guda ɗaya dole ne su kasance masu inganci da amfani. A cikin wannan labarin, za mu kalli manyan tarin fasaha guda biyu: EFK (Elasticsearch) da PLG (Loki) kuma mu bincika gine-gine da bambance-bambancen su.
Farashin EFK
Wataƙila kun riga kun ji game da mashahurin ELK ko EFK. Tarin ya ƙunshi sassa daban-daban: Elasticsearch (ajiya na abu), Logstash ko FluentD (tarin log da tarawa), da Kibana don gani.
Tsarin aiki na yau da kullun yayi kama da haka:
Elasticsearch - ajiyar abubuwa da aka rarraba tare da bincike da ƙididdigar ainihin lokaci. Kyakkyawan bayani don bayanan da aka tsara na rabin-tsari kamar rajistan ayyukan. Ana adana bayanai azaman takaddun JSON, ƙididdiga a ainihin lokacin kuma ana rarraba su a cikin kuɗaɗen tari. Ana amfani da jujjuyawar fihirisa, mai ɗauke da duk keɓaɓɓun kalmomi da takaddun alaƙa don binciken cikakken rubutu, wanda kuma ya dogara ne akan injin binciken Apache Lucene.
FluentD mai tattara bayanai ne wanda ke haɗa bayanai yayin tattarawa da cinye su. Yana ƙoƙarin tsara bayanai a cikin JSON gwargwadon yiwuwa. Gine-ginensa yana da yawa, akwai ƙari , goyon bayan al'umma, don kowane lokaci.
Kibana - kayan aikin gani na bayanai don Elasticsearch tare da ƙarin damar daban-daban, misali, nazarin jerin lokaci, nazarin jadawali, koyan inji da ƙari.
Elasticsearch architecture
Ana adana bayanan tari na Elasticsearch a baje ko'ina cikin duk kudurorin sa. Tari ya ƙunshi nodes da yawa don inganta samuwa da juriya. Kowane kumburi zai iya aiwatar da duk ayyukan tari, amma a cikin manyan ayyuka na aiki, nodes yawanci ana ba su ayyuka ɗaya ne.
Nau'o'in kullin gungun:
- babban kumburi - yana sarrafa gungu, ana buƙatar aƙalla uku, ɗaya yana aiki koyaushe;
- kumburin bayanai - yana adana bayanan da aka lissafta kuma yana yin ayyuka daban-daban tare da shi;
- kumburin ingest - yana tsara bututun mai don canza bayanai kafin yin la'akari;
- daidaita kumburi - buƙatun tuƙi, rage lokacin sarrafa bincike, daidaita ƙididdigar taro;
- kumburin faɗakarwa - ƙaddamar da ayyukan faɗakarwa;
- inji koyo kumburi - sarrafa inji koyo ayyuka.
Hoton da ke ƙasa yana nuna yadda ake adana bayanai da kuma yin kwafi a ko'ina don samun wadatar bayanai mafi girma.
Ana adana kowane bayanan kwafi a cikin jujjuyawar fihirisa, zanen da ke ƙasa yana nuna yadda hakan ke faruwa:
saitin
Ana iya duba cikakkun bayanai , Zan yi amfani da ginshiƙi na helm:
$ helm install efk-stack stable/elastic-stack --set logstash.enabled=false --set fluentd.enabled=true --set fluentd-elasticsFarashin PLG
Kada ka yi mamaki idan ba za ka iya samun wannan gagara ba, kamar yadda aka fi sani da Grafana Loki. A kowane hali, wannan tarin yana samun shahara saboda yana amfani da ingantattun hanyoyin fasaha. Wataƙila kun riga kun ji labarin Grafana, sanannen kayan aikin gani. Mahaliccinsa, wanda Prometheus ya yi wahayi, ya ɓullo da Loki, a kwance a kwance, babban tsarin tara log ɗin aiki. Loki yana nuna metadata kawai, ba mujallun kansu ba, mafita na fasaha wanda ke ba shi damar sauƙin amfani da tsada.
Promtail - wakili don aika rajistan ayyukan daga tsarin aiki zuwa gungu na Loki. Grafana kayan aikin gani ne bisa bayanai daga Loki.
An gina Loki akan ƙa'idodi iri ɗaya da Prometheus, yana mai da shi dacewa da kyau don adanawa da kuma nazarin rajistan ayyukan Kubernetes.
Loki gine-gine
Ana iya gudanar da Loki ko dai a matsayin tsari guda ɗaya ko azaman matakai masu yawa, yana ba da damar sikeli a kwance.
Hakanan yana iya aiki ko dai azaman aikace-aikacen monolithic ko azaman microservice. Gudu a matsayin tsari guda ɗaya na iya zama da amfani ga ci gaban gida ko don ƙaramin saka idanu. Don aiwatar da masana'antu da nauyin aiki mai ƙima, ana ba da shawarar yin amfani da zaɓin microservice. Hanyoyi don rubutawa da bayanan karantawa sun rabu, don haka ana iya daidaita shi da kyau kuma a daidaita shi yadda ake bukata.
Bari mu dubi tsarin gine-ginen tsarin tarin log ba tare da yin cikakken bayani ba:
Kuma ga bayanin (microservice architecture):
Abubuwan:
Promtail - wakili da aka sanya akan nodes (a matsayin saitin sabis), yana cire rajistan ayyukan daga ayyuka kuma yana samun damar Kubernetes API don samun metadata wanda zai yiwa rajistan ayyukan. Sannan ta aika da log ɗin zuwa babban sabis na Loki. Taswirar metadata tana goyan bayan ƙa'idodin sawa iri ɗaya kamar Prometheus.
Rabawa - mai rarraba sabis wanda ke aiki azaman buffer. Don aiwatar da miliyoyin bayanai, yana tattara bayanai masu shigowa, yana matsawa cikin tubalan yayin da ya zo. Da yawa daga cikin maɓuɓɓugar bayanai suna gudana lokaci guda, amma rajistan ayyukan da ke cikin rafin bayanai guda ɗaya ya kamata su bayyana a ɗaya daga cikinsu don duk tubalan sa. An tsara wannan cikin zobe na nutsewa da hashing jere. Don hakuri da kuskure da sakewa, ana yin wannan n sau (3 idan ba a daidaita shi ba).
Ingester - mai karɓar sabis. Tubalan bayanai suna zuwa a matse tare da ƙara rajistan ayyukan. Da zarar katangar ya isa girman girman, toshe yana jujjuya shi zuwa rumbun adana bayanai. Metadata yana zuwa ga fihirisar, kuma bayanai daga toshe log ɗin suna zuwa Chunks (yawanci ma'ajiyar abu). Bayan sake saiti, mai karɓa yana ƙirƙirar sabon toshe inda za'a ƙara sabbin shigarwar.
index - bayanai, DynamoDB, Cassandra, Google BigTable, da dai sauransu.
Ununƙwasa - shingen log a cikin nau'i mai matsewa, yawanci ana adana shi a cikin ma'ajin abu, misali, S3.
Tambaya - hanyar karatun da ke yin duk aikin datti. Yana duba kewayon lokaci da tambarin lokaci, sannan yana duba fihirisar don nemo matches. Bayan haka, yana karanta tubalan bayanai kuma yana tace su don samun sakamako.
Yanzu bari mu ga komai a aikace.
saitin
Hanya mafi sauƙi don shigarwa a Kubernetes ita ce amfani da helm. Muna ɗauka cewa kun riga kun shigar kuma kun daidaita shi ( kusan mai fassara)
Ƙara wurin ajiya kuma shigar da tari.
$ helm repo add loki https://grafana.github.io/loki/charts
$ helm repo update
$ helm upgrade --install loki loki/loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=falseA ƙasa akwai dashboard misali wanda ke nuna bayanai daga Prometheus don Etcd awo da Loki don rajistan ayyukan kwasfan fayiloli da Etcd.
Yanzu bari mu tattauna gine-gine na duka tsarin, da kuma kwatanta su damar da juna.
Daidaita
Harshen tambaya
Elasticsearch yana amfani da Query DSL da harshen tambaya na Lucene don samar da cikakkiyar damar binciken rubutu. Kafaffen ingin bincike ne mai ƙarfi tare da faffadan goyan bayan mai aiki. Tare da shi, zaku iya bincika ta mahallin kuma tsara ta dacewa.
A daya gefen zoben shine LogQL, ana amfani dashi a Loki, wanda zai gaje PromQL (harshen tambaya na Prometheus). Yana amfani da alamun log don tacewa kuma zaɓi bayanan log. Yana yiwuwa a yi amfani da wasu masu aiki da lissafi kamar yadda aka bayyana , amma dangane da iyawa yana bayan Harshen Elastic.
Tun da tambayoyi a Loki suna da alaƙa da alamun, suna da sauƙin daidaitawa tare da ma'auni, kuma a sakamakon haka, sun fi sauƙi don tsara sa ido na aiki tare da.
Ƙimar ƙarfi
Dukansu tari-ruwan suna iya daidaitawa a kwance, amma Loki yana sauƙaƙa saboda yana da rabe-raben karantawa da rubutawa da kuma tsarin gine-ginen microservice. Ana iya keɓance Loki don dacewa da bukatunku kuma ana iya amfani da shi don manyan kundin bayanan log ɗin.
Yawan yawa
Cluster multitenancy jigo ne na gama gari a cikin gajartawar OPEX, duka tambura suna ba da ɗimbin yawa. Akwai da yawa don Elasticsearch rabuwar abokin ciniki: fihirisar raba ga kowane abokin ciniki, tushen abokin ciniki, filayen abokin ciniki na musamman, masu tacewa. Loki yana da a cikin hanyar HTTP X-Scope-OrgID.
kudin
Loki yana da tasiri mai tsada sosai saboda gaskiyar cewa baya nuna bayanan, kawai metadata. Wannan ya cimma da ƙwaƙwalwar ajiya (cache), tun da ajiyar abu ya fi arha fiye da toshe ajiya, wanda ake amfani da shi a cikin gungun Elasticsearch.
ƙarshe
Za a iya amfani da tari na EFK don dalilai iri-iri, yana ba da mafi girman sassauƙa da ƙaƙƙarfan fasalin Kibana don nazari, gani, da tambayoyi. Ana iya ƙara haɓaka ta ta ƙarfin koyon injin.
Tarin Loki yana da amfani a cikin yanayin yanayin Kubernetes saboda tsarin gano metadata. Kuna iya daidaita bayanai cikin sauƙi don saka idanu dangane da jerin lokaci a cikin Grafana da rajistan ayyukan.
Idan ya zo ga farashi da ajiyar log na dogon lokaci, Loki kyakkyawan wurin shiga cikin mafita ga girgije.
Akwai ƙarin zaɓuɓɓuka akan kasuwa - wasu na iya zama mafi kyau a gare ku. Misali, GKE yana da haɗin kai na Stackdriver wanda ke ba da ingantaccen mafita na saka idanu. Ba mu saka su a cikin bincikenmu a wannan labarin ba.
Tunani:
Ma'aikata ne suka fassara da kuma shirya labarin don Habr - kwasa-kwasan kwasa-kwasan, darussan bidiyo da horar da kamfanoni daga kwararrun kwararru (Kubernetes, DevOps, Docker, Mai yiwuwa, Ceph, SRE, Agile)
source: www.habr.com