Kutarisisa kwave chinhu chakakosha chekukura mhinduro dzegore sezvo kuomarara kwemasisitimu akagoverwa kunowedzera. Zvakakosha kunzwisisa maitiro avo. Tinoda maturusi anogona scalable anogona kuunganidza data kubva kune ese masevhisi - uye kupa nyanzvi ine imwe chete interface ine ongororo yekuita, kuratidza kukanganisa, kuwanikwa uye matanda.
Maturusi mamwe chete aya anofanira kunge achishanda uye achibudirira. Muchikamu chino, tichatarisa maviri akakurumbira tekinoroji stacks: EFK (Elasticsearch) uye PLG (Loki) uye tiongorore mavakirwo avo uye mutsauko.
EFK chikamu
Iwe unogona kunge wakambonzwa nezve yakakurumbira ELK kana EFK. Iyo stack ine akati wandei akasiyana zvikamu: Elasticsearch (chinhu chekuchengetedza), Logstash kana FluentD (log kuunganidza uye kuunganidza), uye Kibana yekuona.
Yakajairika mafambiro ekushanda anoita seizvi:
Elasticsearch - yakagoverwa kuchengetedza chinhu nekutsvaga uye chaiyo-nguva analytics. Yakanakisa mhinduro yehafu-yakarongeka data senge matanda. Ruzivo rwunochengetwa semagwaro eJSON, akaiswa mukati menguva chaiyo uye akagoverwa munzvimbo dzese dzemapoka. Inverted index inoshandiswa, ine ese akasiyana mazwi uye magwaro anoenderana neakazara-zvinyorwa kutsvaga, izvo zvinobva zvabva paApache Lucene yekutsvaga injini.
FluentD muunganidzi wedata unobatanidza data painounganidza nekuidya. Inoedza kuronga data muJSON zvakanyanya sezvinobvira. Mavakirwo ayo anowedzera, kune akawanda , inotsigirwa nenharaunda, yezviitiko zvese.
kibana - chishandiso chekuona data cheElasticsearch chine akasiyana ekuwedzera masimba, semuenzaniso, nguva yekuongorora, kuongororwa kwegirafu, kudzidza muchina uye nezvimwe.
Elasticsearch architecture
Elasticsearch cluster data inochengetwa yakapararira munzvimbo dzayo dzese. Cluster ine akawanda node ekuvandudza kuwanikwa uye kusimba. Chero node inogona kuita mabasa ese esumbu, asi muhukuru-out deployments, node dzinowanzo kupihwa mabasa ega ega.
Mhando dzeCluster node:
- master node - inotarisira sumbu, zvinenge zvitatu zvinodiwa, imwe inogara ichishanda;
- data node - inochengetedza data indexed uye inoita mabasa akasiyana-siyana nayo;
- ingest node - inoronga mapaipi ekushandura data isati yanyora indexing;
- coordinating node - routing zvikumbiro, kuderedza danho rekutsvaga kutsvaga, kuronga misa indexing;
- kunyevera node - kutanga mabasa ekuzivisa;
- muchina kudzidza node - kugadzirisa muchina kudzidza mabasa.
Dhiagiramu iri pazasi inoratidza kuti data rinochengetwa sei uye rinodzokororwa munzvimbo dzese kuti uwane kuwanikwa kwedata.
Imwe neimwe replica data inochengetwa mune inverted index, dhayagiramu iri pazasi inoratidza kuti izvi zvinoitika sei:
Kuiswa
Details inogona kutariswa , ndichashandisa helm chart:
$ helm install efk-stack stable/elastic-stack --set logstash.enabled=false --set fluentd.enabled=true --set fluentd-elasticsPLG stack
Usashamisika kana iwe usingakwanise kuwana iyi acronym, sezvo ichizivikanwa zviri nani seGrafana Loki. Chero zvazvingaitika, iyi stack iri kuwana mukurumbira nekuti inoshandisa yakasimbiswa tekinoroji mhinduro. Unogona kunge wakambonzwa nezve Grafana, chishandiso chakakurumbira chekuona. Vagadziri vayo, vakafemerwa naPrometheus, vakagadzira Loki, yakatwasuka, yakakwira-inoshanda log aggregation system. Loki inoratidzira chete metadata, kwete majenari pachawo, mhinduro yehunyanzvi inobvumira kuti ive nyore kushandisa uye inodhura-inoshanda.
Promtail - mumiriri wekutumira matanda kubva kune inoshanda sisitimu kune iyo Loki cluster. grafana chishandiso chekuona chinobva pane data kubva kuLoki.
Loki yakavakirwa pamisimboti yakafanana nePrometheus, ichiita kuti ive yakakodzera kuchengetedza uye kuongorora Kubernetes matanda.
Loki architecture
Loki inogona kumhanyiswa senge senge nzira imwe chete kana seakawanda maitiro, ichibvumira kuyera kuyera.
Inogona zvakare kushanda senge monolithic application kana se microservice. Kumhanya senzira imwe chete kunogona kubatsira pakuvandudza kwenzvimbo kana kutarisisa kudiki. Nekuita maindasitiri uye scalable basa rekuita, zvinokurudzirwa kushandisa iyo microservice sarudzo. Nzira dzekunyora nekuverenga data dzakapatsanurwa, saka inogona kukwenenzverwa uye kuyerwa sezvinodiwa.
Ngatitarisei mavakirwo eiyo log yekuunganidza system tisingaende mune zvakadzama:
Uye heino tsananguro (microservice architecture):
Zvinoriumba:
Promtail - mumiririri akaiswa pane node (seseti yemasevhisi), inobvisa matanda kubva kumabasa uye inowana iyo Kubernetes API kuti iwane metadata inozomaka matanda. Inobva yatumira iyo logi kune iyo huru Loki sevhisi. Metadata mepu inotsigira iwo iwo mareti ekumaka sePrometheus.
Distributor - mugoveri webasa anoshanda sebhafa. Kugadzirisa mamirioni emarekodhi, inorongedza inouya data, ichiimanikidza mumabhuraki sezvainosvika. Madhata akati wandei ari kushanda panguva imwe chete, asi matanda eimwe inouya data rwizi anofanirwa kungoonekwa mune rimwe rawo kune ese mabhururu. Izvi zvakarongwa kuita mhete yemasingi uye sequential hashing. Nekutadza kushivirira uye kuregeredza, izvi zvinoitwa n nguva (3 kana isina kugadzirirwa).
Ingester - mugamuchiri webasa. Data blocks inosvika yakamanikidzwa nematanda akawedzerwa. Kamwe bhuroka rave nehukuru hwakakwana, iyo block inonyudzwa kune database. Metadata inoenda kune index, uye data kubva kurogi block inoenda kuChunks (kazhinji chinhu chekuchengetedza). Mushure mekugadzirisazve, mugamuchiri anogadzira bhuroka idzva uko zvinyorwa zvitsva zvichawedzerwa.
indekisi - dhatabhesi, DynamoDB, Cassandra, Google BigTable, nezvimwe.
Chunks - log blocks mune yakamanikidzwa fomu, inowanzochengetwa mukuchengetedza chinhu, semuenzaniso, S3.
Querier - nzira yekuverenga iyo inoita basa rose rakasviba. Inotarisa pane yenguva uye timestamp, uyezve inotarisa index kuti iwane machisi. Tevere, inoverenga mabhuraki e data uye inoasefa kuti iwane mhedzisiro.
Zvino ngationei zvose zviri kuitika.
Kuiswa
Nzira iri nyore yekuisa muKubernetes ndeye kushandisa helm. Isu tinofungidzira kuti iwe wakatoisa uye nekuigadzirisa ( approx. mushanduri)
Wedzera repository uye isa stack.
$ helm repo add loki https://grafana.github.io/loki/charts
$ helm repo update
$ helm upgrade --install loki loki/loki-stack --set grafana.enabled=true,prometheus.enabled=true,prometheus.alertmanager.persistentVolume.enabled=false,prometheus.server.persistentVolume.enabled=falsePazasi pane muenzaniso dashboard inoratidza data kubva kuPrometheus yeEtcd metrics uye Loki yeEtcd pod logs.
Zvino ngatikurukurei mavakirwo ezvose zviri zviviri masisitimu, uye zvakare enzanisa kugona kwavo kune mumwe nemumwe.
Kuenzanisa
Mutauro wekubvunza
Elasticsearch inoshandisa Query DSL uye Lucene query mutauro kupa yakazara-mavara ekutsvaga kugona. Iyo yakasimbiswa, ine simba yekutsvaga injini ine yakafara opareta rutsigiro. Nayo, unogona kutsvaga nemamiriro ezvinhu uye kuronga maererano nekwavo.
Kune rumwe rutivi rwemhete ndeye LogQL, inoshandiswa muLoki, mutsivi wePromQL (Prometheus query language). Inoshandisa log tags kusefa uye sarudza log data. Zvinokwanisika kushandisa vamwe vashandisi uye arithmetic sezvakatsanangurwa , asi maererano nekugona inosara kumashure kweElastic language.
Sezvo mibvunzo muLoki yakabatana nema tag, ari nyore kuwiriranisa nemametrics, uye semhedzisiro, ari nyore kuronga mashandiro ekutarisa nawo.
Kukwanisika
Maturiki ese ari maviri akachinjika, asi Loki inoita kuti zvive nyore nekuti ine yakaparadzana kuverenga nekunyora nzira uye microservice architecture. Loki inogona kugadzirwa kuti ienderane nezvido zvako uye inogona kushandiswa kune yakakura kwazvo mavhoriyamu edata regi.
Multitenancy
Cluster multitenancy idingindira rakajairwa muOPEX pfupiso, ese mastacks anopa multitenancy. Kune akati wandei eElasticsearch kupatsanurwa kwevatengi: indekisi yakaparadzana yemutengi wega wega, mutengi-yakavakirwa nzira, yakasarudzika minda yevatengi, mafirita ekutsvaga. Loki ane muchimiro cheHTTP X-Scope-OrgID musoro.
mari
Loki inodhura zvakanyanya nekuda kwekuti haina indexed data, chete metadata. Izvi zvinobudirira uye ndangariro (cache), sezvo kuchengetedza chinhu kwakachipa pane block kuchengetedza, iyo inoshandiswa mumasumbu eElasticsearch.
mhedziso
Iyo EFK stack inogona kushandiswa kune zvakasiyana siyana, ichipa yakanyanya kuchinjika uye chimiro-chakapfuma Kibana interface yeanalytics, kuona, uye mibvunzo. Inogona kuwedzeredzwa nehunyanzvi hwekudzidza muchina.
Iyo Loki stack inobatsira muKubernetes ecosystem nekuda kweiyo metadata yekutsvaga maitiro. Iwe unogona nyore kurongedza data rekutarisa zvichibva pane yenguva yakatevedzana muGrafana uye matanda.
Kana zvasvika pamutengo uye yenguva refu yekuchengetedza log, Loki inzvimbo yakanaka yekupinda mumafu mhinduro.
Pane dzimwe nzira pamusika - dzimwe dzinogona kunge dziri nani kwauri. Semuenzaniso, GKE ine Stackdriver yekubatanidza iyo inopa yakanakisa yekutarisa mhinduro. Hatina kuvabatanidza mukuongorora kwedu munyaya ino.
Mareferensi:
Chinyorwa chakaturikirwa uye chakagadzirirwa Habr nevashandi - makosi akasimba, makosi evhidhiyo uye kudzidziswa kwemakambani kubva kune vanodzidzira nyanzvi (Kubernetes, DevOps, Docker, Ansible, Ceph, SRE, Agile)
Source: www.habr.com