ProHoster > Блог > Nchịkwa > Ngwa Autoscaling Kubernetes na-eji Prometheus na KEDA

Ngwa Autoscaling Kubernetes na-eji Prometheus na KEDA

Ngwa Autoscaling Kubernetes na-eji Prometheus na KEDANwoke Balloon nke Cimuanos dere

Scalability bụ isi ihe achọrọ maka ngwa igwe ojii. Site na Kubernetes, ịmegharị ngwa dị mfe dị ka ịbawanye ọnụ ọgụgụ nke oyiri maka mbugharị kwesịrị ekwesị ma ọ bụ ReplicaSet - mana ọ bụ usoro ntuziaka.

Kubernetes na-enye ohere ka etinyere ngwa ngwa na-akpaghị aka (ya bụ, Pods na ntinye ma ọ bụ ReplicaSet) n'ụzọ nkwupụta site na iji nkọwapụta Horizontal Pod Autoscaler. Ụkpụrụ ndabara maka ịchacha akpaka bụ metrik ojiji CPU (metrics metrics), mana ị nwere ike jikọta metrik omenala na nke a na-enye na mpụga.

otu Kubernetes aaS sitere na Mail.ru tụgharịrị asụsụ edemede maka otu esi eji metrik mpụga iji tụọ ngwa Kubernetes na-akpaghị aka. Iji gosi ka ihe niile si arụ ọrụ, onye odee na-eji metrik nweta arịrịọ HTTP, nke anakọtara site na iji Prometheus.

Kama ịmegharị akpaaka kwụ ọtọ, Kubernetes Event Driven Autoscaling (KEDA) na-eji, onye ọrụ Kubernetes mepere emepe. Ọ na-ejikọta ya na Horizontal Pod Autoscaler iji nye autoscaling enweghị nkebi (gụnyere na / site na efu) maka ibu ọrụ na-ebute mmemme. Koodu dị na GitHub.

Nchịkọta nkenke nke usoro ahụ

Ngwa Autoscaling Kubernetes na-eji Prometheus na KEDA

Eserese ahụ na-egosi nkọwa dị nkenke nke ka ihe niile si arụ ọrụ:

  1. Ngwa a na-enye metric hit HTTP n'ụdị Prometheus.
  2. A haziri Prometheus ịnakọta metrik ndị a.
  3. A haziri Prometheus scaler na KEDA ka ọ tụọ ngwa ngwa ngwa dabere na ọnụọgụ HTTP hits.

Ugbu a, m ga-agwa gị n'ụzọ zuru ezu banyere nke ọ bụla mmewere.

KEDA na Prometheus

Prometheus bụ ngwa nleba anya na nleba anya sistemụ mepere emepe, akụkụ Ngwé ojii Ngwá Ọrụ Ngwá Ọrụ. Na-anakọta metrik site na isi mmalite dị iche iche ma chekwaa ya ka data usoro oge. Iji hụ data ị nwere ike iji Grafana ma ọ bụ ngwaọrụ ọhụụ ndị ọzọ na-arụ ọrụ na Kubernetes API.

KEDA na-akwado echiche nke scaler - ọ na-arụ ọrụ dị ka àkwà mmiri n'etiti KEDA na usoro mpụga. Mmezu nke scaler bụ kpọmkwem maka sistemụ ebumnuche ọ bụla wee wepụta data na ya. KEDA na-eji ha na-ejikwa nha akpaka.

Scalers na-akwado ọtụtụ isi mmalite data, dịka ọmụmaatụ, Kafka, Redis, Prometheus. Nke ahụ bụ, enwere ike iji KEDA mee ka mkpokọta Kubernetes na-akpaghị aka na-eji Prometheus metrics dị ka njirisi.

Ngwa nwale

Ngwa ule Golang na-enye ohere site na HTTP ma rụọ ọrụ abụọ dị mkpa:

  1. Na-eji ọbá akwụkwọ ndị ahịa Prometheus Go iji rụọ ngwa a wee weta metric http_requests, nke nwere ọnụ ọgụgụ dị egwu. Ebe njedebe ebe Prometheus metrics dị na URI /metrics. 
    var httpRequestsCounter = promauto.NewCounter(prometheus.CounterOpts{
       Name: "http_requests",
       Help: "number of http requests",
   })
  2. Na nzaghachi maka arịrịọ GET ngwa a na-abawanye uru igodo (access_count) na Redis. Nke a bụ ụzọ dị mfe iji rụọ ọrụ ahụ dịka akụkụ nke onye na-ahụ maka HTTP ma lelee Prometheus metrics. Uru metrik ga-abụrịrị otu ihe ahụ access_count na Redis. 
    func main() {
       http.Handle("/metrics", promhttp.Handler())
       http.HandleFunc("/test", func(w http.ResponseWriter, r 
*http.Request) {
           defer httpRequestsCounter.Inc()
           count, err := client.Incr(redisCounterName).Result()
           if err != nil {
               fmt.Println("Unable to increment redis counter", err)
               os.Exit(1)
           }
           resp := "Accessed on " + time.Now().String() + "nAccess count " + strconv.Itoa(int(count))
           w.Write([]byte(resp))
       })
       http.ListenAndServe(":8080", nil)
   }

A na-ebuga ngwa a na Kubernetes site na Deployment. A na-emepụtakwa ọrụ ClusterIP, ọ na-enye ohere ka ihe nkesa Prometheus nweta metric ngwa.

ebe a ngosipụta ngosipụta maka ngwa ahụ.

Ihe nkesa Prometheus

Ngosipụta ntinye nke Prometheus nwere:

  • ConfigMap - ịnyefe nhazi Prometheus;
  • Deployment - maka ibuga Prometheus na ụyọkọ Kubernetes;
  • ClusterIP - ọrụ maka ịnweta UI Prometheus;
  • ClusterRole, ClusterRoleBinding и ServiceAccount - maka nchọpụta akpaaka nke ọrụ na Kubernetes (Nchọpụta akpaaka).

ebe a gosipụtara maka ịgba ọsọ Prometheus.

KEDA Prometheus ScaledObject

The scaler na-arụ ọrụ dị ka àkwà mmiri dị n'etiti KEDA na usoro mpụga nke achọrọ iji nweta metrics. ScaledObject bụ akụrụngwa omenala nke kwesịrị ibunye iji mekọrịta ntinye ya na isi mmalite ihe omume, na nke a Prometheus.

ScaledObject nwere ozi nlegharị anya, metadata isi mmalite mmemme (dị ka nzuzo njikọ, aha kwụ n'ahịrị), oge ntuli aka, oge mgbake na data ndị ọzọ. Ọ na-ebute ihe enyemaka autoscaling kwekọrọ (HPA definition) iji tụọ ntinye ya.

Mgbe ihe ScaledObject ehichapụrụ, a kpochapụrụ nkọwa HPA kwekọrọ.

Nke a bụ nkọwa ScaledObject maka ihe atụ anyị, ọ na-eji ihe nrịbama Prometheus:

apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
 name: prometheus-scaledobject
 namespace: default
 labels:
   deploymentName: go-prom-app
spec:
 scaleTargetRef:
   deploymentName: go-prom-app
 pollingInterval: 15
 cooldownPeriod:  30
 minReplicaCount: 1
 maxReplicaCount: 10
 triggers:
 - type: prometheus
   metadata:
     serverAddress: 
http://prometheus-service.default.svc.cluster.local:9090
     metricName: access_frequency
     threshold: '3'
     query: sum(rate(http_requests[2m]))

Biko rịba ama isi ihe ndị a:

  1. Ọ na-atụ aka Deployment N'aha go-prom-app.
  2. Ụdị mkpali - Prometheus. Akpọrọ adreesị ihe nkesa Prometheus yana aha metrik, ọnụ ụzọ na Ajụjụ PromQL, nke a ga-eji. Ajụjụ PromQL - sum(rate(http_requests[2m])).
  3. Dị ka pollingInterval,KEDA na-arịọ maka ebumnuche n'aka Prometheus kwa sekọnd iri na ise ọ bụla. Ọ dịkarịa ala otu n'okpuru (minReplicaCount), na ọnụ ọgụgụ kachasị nke pọd anaghị agafe maxReplicaCount (na ihe atụ a - iri).

Enwere ike itinye ya minReplicaCount nhata ka efu. N'okwu a, KEDA na-arụ ọrụ ntinye efu-na-otu wee kpughee HPA maka nhazigharị akpaaka ọzọ. Usoro ntụgharị ahụ nwekwara ike, ya bụ, ịcha site na otu gaa na efu. Na ọmụmaatụ, anyị ahọrọghị efu n'ihi na nke a bụ ọrụ HTTP ọ bụghị usoro achọrọ.

Anwansi dị n'ime autoscaling

A na-eji ọnụ ụzọ eme ihe dị ka ihe na-akpali akpali iji mee ka ntinye ahụ dị elu. N'ihe atụ anyị, ajụjụ PromQL sum(rate (http_requests [2m])) weghachite ọnụego arịrịọ HTTP achịkọtara (arịrịọ kwa nkeji), tụrụ n'ime nkeji abụọ gara aga.

Ebe ọ bụ na ọnụ ahịa ọnụ ụzọ bụ atọ, ọ pụtara na a ga-enwe otu n'okpuru mgbe uru ahụ sum(rate (http_requests [2m])) ihe na-erughị atọ. Ọ bụrụ na ọnụ ahịa na-abawanye, a na-agbakwunye mgbakwunye mgbakwunye oge ọ bụla sum(rate (http_requests [2m])) na-abawanye site na atọ. Dịka ọmụmaatụ, ọ bụrụ na uru sitere na 12 ruo 14, mgbe ahụ, ọnụ ọgụgụ nke pọd bụ anọ.

Ugbu a, ka anyị gbalịa ịhazi ya!

presetting

Naanị ihe ị chọrọ bụ ụyọkọ Kubernetes na akụrụngwa ahaziri kubectl. Ihe atụ a na-eji ụyọkọ minikube, ma ị nwere ike were nke ọ bụla ọzọ. Iji wụnye ụyọkọ enwere njikwa.

Wụnye ụdị kachasị ọhụrụ na Mac:

curl -Lo minikube 
https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 
&& chmod +x minikube
sudo mkdir -p /usr/local/bin/
sudo install minikube /usr/local/bin/

Tọọ kubectlịbanye na ụyọkọ Kubernetes.

Wụnye ụdị kachasị ọhụrụ na Mac:

curl -LO 
"https://storage.googleapis.com/kubernetes-release/release/$(curl -s
https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl"
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
kubectl version

KEDA echichi

Ị nwere ike ibuga KEDA n'ọtụtụ ụzọ, edepụtara ha na ya akwụkwọ. Ana m eji YAML monolithic:

kubectl apply -f
https://raw.githubusercontent.com/kedacore/keda/master/deploy/KedaScaleController.yaml

Awụnyere KEDA na akụkụ ya n'ime oghere aha keda. Iwu ka ịlele:

kubectl get pods -n keda

Chere ka onye ọrụ KEDA malite wee gaa na Running State. Ma mgbe nke ahụ gasịrị, gaa n'ihu.

Ịwụnye Redis site na iji Helm

Ọ bụrụ na ị nweghị Helm arụnyere, jiri nke a ndu. Iwu iji wụnye na Mac:

brew install kubernetes-helm
helm init --history-max 200

helm init malite interface ahịrị iwu mpaghara wee tinyekwa ya Tiller gaa na ụyọkọ Kubernetes.

kubectl get pods -n kube-system | grep tiller

Chere ka Tiller pod banye steeti na-agba ọsọ.

Ihe ntụgharị asụsụ: Onye ode akwụkwọ na-eji Helm@2, nke chọrọ ka etinyere akụrụngwa nkesa Tiller. Ugbu a Helm@3 dị mkpa, ọ chọghị akụkụ nkesa.

Mgbe ị wụnye Helm, otu iwu zuru ezu ịmalite Redis:

helm install --name redis-server --set cluster.enabled=false --set 
usePassword=false stable/redis

Chọpụta na Redis amalitela nke ọma:

kubectl get pods/redis-server-master-0

Chere ka Redis banye na steeti Running.

Nkwanye ngwa

Iwu ibunye:

kubectl apply -f go-app.yaml

//output
deployment.apps/go-prom-app created
service/go-prom-app-service created

Lelee na ihe niile amalitela:

kubectl get pods -l=app=go-prom-app

Chere ka Redis banye steeti Running.

Na-ebunye ihe nkesa Prometheus

Ihe ngosi Prometheus na-eji Nchọpụta ọrụ Kubernetes maka Prometheus. Ọ na-enye ohere ịchọpụta pọd ngwa ngwa dabere na akara ọrụ.

kubernetes_sd_configs:
   - role: service
   relabel_configs:
   - source_labels: [__meta_kubernetes_service_label_run]
     regex: go-prom-app-service
     action: keep

Iji bukwasa:

kubectl apply -f prometheus.yaml

//output
clusterrole.rbac.authorization.k8s.io/prometheus created
serviceaccount/default configured
clusterrolebinding.rbac.authorization.k8s.io/prometheus created
configmap/prom-conf created
deployment.extensions/prometheus-deployment created
service/prometheus-service created

Lelee na ihe niile amalitela:

kubectl get pods -l=app=prometheus-server

Chere ka Prometheus banye na steeti Running.

Jiri kubectl port-forward iji nweta interface onye ọrụ Prometheus (ma ọ bụ ihe nkesa API) na http://localhost:9090.

kubectl port-forward service/prometheus-service 9090

Na-ebuga KEDA Autoscaling Nhazi

Iwu imepụta ScaledObject:

kubectl apply -f keda-prometheus-scaledobject.yaml

Lelee ndekọ ndị ọrụ KEDA:

KEDA_POD_NAME=$(kubectl get pods -n keda 
-o=jsonpath='{.items[0].metadata.name}')
kubectl logs $KEDA_POD_NAME -n keda

Nsonaazụ dị ka nke a:

time="2019-10-15T09:38:28Z" level=info msg="Watching ScaledObject:
default/prometheus-scaledobject"
time="2019-10-15T09:38:28Z" level=info msg="Created HPA with 
namespace default and name keda-hpa-go-prom-app"

Lelee n'okpuru ngwa. Otu ihe atụ ga-agba ọsọ n'ihi na minReplicaCount nha 1:

kubectl get pods -l=app=go-prom-app

Chọpụta na emepụtara akụrụngwa HPA nke ọma:

kubectl get hpa

Ị ga-ahụ ihe dị ka:

NAME                   REFERENCE                TARGETS     MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-go-prom-app   Deployment/go-prom-app   0/3 (avg)   1         10        1          45s

Nyocha ahụike: ịnweta ngwa

Iji nweta ebe ngwụcha REST ngwa anyị, gbaa ọsọ:

kubectl port-forward service/go-prom-app-service 8080

Ị nwere ike nweta ngwa Go gị site na iji adreesị http://localhost:8080. Iji mee nke a, gbanye iwu:

curl http://localhost:8080/test

Nsonaazụ dị ka nke a:

Accessed on 2019-10-21 11:29:10.560385986 +0000 UTC 
m=+406004.817901246
Access count 1

N'oge a, leleekwa Redis. Ị ga-ahụ igodo ahụ access_count abawanye na 1:

kubectl exec -it redis-server-master-0 -- redis-cli get access_count
//output
"1"

Gbaa mbọ hụ na uru metric bụ http_requests otu ihe:

curl http://localhost:8080/metrics | grep http_requests
//output
# HELP http_requests number of http requests
# TYPE http_requests counter
http_requests 1

Ibu Okike

Anyị ga -eji hey - akụrụngwa maka ịmepụta ibu:

curl -o hey https://storage.googleapis.com/hey-release/hey_darwin_amd64 
&& chmod a+x hey

Ịnwekwara ike ibudata akụrụngwa maka Linux ma ọ bụ Windows.

Gbaa ya:

./hey http://localhost:8080/test

Site na ndabara, akụrụngwa na-eziga arịrịọ 200. Ị nwere ike nyochaa nke a site na iji Prometheus metrics yana Redis.

curl http://localhost:8080/metrics | grep http_requests
//output
# HELP http_requests number of http requests
# TYPE http_requests counter
http_requests 201
kubectl exec -it redis-server-master-0 -- redis-cli get access_count
//output
201

Kwado uru nke metric n'ezie (nke ajụjụ PromQL weghachiri ya):

curl -g 
'http://localhost:9090/api/v1/query?query=sum(rate(http_requests[2m]))'
//output
{"status":"success","data":{"resultType":"vector","result":[{"metric":{},"value":[1571734214.228,"1.686057971014493"]}]}}

N'okwu a, nsonaazụ ya bụ 1,686057971014493 a na-egosipụtakwa n'ọhịa value. Nke a ezughị maka ịchacha, ebe ọ bụ na ọnụ ụzọ anyị debere bụ 3.

Ibu ọzọ!

N'ọnụ ọnụ ọhụrụ ahụ, nyochaa ọnụ ọgụgụ pọd ngwa:

kubectl get pods -l=app=go-prom-app -w

Ka anyị bulie ibu site na iji iwu: 

./hey -n 2000 http://localhost:8080/test

Mgbe obere oge gachara, ị ga-ahụ HPA ka ọ na-ebunye ọkwa ahụ yana ịmalite pọd ọhụrụ. Lelee HPA gị ka ijide n'aka:

kubectl get hpa
NAME                   REFERENCE                TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-go-prom-app   Deployment/go-prom-app   1830m/3 (avg)   1         10        6          4m22s

Ọ bụrụ na ibu ahụ na-ekwekọghị ekwekọ, a ga-ebelata ntinye ya ruo ebe naanị otu pọd na-agba ọsọ. Ọ bụrụ na ịchọrọ ịlele metric n'ezie (nke ajụjụ PromQL weghachiri), wee jiri iwu a:

curl -g 
'http://localhost:9090/api/v1/query?query=sum(rate(http_requests[2m]))'

Nhicha

//Delete KEDA
kubectl delete namespace keda
//Delete the app, Prometheus server and KEDA scaled object
kubectl delete -f .
//Delete Redis
helm del --purge redis-server

nkwubi

KEDA na-enye gị ohere ịbawanye ntinye nke Kubernetes gị ozugbo (ruo/site na efu) dabere na data sitere na metrik mpụga. Dịka ọmụmaatụ, dabere na metrics Prometheus, ogologo kwụ n'ahịrị na Redis, latency ndị ahịa na isiokwu Kafka.

KEDA jikọtara ya na isi mmalite ma na-enyekwa metrik ya site na Metrics Server na Horizontal Pod Autoscaler.

Jisie!

Kedu ihe ọzọ ị ga-agụ:

  1. Omume kacha mma na omume kacha mma maka ịgba ọsọ na Kubernetes na gburugburu mmepụta ihe.
  2. Ngwa 90+ bara uru maka Kubernetes: ntinye, njikwa, nlekota oru, nchekwa na ndị ọzọ.
  3. Ọwa anyị gburugburu Kubernetes na Telegram.

isi: www.habr.com

Tinye a comment