ProHoster > Блог > Whakahaerenga > Nga tono a Kubernetes Autoscaling me Prometheus me KEDA

Nga tono a Kubernetes Autoscaling me Prometheus me KEDA

Nga tono a Kubernetes Autoscaling me Prometheus me KEDATangata Poihau na Cimuanos

Ko te tauineine he whakaritenga matua mo nga tono kapua. Ki nga Kubernetes, he ngawari noa iho te whakamaarama i tetahi tono penei i te whakanui ake i te maha o nga tauira mo te whakatakotoranga tika, ReplicaSet — engari he mahi a-ringa.

Ka whakaaetia e Kubernetes nga tono kia whakahiahia aunoatia (arā, Pods i roto i te horahanga ReplicaSet) i roto i te tikanga whakapuakanga ma te whakamahi i te whakatakotoranga Whakapae Pod Autoscaler. Ko te paearu taunoa mo te tauine aunoa ko nga inenga whakamahi PTM (rauemi rauemi), engari ka taea e koe te whakauru i nga inenga ritenga me nga inenga o waho.

rōpū Kubernetes aaS mai i Mail.ru i whakamaoritia he tuhinga mo te whakamahi i nga inenga o waho ki te tauine aunoa i te tono Kubernetes. Hei whakaatu i te mahi o nga mea katoa, ka whakamahia e te kaituhi nga inenga tono urunga HTTP, ka kohia ma te whakamahi i te Prometheus.

Hei utu mo te whakahiato-aunoa o nga poro, ka whakamahia te Kubernetes Event Driven Autoscaling (KEDA), he kaiwhakahaere Kubernetes puna tuwhera. Ka tuitui taketake me te Horizontal Pod Autoscaler ki te whakarato i te whakakao aunoa (tae atu ki te/mai i te kore) mo nga kawenga mahi-a-takahanga. Waehere e waatea ana i GitHub.

He tirohanga poto mo te punaha

Nga tono a Kubernetes Autoscaling me Prometheus me KEDA

E whakaatu ana te hoahoa i te whakaahuatanga poto mo te mahi o nga mea katoa:

  1. Ka whakaratohia e te tono nga inenga tatauranga hit HTTP i roto i te whakatakotoranga Prometheus.
  2. Kua whirihorahia a Prometheus ki te kohikohi i enei inenga.
  3. Ko te Prometheus scaler i KEDA kua whirihorahia ki te tauine aunoa i te tono i runga i te maha o nga whiu HTTP.

Inaianei ka korero au ki a koe mo ia huānga.

KEDA me Prometheus

Ko te Prometheus he punaha punaha tuwhera mo te tirotiro me te kete taputapu mataara, he waahanga Kapua Te Punaha Rorohiko Rapu. Ka kohia nga inenga mai i nga momo puna ka penapena hei raraunga raupapa wa. Hei tiro i nga raraunga ka taea e koe te whakamahi Karepe etahi atu taputapu whakaata ranei e mahi ana me te Kubernetes API.

Kei te tautoko a KEDA i te ariā o te tauine - ka noho hei piriti i waenga i te KEDA me te punaha o waho. Ko te whakatinanatanga o te scaler he mea motuhake ki ia punaha whaainga me te tango raraunga mai i a ia. Ka whakamahia e KEDA ki te whakahaere i te tauine aunoa.

Ka tautokohia e nga kaitarai nga puna raraunga maha, hei tauira, Kafka, Redis, Prometheus. Arā, ka taea te whakamahi KEDA ki te āwhata aunoa i ngā tukunga Kubernetes mā te whakamahi i ngā inenga Prometheus hei paearu.

Tono whakamatautau

Ko te tono whakamatautau a Golang e tuku uru ana ma te HTTP me te mahi i nga mahi nui e rua:

  1. Ka whakamahi i te whare pukapuka kiritaki a Prometheus Go ki te whakamahi i te tono me te whakarato i te inenga http_requests, kei roto he tatau pa. Kei te URI te waahi mutunga kei te waatea nga inenga Prometheus /metrics. 
    var httpRequestsCounter = promauto.NewCounter(prometheus.CounterOpts{
       Name: "http_requests",
       Help: "number of http requests",
   })
  2. Hei whakautu mo te tono GET ka whakanui te tono i te uara o te matua (access_count) i Redis. He huarahi ngawari tenei ki te mahi i nga mahi hei waahanga o te kaihautu HTTP me te tirotiro hoki i nga inenga Prometheus. Me rite te uara ine ki te uara access_count i Redis. 
    func main() {
       http.Handle("/metrics", promhttp.Handler())
       http.HandleFunc("/test", func(w http.ResponseWriter, r 
*http.Request) {
           defer httpRequestsCounter.Inc()
           count, err := client.Incr(redisCounterName).Result()
           if err != nil {
               fmt.Println("Unable to increment redis counter", err)
               os.Exit(1)
           }
           resp := "Accessed on " + time.Now().String() + "nAccess count " + strconv.Itoa(int(count))
           w.Write([]byte(resp))
       })
       http.ListenAndServe(":8080", nil)
   }

Ka tukuna te tono ki a Kubernetes ma Deployment. Ka hangaia ano he ratonga ClusterIP, ka taea e te tūmau Prometheus te tiki inenga tono.

konei whakaaturanga tukunga mo te tono.

Tūmau Prometheus

Ko te whakaaturanga tukunga Prometheus ko:

  • ConfigMap — ki te whakawhiti i te whirihora Prometheus;
  • Deployment — mo te tuku Prometheus i roto i te roopu Kubernetes;
  • ClusterIP — ratonga mo te uru ki te UI Prometheus;
  • ClusterRole, ClusterRoleBinding и ServiceAccount — mo te rapu-aunoa i nga ratonga i Kubernetes (Te kitenga-aunoa).

konei whakaaturanga mo te whakahaere Prometheus.

KEDA Prometheus ScaledObject

Ka noho te kaiwhatu hei piriti i waenga i te KEDA me te punaha o waho e tika ana kia whiwhi inenga. ScaledObject he rauemi ritenga e tika ana kia tukuna ki te tukutahi i te tukunga me te puna takahanga, i tenei keehi ko Prometheus.

ScaledObject kei roto nga korero whakahiatotanga, metadata puna takahanga (pēnei i nga mea ngaro hononga, ingoa tūtira), wa pooti, ​​wa whakaora, me etahi atu raraunga. Ka puta ko te rauemi whakahiato-aunoa e rite ana (whakamaramatanga HPA) hei tauine i te horahanga.

Ina he ahanoa ScaledObject kua mukua, kua whakawāteatia te whakamāramatanga HPA e hāngai ana.

Anei te whakamāramatanga ScaledObject mo ta tatou tauira, ka whakamahia he tauine Prometheus:

apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
 name: prometheus-scaledobject
 namespace: default
 labels:
   deploymentName: go-prom-app
spec:
 scaleTargetRef:
   deploymentName: go-prom-app
 pollingInterval: 15
 cooldownPeriod:  30
 minReplicaCount: 1
 maxReplicaCount: 10
 triggers:
 - type: prometheus
   metadata:
     serverAddress: 
http://prometheus-service.default.svc.cluster.local:9090
     metricName: access_frequency
     threshold: '3'
     query: sum(rate(http_requests[2m]))

Whakaarohia nga waahanga e whai ake nei:

  1. Ka tohu ia Deployment Me te ingoa go-prom-app.
  2. Momo keu - Prometheus. Kei te whakahuahia te wahitau tūmau Prometheus me te ingoa ineine, te paepae me te Uiui PromQL, ka whakamahia. Uiui PromQL - sum(rate(http_requests[2m])).
  3. E ai ki pollingInterval, Ka tono a KEDA i tetahi whaainga mai i a Prometheus ia tekau ma rima hēkona. Kia kotahi i raro (minReplicaCount), karekau e nui ake te maha o nga poti maxReplicaCount (i tenei tauira - tekau).

Ka taea te whakauru minReplicaCount rite ki te kore. I tenei keehi, ka whakahohehia e KEDA te tukunga kore-ki-tetahi katahi ka whakaatu i te HPA mo te whakatikatika aunoa. Ka taea hoki te raupapa whakamuri, ara, te tauine mai i te kotahi ki te kore. I roto i te tauira, kaore matou i whiriwhiri i te kore no te mea he ratonga HTTP tenei, ehara i te punaha tono.

Ko te makutu i roto i te tauine aunoa

Ka whakamahia te paepae hei keu ki te whakarahi i te horahanga. I roto i ta maatau tauira, ko te patai PromQL sum(rate (http_requests [2m])) ka whakahoki i te reiti tono HTTP whakahiato (tono mo ia hekona), i ine i nga meneti e rua kua hipa.

I te mea ko te uara paepae e toru, ko te tikanga ka kotahi i raro i te waa o te uara sum(rate (http_requests [2m])) iti iho i te toru. Mena ka piki ake te uara, ka taapirihia he taapiri i ia wa sum(rate (http_requests [2m])) ka piki ma te toru. Hei tauira, mena mai i te 12 ki te 14 te uara, ka wha te maha o nga pene.

Inaianei me ngana ki te whakarite!

Tuhinga o mua

Ko nga mea katoa e hiahiatia ana e koe ko te roopu Kubernetes me tetahi taputapu whirihora kubectl. Ka whakamahia e tenei tauira he tautau minikube, engari ka taea e koe te tango i tetahi atu. Hei whakauru i te tautau kei reira rangatira.

Tāutahia te putanga hōu ki Mac:

curl -Lo minikube 
https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 
&& chmod +x minikube
sudo mkdir -p /usr/local/bin/
sudo install minikube /usr/local/bin/

Tāuta kubectlkia uru atu ki te roopu Kubernetes.

Tāutahia te putanga hōu ki Mac:

curl -LO 
"https://storage.googleapis.com/kubernetes-release/release/$(curl -s
https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl"
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
kubectl version

KEDA tāutanga

Ka taea e koe te tuku KEDA i roto i nga huarahi maha, kua whakarārangihia ki roto tuhinga. Kei te whakamahi ahau i te YAML monolithic:

kubectl apply -f
https://raw.githubusercontent.com/kedacore/keda/master/deploy/KedaScaleController.yaml

Kua whakauruhia a KEDA me ona waahanga ki te mokowā ingoa keda. Tono ki te taki:

kubectl get pods -n keda

Taria kia timata te Kaiwhakahaere KEDA ka haere ki Running State. A muri iho, haere tonu.

Tāuta Redis mā te Helm

Mena karekau koe i whakauru i te Helm, whakamahia tenei ārahitanga. Tonoa kia tāuta ki runga Mac:

brew install kubernetes-helm
helm init --history-max 200

helm init ka arawhiti i te atanga raina whakahau a rohe me te whakauru ano hoki Tiller ki te kāhui Kubernetes.

kubectl get pods -n kube-system | grep tiller

Tatari kia uru te Pokao Tiller ki te ahua Rere.

Panui a te kaiwhakamaori: Ka whakamahi te kaituhi i a Helm@2, e hiahia ana kia whakauruhia te waahanga tūmau Tiller. Inaianei he mea tika a Helm@3, kaore e hiahiatia he waahanga tūmau.

Whai muri i te whakauru i te Helm, kotahi te whakahau hei timata i a Redis:

helm install --name redis-server --set cluster.enabled=false --set 
usePassword=false stable/redis

Manatokohia kua angitu a Redis:

kubectl get pods/redis-server-master-0

Taria kia haere a Redis ki te kawanatanga Running.

Taupānga Tukunga

Tono tukunga:

kubectl apply -f go-app.yaml

//output
deployment.apps/go-prom-app created
service/go-prom-app-service created

Tirohia kua timata nga mea katoa:

kubectl get pods -l=app=go-prom-app

Taria kia uru a Redis ki te kawanatanga Running.

Te tuku i te Tūmau Prometheus

Ka whakamahia e te whakaaturanga Prometheus Whakakitenga Ratonga Kubernetes mo Prometheus. Ka taea e ia te kite i nga putunga tono i runga i te tapanga ratonga.

kubernetes_sd_configs:
   - role: service
   relabel_configs:
   - source_labels: [__meta_kubernetes_service_label_run]
     regex: go-prom-app-service
     action: keep

Hei tohatoha:

kubectl apply -f prometheus.yaml

//output
clusterrole.rbac.authorization.k8s.io/prometheus created
serviceaccount/default configured
clusterrolebinding.rbac.authorization.k8s.io/prometheus created
configmap/prom-conf created
deployment.extensions/prometheus-deployment created
service/prometheus-service created

Tirohia kua timata nga mea katoa:

kubectl get pods -l=app=prometheus-server

Taria kia haere a Prometheus ki te kawanatanga Running.

Whakamahia kubectl port-forward ki te uru ki te atanga kaiwhakamahi Prometheus (tūmau API ranei) i http://localhost:9090.

kubectl port-forward service/prometheus-service 9090

Te whakamahi i te Whirihoranga Tauira Aunoa KEDA

Whakahau ki te hanga ScaledObject:

kubectl apply -f keda-prometheus-scaledobject.yaml

Tirohia nga raarangi kaiwhakahaere KEDA:

KEDA_POD_NAME=$(kubectl get pods -n keda 
-o=jsonpath='{.items[0].metadata.name}')
kubectl logs $KEDA_POD_NAME -n keda

He penei te ahua o te hua:

time="2019-10-15T09:38:28Z" level=info msg="Watching ScaledObject:
default/prometheus-scaledobject"
time="2019-10-15T09:38:28Z" level=info msg="Created HPA with 
namespace default and name keda-hpa-go-prom-app"

Tirohia i raro i nga tono. Me rere tetahi tauira na te mea minReplicaCount rite 1:

kubectl get pods -l=app=go-prom-app

Manatokohia kua tutuki pai te hanga rauemi HPA:

kubectl get hpa

Me kite koe i tetahi mea penei:

NAME                   REFERENCE                TARGETS     MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-go-prom-app   Deployment/go-prom-app   0/3 (avg)   1         10        1          45s

Tirohanga hauora: uru tono

Hei uru atu ki te waahi mutunga REST o ta maatau tono, whakahaere:

kubectl port-forward service/go-prom-app-service 8080

Ka taea e koe te uru atu ki to taupānga Haere ma te whakamahi i te wahitau http://localhost:8080. Ki te mahi i tenei, whakahaerehia te whakahau:

curl http://localhost:8080/test

He penei te ahua o te hua:

Accessed on 2019-10-21 11:29:10.560385986 +0000 UTC 
m=+406004.817901246
Access count 1

I tenei wa tirohia ano a Redis. Ka kite koe ko te ki access_count kua piki ki te 1:

kubectl exec -it redis-server-master-0 -- redis-cli get access_count
//output
"1"

Me mohio ko te uara ine http_requests rite tonu:

curl http://localhost:8080/metrics | grep http_requests
//output
# HELP http_requests number of http requests
# TYPE http_requests counter
http_requests 1

Waihanga Utaina

Ka whakamahia e matou hey — whaipainga mo te whakaputa kawenga:

curl -o hey https://storage.googleapis.com/hey-release/hey_darwin_amd64 
&& chmod a+x hey

Ka taea hoki e koe te tango i te taputapu mo Linux ranei Windows.

Whakahaerehia:

./hey http://localhost:8080/test

Ma te taunoa, ka tukuna e te taputapu nga tono 200. Ka taea e koe te manatoko i tenei ma te whakamahi i nga inenga Prometheus me Redis.

curl http://localhost:8080/metrics | grep http_requests
//output
# HELP http_requests number of http requests
# TYPE http_requests counter
http_requests 201
kubectl exec -it redis-server-master-0 -- redis-cli get access_count
//output
201

Whakamanahia te uara o te ine tika (i whakahokia mai e te uinga PromQL):

curl -g 
'http://localhost:9090/api/v1/query?query=sum(rate(http_requests[2m]))'
//output
{"status":"success","data":{"resultType":"vector","result":[{"metric":{},"value":[1571734214.228,"1.686057971014493"]}]}}

I tenei keehi ko te hua tuturu 1,686057971014493 ka whakaatuhia ki te mara value. He iti rawa tenei mo te whakatauine, i te mea ko te paepae i whakaritea e matou ko te 3.

He kawenga ake!

I roto i te tauranga hou, aroturuki i te maha o nga putunga tono:

kubectl get pods -l=app=go-prom-app -w

Me whakanui ake te uta ma te whakamahi i te whakahau: 

./hey -n 2000 http://localhost:8080/test

Whai muri i etahi wa, ka kite koe i te HPA e whakaheke ana i te tukunga me te whakarewa i nga putunga hou. Tirohia to HPA kia mohio ai:

kubectl get hpa
NAME                   REFERENCE                TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-go-prom-app   Deployment/go-prom-app   1830m/3 (avg)   1         10        6          4m22s

Mena kaore i te rite te kawenga, ka whakahekehia te tukunga ki te waahi kotahi anake te putunga e rere ana. Mena kei te pirangi koe ki te tirotiro i te ine tika (i whakahokia mai e te patai PromQL), katahi ka whakamahi i te whakahau:

curl -g 
'http://localhost:9090/api/v1/query?query=sum(rate(http_requests[2m]))'

Te horoi

//Delete KEDA
kubectl delete namespace keda
//Delete the app, Prometheus server and KEDA scaled object
kubectl delete -f .
//Delete Redis
helm del --purge redis-server

mutunga

Ma te KEDA ka taea e koe te whakarahi aunoa i o tukunga Kubernetes (ki/mai i te kore) i runga i nga raraunga mai i nga inenga o waho. Hei tauira, i runga i nga inenga Prometheus, te roa o te tutira i Redis, te roanga o nga kaihoko ki te kaupapa Kafka.

Kei te whakauru a KEDA ki tetahi puna o waho me te whakarato hoki i ana inenga na roto i te Tūmau Metrics ki te Horizontal Pod Autoscaler.

Ngā manaakitanga pai!

He aha atu hei panui:

  1. Nga mahi pai me nga tikanga pai mo te whakahaere ipu me nga Kubernetes i nga taiao whakangao.
  2. 90+ taputapu whai hua mo Kubernetes: te whakatakotoranga, te whakahaere, te aroturuki, te haumaru me te maha atu.
  3. To tatou hongere huri noa i Kubernetes i Telegram.

Source: will.com

Tāpiri i te kōrero