ProHoster > Π‘Π»ΠΎΠ³ > Maamulka > Codsiyada Kubernetes Autoscaling ee Prometheus iyo KEDA

Codsiyada Kubernetes Autoscaling ee Prometheus iyo KEDA

Codsiyada Kubernetes Autoscaling ee Prometheus iyo KEDABalloon Man by Cimuanos

Miisaankani waa shuruudda muhiimka ah ee codsiyada daruuraha. Kubernetes, cabirida arji waa mid fudud sida kordhinta tirada nuqulada ku haboon ee la diri karo ama ReplicaSet - laakiin waa hab-socodka gacanta.

Kubernetes waxay ogolaataa codsiyada in si toos ah loo miisaamo (tusaale ahaan, Pods ee meelaynta ama ReplicaSet) si caddaynaya iyadoo la adeegsanayo qeexida Horizontal Pod Autoscaler. Shuruudaha caadiga ah ee cabirka tooska ah waa cabbirada isticmaalka CPU (miisaanka kheyraadka), laakiin waxaad isku dari kartaa cabbirada caadiga ah iyo kuwa dibadda laga bixiyo.

kooxda Kubernetes aaS ka Mail.ru tarjumay maqaal ku saabsan sida loo isticmaalo cabbirrada dibadda si si toos ah loo cabbiro codsiga Kubernetes. Si loo tuso sida wax walba u shaqeeyaan, qoraagu wuxuu adeegsadaa cabbirka codsiga gelitaanka HTTP, kuwaas oo la soo ururiyo iyadoo la adeegsanayo Prometheus.

Halkii laga heli lahaa toosan toosan ee boodhadhka, Kubernetes Event Driven Autoscaling (KEDA) ayaa la isticmaalaa, oo ah hawlwadeenka Kubernetes il furan. Waxay ku biirtay asal ahaan Horizontal Pod Autoscaler si ay u bixiso otomaatig aan toos ahayn (ay ku jirto ilaa/ eber) culeyska shaqada ee ay wado dhacdada. Koodhka ayaa laga heli karaa GitHub.

Dulmar kooban oo ku saabsan nidaamka

Codsiyada Kubernetes Autoscaling ee Prometheus iyo KEDA

Jaantusku wuxuu muujinayaa sharraxaad kooban oo ku saabsan sida wax walba u shaqeeyaan:

  1. Codsigu wuxuu ku bixiyaa cabbirka tirinta HTTP ee qaabka Prometheus.
  2. Prometheus waxaa loo habeeyey si uu u ururiyo cabbiradan.
  3. Miisaanka Prometheus ee KEDA waxaa loo habeeyey inuu si toos ah u cabbiro codsiga ku salaysan tirada hits HTTP.

Hadda waxaan si faahfaahsan kaaga sheegi doonaa shay kasta.

KEDA iyo Prometheus

Prometheus waa agabka la socodka iyo feejignaanta nidaamka isha furan, qayb Aasaaska Kombuyuutarka Dhalada Bulsho ee Cloud. Wuxuu ka soo ururiyaa cabbirada ilo kala duwan wuxuuna u kaydiyaa sidii xog taxane ah oo wakhti ah. Si aad u sawirto xogta waxaad isticmaali kartaa Grafana ama qalab kale oo muuqaal ah oo la shaqeeya Kubernetes API.

KEDA waxay taageertaa fikradda miisaan-qaade - waxay u shaqeysaa sidii buundo u dhaxaysa KEDA iyo nidaamka dibadda. Hirgelinta miisaanku waxay gaar u tahay nidaam kasta oo la beegsanayo waxayna ka soo saartaa xogta. KEDA waxay markaa u isticmaashaa iyaga si ay u xakamayso ismiidaminta tooska ah.

Scalers waxay taageeraan ilo xogeedyo badan, tusaale ahaan, Kafka, Redis, Prometheus. Taasi waa, KEDA waxaa loo isticmaali karaa in si toos ah loo cabbiro Kubernetes geynta iyadoo la adeegsanayo cabbirrada Prometheus sida shuruudaha.

Codsiga tijaabi

Codsiga tijaabada Golang wuxuu bixiyaa marin u helka HTTP wuxuuna qabtaa laba hawlood oo muhiim ah:

  1. Wuxuu adeegsadaa maktabadda macmiilka ee Prometheus Go si uu u qalabeeyo arjiga oo uu u bixiyo mitirka http_requests, kaas oo ka kooban tiro garaac ah. Halka ugu dambaysa ee mitirka Prometheus laga heli karo waxay ku taal URI /metrics. 
    var httpRequestsCounter = promauto.NewCounter(prometheus.CounterOpts{
       Name: "http_requests",
       Help: "number of http requests",
   })
  2. Iyadoo laga jawaabayo codsi GET Codsigu wuxuu kordhiyaa qiimaha furaha (access_count) ee Redis. Tani waa hab sahlan oo lagu sameeyo shaqada iyada oo qayb ka ah maamulaha HTTP iyo sidoo kale hubi cabbirada Prometheus. Qiimaha mitirku waa inuu la mid yahay qiimaha access_count Redis. 
    func main() {
       http.Handle("/metrics", promhttp.Handler())
       http.HandleFunc("/test", func(w http.ResponseWriter, r 
*http.Request) {
           defer httpRequestsCounter.Inc()
           count, err := client.Incr(redisCounterName).Result()
           if err != nil {
               fmt.Println("Unable to increment redis counter", err)
               os.Exit(1)
           }
           resp := "Accessed on " + time.Now().String() + "nAccess count " + strconv.Itoa(int(count))
           w.Write([]byte(resp))
       })
       http.ListenAndServe(":8080", nil)
   }

Codsiga waxaa la geeyaa Kubernetes iyada oo loo marayo Deployment. Adeeg ayaa sidoo kale la abuuray ClusterIP, waxay u ogolaataa server-ka Prometheus inuu helo cabbirada codsiga.

halkan caddaynta geynta codsiga.

Prometheus Server

Muujinta geynta Prometheus waxay ka kooban tahay:

  • ConfigMap - si loo wareejiyo Prometheus config;
  • Deployment - in la geeyo Prometheus ee kutlada Kubernetes;
  • ClusterIP - adeegga gelitaanka UI Prometheus;
  • ClusterRole, ClusterRoleBinding ΠΈ ServiceAccount - si toos ah loogu ogaado adeegyada Kubernetes (Auto-discovery).

halkan muujinta socodsiinta Prometheus.

KEDA Prometheus ScaledObject

Miisaanku wuxuu u shaqeeyaa sidii buundo u dhaxaysa KEDA iyo nidaamka dibadda kaas oo cabbirada loo baahan yahay in laga helo. ScaledObject waa kheyraad caado ah oo u baahan in la geeyo si loo waafajiyo daabulidda isha dhacdada, kiiskan Prometheus.

ScaledObject waxa ku jira xogta meelaynta, xogta badan ee isha dhacdada (sida sirta xidhiidhka, magaca safka), wakhtiga doorashada, muddada soo kabashada, iyo xog kale. Waxay keentaa kheyraadka autoscaling ee u dhigma (Qeexida HPA) si loo cabbiro hawlgelinta.

Marka shay ScaledObject waa la tirtiray, qeexida HPA ee u dhiganta waa la nadiifiyay.

Waa kan qeexitaanku ScaledObject Tusaale ahaan, waxay isticmaashaa miisaan Prometheus:

apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
 name: prometheus-scaledobject
 namespace: default
 labels:
   deploymentName: go-prom-app
spec:
 scaleTargetRef:
   deploymentName: go-prom-app
 pollingInterval: 15
 cooldownPeriod:  30
 minReplicaCount: 1
 maxReplicaCount: 10
 triggers:
 - type: prometheus
   metadata:
     serverAddress: 
http://prometheus-service.default.svc.cluster.local:9090
     metricName: access_frequency
     threshold: '3'
     query: sum(rate(http_requests[2m]))

Ka fiirso qodobadan soo socda:

  1. Wuxuu tilmaamayaa Deployment Magac leh go-prom-app.
  2. Nooca kicinta - Prometheus. Ciwaanka Prometheus ee server-ka waxaa lagu xusay magaca mitirka ah, marinka iyo Waydii PromQL, kaas oo la isticmaali doono. Weydiinta PromQL - sum(rate(http_requests[2m])).
  3. Sida laga soo xigtay pollingInterval,KEDA waxay ka codsataa bartilmaameedka Prometheus shan iyo toban ilbiriqsi kasta. Ugu yaraan mid ka hooseeya (minReplicaCount), iyo tirada ugu badan ee galalka aan ka badnayn maxReplicaCount (tusaale ahaan - toban).

Ku rakibi kara minReplicaCount oo le'eg eber. Xaaladdan oo kale, KEDA waxay hawlgelisaa meelaynta eber-ka-hal ka dibna waxay soo bandhigtaa HPA si ay u miisaanto si toos ah. Nidaamka rogaal celiska ah sidoo kale waa suurtagal, taas oo ah, in la miisaamo hal ilaa eber. Tusaalaha, ma aanan dooran eber sababtoo ah kani waa adeeg HTTP ee maaha nidaam dalabaad ah.

Sixirka gudaha autoscaling

Meesha waxa loo isticmaalaa kicin ahaan si loo cabbiro geynta. Tusaalahayaga, weydiinta PromQL sum(rate (http_requests [2m])) waxay soo celisaa celceliska codsiga HTTP isku-darka ah (codsiyada ilbiriqsi kasta), oo la qiyaasay labadii daqiiqo ee u dambeeyay.

Maadaama qiimihiisu yahay saddex, waxay la macno tahay in uu mid hoos jiri doono inta uu qiimuhu socdo sum(rate (http_requests [2m])) in ka yar saddex. Haddii qiimuhu kordho, qayb dheeraad ah ayaa lagu daraa mar kasta sum(rate (http_requests [2m])) waxay ku kordhisaa saddex. Tusaale ahaan, haddii qiimihiisu yahay 12 ilaa 14, markaa tirada boodhku waa afar.

Hadda aan isku dayno dejinta!

habayn

Waxa kaliya ee aad u baahan tahay waa kutlada Kubernetes iyo utility habaysan kubectl. Tusaalahani waxa uu isticmaalayaa koox minikube, laakiin waxaad qaadan kartaa mid kasta oo kale. Si loo rakibo koox waxaa jira hage.

Ku rakib nooca ugu dambeeyay Mac:

curl -Lo minikube 
https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 
&& chmod +x minikube
sudo mkdir -p /usr/local/bin/
sudo install minikube /usr/local/bin/

Deji kubectlsi aad u gasho kooxda Kubernetes.

Ku rakib nooca ugu dambeeyay Mac:

curl -LO 
"https://storage.googleapis.com/kubernetes-release/release/$(curl -s
https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl"
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
kubectl version

rakibida KEDA

Waxaad u geyn kartaa KEDA dhowr siyaabood, waxay ku qoran yihiin dukumentiyo. Waxaan isticmaalayaa YAML monolithic:

kubectl apply -f
https://raw.githubusercontent.com/kedacore/keda/master/deploy/KedaScaleController.yaml

KEDA iyo qaybaheeda ayaa lagu rakibay meesha magaca keda. Amarka si loo hubiyo:

kubectl get pods -n keda

Sug inta uu hawl-wadeenka KEDA bilaabayo oo aad Running State. Oo intaas ka dib, sii wad.

Ku rakibida Redis adoo isticmaalaya Helm

Haddii aadan ku rakibin Helm, isticmaal kan hogaaminta. Amarka lagu rakibo Mac:

brew install kubernetes-helm
helm init --history-max 200

helm init wuxuu bilaabaa interface-ka khadka taliska wuxuuna sidoo kale rakibaa Tiller ilaa kooxda Kubernetes.

kubectl get pods -n kube-system | grep tiller

Sug inta uu tiller-ga soo galo gobolka Orodka.

Qoraalka turjubaanka: Qoraagu wuxuu isticmaalaa Helm@2, kaas oo u baahan in qaybta server-ka ee Tiller la rakibo. Hadda Helm@3 waa khuseeya, uma baahna qayb server ah.

Kadib rakibidda Helm, hal amar ayaa ku filan inuu bilaabo Redis:

helm install --name redis-server --set cluster.enabled=false --set 
usePassword=false stable/redis

Xaqiiji in Redis uu ku bilowday si guul leh:

kubectl get pods/redis-server-master-0

Sug in Redis uu galo gobolka Running.

Soo Dejinta Codsiga

Amarka geynta:

kubectl apply -f go-app.yaml

//output
deployment.apps/go-prom-app created
service/go-prom-app-service created

Hubi in wax walba ay bilaabeen:

kubectl get pods -l=app=go-prom-app

Sug inta uu Redis soo galo gobolka Running.

Gelinaya Server-ka Prometheus

Isticmaalka Prometheus Helitaanka Adeegga Kubernetes ee Prometheus. Waxay u ogolaataa helitaanka firfircooni ah ee sanduuqyada codsiga ee ku salaysan summada adeegga.

kubernetes_sd_configs:
   - role: service
   relabel_configs:
   - source_labels: [__meta_kubernetes_service_label_run]
     regex: go-prom-app-service
     action: keep

Si loo geeyo:

kubectl apply -f prometheus.yaml

//output
clusterrole.rbac.authorization.k8s.io/prometheus created
serviceaccount/default configured
clusterrolebinding.rbac.authorization.k8s.io/prometheus created
configmap/prom-conf created
deployment.extensions/prometheus-deployment created
service/prometheus-service created

Hubi in wax walba ay bilaabeen:

kubectl get pods -l=app=prometheus-server

Sug Prometheus si uu u galo gobolka Running.

Adeegso kubectl port-forward si aad u gasho is dhexgalka isticmaalaha Prometheus (ama server API) ee http://localhost:9090.

kubectl port-forward service/prometheus-service 9090

Gelinaya isku xidhka KEDA Autoscaling

Amarka abuurista ScaledObject:

kubectl apply -f keda-prometheus-scaledobject.yaml

Fiiri diiwaanka hawlwadeenka KEDA:

KEDA_POD_NAME=$(kubectl get pods -n keda 
-o=jsonpath='{.items[0].metadata.name}')
kubectl logs $KEDA_POD_NAME -n keda

Natiijadu waxay u egtahay sidan:

time="2019-10-15T09:38:28Z" level=info msg="Watching ScaledObject:
default/prometheus-scaledobject"
time="2019-10-15T09:38:28Z" level=info msg="Created HPA with 
namespace default and name keda-hpa-go-prom-app"

Hoos ka hubi codsiyada Hal tusaale waa inuu socdaa sababtoo ah minReplicaCount waxay la mid tahay 1:

kubectl get pods -l=app=go-prom-app

Xaqiiji in agabka HPA loo abuuray si guul leh:

kubectl get hpa

Waa inaad aragto wax sida:

NAME                   REFERENCE                TARGETS     MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-go-prom-app   Deployment/go-prom-app   0/3 (avg)   1         10        1          45s

Hubinta caafimaadka: gelitaanka codsiga

Si aad u gasho barta ugu dambaysa ee REST ee codsigeena, orod:

kubectl port-forward service/go-prom-app-service 8080

Hadda waxaad geli kartaa barnaamijkaaga Go adoo isticmaalaya ciwaanka http://localhost:8080. Si tan loo sameeyo, socodsii amarka:

curl http://localhost:8080/test

Natiijadu waxay u egtahay sidan:

Accessed on 2019-10-21 11:29:10.560385986 +0000 UTC 
m=+406004.817901246
Access count 1

Halkaa marka ay marayso sidoo kale hubi Redis. Waxaad arki doontaa in furaha access_count koray 1:

kubectl exec -it redis-server-master-0 -- redis-cli get access_count
//output
"1"

Hubi in qiimaha mitirku yahay http_requests isku mid:

curl http://localhost:8080/metrics | grep http_requests
//output
# HELP http_requests number of http requests
# TYPE http_requests counter
http_requests 1

Abuurista Xamuulka

Waan isticmaali doonaa hey - utility loogu talagalay abuurista rarka:

curl -o hey https://storage.googleapis.com/hey-release/hey_darwin_amd64 
&& chmod a+x hey

Waxa kale oo aad kala soo bixi kartaa utility for Linux ama Windows.

Orod

./hey http://localhost:8080/test

Sida caadiga ah, utility wuxuu soo diri 200 codsi. Waxaad tan ku xaqiijin kartaa adiga oo isticmaalaya metrik Prometheus iyo sidoo kale Redis.

curl http://localhost:8080/metrics | grep http_requests
//output
# HELP http_requests number of http requests
# TYPE http_requests counter
http_requests 201
kubectl exec -it redis-server-master-0 -- redis-cli get access_count
//output
201

Xaqiiji qiimaha mitirka dhabta ah (waxaa soo celiyay weydiinta PromQL):

curl -g 
'http://localhost:9090/api/v1/query?query=sum(rate(http_requests[2m]))'
//output
{"status":"success","data":{"resultType":"vector","result":[{"metric":{},"value":[1571734214.228,"1.686057971014493"]}]}}

Xaaladdan oo kale natiijada dhabta ah waa 1,686057971014493 waxaana lagu soo bandhigay garoonka dhexdiisa value. Tani kuma filna miisaanka, maadaama xadka aanu dejinay uu yahay 3.

Culays badan!

Terminalka cusub, la soco tirada sanduuqyada codsiga:

kubectl get pods -l=app=go-prom-app -w

Aan kordhino culeyska anagoo adeegsanayna amarka: 

./hey -n 2000 http://localhost:8080/test

Muddo ka dib, waxaad arki doontaa HPA oo cabbiraysa hawlgelinta oo bilaabaysa boodhyo cusub. Hubi HPA-gaaga si aad u hubiso:

kubectl get hpa
NAME                   REFERENCE                TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-go-prom-app   Deployment/go-prom-app   1830m/3 (avg)   1         10        6          4m22s

Haddii rarku aanu iswaafaqayn, hawlgelinta waa la dhimi doonaa ilaa meesha uu hal boodh oo keliya socdo. Haddii aad rabto inaad hubiso mitirka dhabta ah (waxaa soo celiyay weydiinta PromQL), ka dibna isticmaal amarka:

curl -g 
'http://localhost:9090/api/v1/query?query=sum(rate(http_requests[2m]))'

Nadiifinta

//Delete KEDA
kubectl delete namespace keda
//Delete the app, Prometheus server and KEDA scaled object
kubectl delete -f .
//Delete Redis
helm del --purge redis-server

gunaanad

KEDA waxay kuu ogolaanaysaa inaad si toos ah u cabbirto Kubernetes geyntaada (laga bilaabo eber) iyadoo lagu salaynayo xogta cabbirada dibadda. Tusaale ahaan, ku salaysan cabbirka Prometheus, dhererka safka ee Redis, daahitaanka macaamilka ee mawduuca Kafka.

KEDA waxay la midaysaa il dibadeed waxayna sidoo kale bixisaa cabbirkeeda iyada oo loo marayo Metrics Server ilaa Horizontal Pod Autoscaler.

nasiib wacan!

Maxaa kale oo la akhriyaa:

  1. Hababka ugu fiican iyo hababka ugu wanaagsan ee lagu socodsiiyo weelasha iyo Kubernetes ee deegaanka wax soo saarka.
  2. 90+ qalab waxtar u leh Kubernetes: geynta, maamulka, la socodka, amniga iyo in ka badan.
  3. Kanaalkayaga Kubernetes ee Telegram.

Source: www.habr.com

Add a comment