ProHoster > Π‘Π»ΠΎΠ³ > Administrasi > Aplikasi Autoscaling Kubernetes nggunakake Prometheus lan KEDA

Aplikasi Autoscaling Kubernetes nggunakake Prometheus lan KEDA

Aplikasi Autoscaling Kubernetes nggunakake Prometheus lan KEDABalon Man dening Cimuanos

Skalabilitas minangka syarat utama kanggo aplikasi awan. Kanthi Kubernetes, skala aplikasi gampang kaya nambah jumlah replika kanggo panyebaran sing cocog utawa ReplicaSet - nanging proses manual.

Kubernetes ngidini aplikasi kanthi otomatis skala (yaiku Pods ing panyebaran utawa ReplicaSet) kanthi cara deklaratif nggunakake spesifikasi Horizontal Pod Autoscaler. Kriteria standar kanggo skala otomatis yaiku metrik panggunaan CPU (metrik sumber daya), nanging sampeyan bisa nggabungake metrik khusus lan eksternal sing kasedhiya.

tim Kubernetes aaS saka Mail.ru nerjemahake artikel babagan carane nggunakake metrik eksternal kanggo ukuran aplikasi Kubernetes kanthi otomatis. Kanggo nuduhake cara kerjane kabeh, penulis nggunakake metrik panjalukan akses HTTP, sing diklumpukake nggunakake Prometheus.

Tinimbang skala otomatis polong, Kubernetes Event Driven Autoscaling (KEDA) digunakake, operator Kubernetes open source. Iki nggabungake native karo Horizontal Pod Autoscaler kanggo nyedhiyakake autoscaling sing lancar (kalebu menyang / saka nol) kanggo beban kerja sing didorong acara. Kode kasedhiya ing GitHub.

Ringkesan Brief saka sistem

Aplikasi Autoscaling Kubernetes nggunakake Prometheus lan KEDA

Diagram kasebut nuduhake katrangan ringkes babagan cara kerjane:

  1. Aplikasi kasebut nyedhiyakake metrik jumlah hit HTTP ing format Prometheus.
  2. Prometheus dikonfigurasi kanggo ngumpulake metrik kasebut.
  3. Prometheus scaler ing KEDA dikonfigurasi kanthi otomatis skala aplikasi adhedhasar jumlah HTTP hit.

Saiki aku bakal ngandhani kanthi rinci babagan saben unsur.

KEDA lan Prometheus

Prometheus minangka alat ngawasi lan waspada sistem open source, bagean Yayasan Komputasi Native Cloud. Nglumpukake metrik saka macem-macem sumber lan simpen minangka data seri wektu. Kanggo nggambarake data sing bisa digunakake Grafana utawa alat visualisasi liyane sing bisa digunakake karo API Kubernetes.

KEDA ndhukung konsep scaler - iku minangka jembatan antarane KEDA lan sistem eksternal. Implementasi scaler khusus kanggo saben sistem target lan ngekstrak data saka iku. KEDA banjur digunakake kanggo ngontrol skala otomatis.

Scaler ndhukung macem-macem sumber data, contone, Kafka, Redis, Prometheus. Tegese, KEDA bisa digunakake kanggo skala penyebaran Kubernetes kanthi otomatis nggunakake metrik Prometheus minangka kritΓ©ria.

Aplikasi tes

Aplikasi tes Golang nyedhiyakake akses liwat HTTP lan nindakake rong fungsi penting:

  1. Migunakake perpustakaan klien Prometheus Go kanggo instrumen aplikasi lan nyedhiyakake metrik http_requests, sing ngemot jumlah hit. Titik pungkasan ing ngendi metrik Prometheus kasedhiya ing URI /metrics. 
    var httpRequestsCounter = promauto.NewCounter(prometheus.CounterOpts{
       Name: "http_requests",
       Help: "number of http requests",
   })
  2. Nanggepi panjalukan GET aplikasi nambah nilai kunci (access_count) ing Redis. Iki minangka cara sing gampang kanggo nindakake pakaryan minangka bagean saka panangan HTTP lan uga mriksa metrik Prometheus. Nilai metrik kudu padha karo nilai access_count ing Redis. 
    func main() {
       http.Handle("/metrics", promhttp.Handler())
       http.HandleFunc("/test", func(w http.ResponseWriter, r 
*http.Request) {
           defer httpRequestsCounter.Inc()
           count, err := client.Incr(redisCounterName).Result()
           if err != nil {
               fmt.Println("Unable to increment redis counter", err)
               os.Exit(1)
           }
           resp := "Accessed on " + time.Now().String() + "nAccess count " + strconv.Itoa(int(count))
           w.Write([]byte(resp))
       })
       http.ListenAndServe(":8080", nil)
   }

Aplikasi kasebut disebarake menyang Kubernetes liwat Deployment. Layanan uga digawe ClusterIP, ngidini server Prometheus entuk metrik aplikasi.

kene panyebaran manifest kanggo aplikasi.

Server Prometheus

Manifes penyebaran Prometheus kalebu:

  • ConfigMap - kanggo nransfer konfigurasi Prometheus;
  • Deployment - kanggo nyebarake Prometheus ing kluster Kubernetes;
  • ClusterIP - layanan kanggo akses menyang UI Prometheus;
  • ClusterRole, ClusterRoleBinding ΠΈ ServiceAccount β€” kanggo deteksi otomatis layanan ing Kubernetes (Otomatis panemuan).

kene manifest kanggo mlaku Prometheus.

KEDA Prometheus ScaledObject

Scaler minangka jembatan antarane KEDA lan sistem eksternal sing kudu dipikolehi metrik. ScaledObject minangka sumber daya khusus sing kudu disebarake kanggo nyinkronake penyebaran karo sumber acara, ing kasus iki Prometheus.

ScaledObject ngemot informasi skala penyebaran, metadata sumber acara (kayata rahasia sambungan, jeneng antrian), interval polling, periode pemulihan, lan data liyane. Iki nyebabake sumber autoscaling sing cocog (definisi HPA) kanggo skala penyebaran.

Nalika obyek ScaledObject wis dibusak, definisi HPA cocog dibusak.

Punika definisi ScaledObject contone, nggunakake scaler Prometheus:

apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
 name: prometheus-scaledobject
 namespace: default
 labels:
   deploymentName: go-prom-app
spec:
 scaleTargetRef:
   deploymentName: go-prom-app
 pollingInterval: 15
 cooldownPeriod:  30
 minReplicaCount: 1
 maxReplicaCount: 10
 triggers:
 - type: prometheus
   metadata:
     serverAddress: 
http://prometheus-service.default.svc.cluster.local:9090
     metricName: access_frequency
     threshold: '3'
     query: sum(rate(http_requests[2m]))

Coba titik ing ngisor iki:

  1. Dheweke nuduhake Deployment Kanthi jeneng go-prom-app.
  2. Tipe pemicu - Prometheus. Alamat server Prometheus kasebut bebarengan karo jeneng metrik, ambang lan pitakon PromQL, sing bakal digunakake. Pitakon PromQL - sum(rate(http_requests[2m])).
  3. Miturut pollingInterval,KEDA njaluk target saka Prometheus saben limalas detik. Paling ora siji (minReplicaCount), lan jumlah maksimum polong ora ngluwihi maxReplicaCount (ing conto iki - sepuluh).

Bisa diinstal minReplicaCount padha karo nul. Ing kasus iki, KEDA ngaktifake panyebaran nol-kanggo-siji lan banjur mbukak HPA kanggo skala otomatis luwih lanjut. Urutan mbalikke uga bisa, yaiku, skala saka siji nganti nol. Ing conto, kita ora milih nol amarga iki layanan HTTP lan dudu sistem on-demand.

Piandel ing autoscaling

Ambang digunakake minangka pemicu kanggo skala penyebaran. Ing conto kita, pitakon PromQL sum(rate (http_requests [2m])) ngasilake tingkat panjalukan HTTP sing dikumpulake (panjaluk per detik), diukur sajrone rong menit pungkasan.

Wiwit nilai ambang telu, tegese bakal ana siji ing ngisor nalika nilai kasebut sum(rate (http_requests [2m])) kurang telu. Yen regane mundhak, sub tambahan ditambahake saben wektu sum(rate (http_requests [2m])) mundhak telu. Contone, yen nilai saka 12 kanggo 14, banjur nomer pods papat.

Saiki ayo nyoba nyetel!

prasetel

Kabeh sing dibutuhake yaiku kluster Kubernetes lan sarana sing dikonfigurasi kubectl. Conto iki nggunakake kluster minikube, nanging sampeyan bisa njupuk liyane. Kanggo nginstal cluster ana nuntun.

Instal versi paling anyar ing Mac:

curl -Lo minikube 
https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64 
&& chmod +x minikube
sudo mkdir -p /usr/local/bin/
sudo install minikube /usr/local/bin/

Instal kubectlkanggo ngakses kluster Kubernetes.

Instal versi paling anyar ing Mac:

curl -LO 
"https://storage.googleapis.com/kubernetes-release/release/$(curl -s
https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/darwin/amd64/kubectl"
chmod +x ./kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
kubectl version

instalasi KEDA

Sampeyan bisa nyebarke KEDA kanthi pirang-pirang cara, padha kadhaptar ing dokumentasi. Aku nggunakake YAML monolitik:

kubectl apply -f
https://raw.githubusercontent.com/kedacore/keda/master/deploy/KedaScaleController.yaml

KEDA lan komponene dipasang ing ruang jeneng keda. Perintah kanggo mriksa:

kubectl get pods -n keda

Ngenteni Operator KEDA miwiti lan pindhah menyang Running State. Lan sawise iku, terus.

Nginstal Redis nggunakake Helm

Yen sampeyan ora duwe Helm diinstal, gunakake iki kepemimpinan. Printah kanggo nginstal ing Mac:

brew install kubernetes-helm
helm init --history-max 200

helm init initializes antarmuka baris printah lokal lan uga nginstal Tiller menyang kluster Kubernetes.

kubectl get pods -n kube-system | grep tiller

Ngenteni pod Tiller mlebu ing status Running.

Cathetan penerjemah: Penulis nggunakake Helm@2, sing mbutuhake komponen server Tiller diinstal. Saiki Helm@3 cocog, ora mbutuhake bagean server.

Sawise nginstal Helm, siji printah cukup kanggo miwiti Redis:

helm install --name redis-server --set cluster.enabled=false --set 
usePassword=false stable/redis

Verifikasi manawa Redis wis sukses:

kubectl get pods/redis-server-master-0

Ngenteni Redis menyang negara Running.

Panyebaran Aplikasi

Perintah penyebaran:

kubectl apply -f go-app.yaml

//output
deployment.apps/go-prom-app created
service/go-prom-app-service created

Priksa manawa kabeh wis diwiwiti:

kubectl get pods -l=app=go-prom-app

Ngenteni Redis mlebu negara Running.

Nggunakake Server Prometheus

The Prometheus manifest nggunakake Penemuan Layanan Kubernetes kanggo Prometheus. Iki ngidini panemuan dinamis pod aplikasi adhedhasar label layanan.

kubernetes_sd_configs:
   - role: service
   relabel_configs:
   - source_labels: [__meta_kubernetes_service_label_run]
     regex: go-prom-app-service
     action: keep

Kanggo nyebarake:

kubectl apply -f prometheus.yaml

//output
clusterrole.rbac.authorization.k8s.io/prometheus created
serviceaccount/default configured
clusterrolebinding.rbac.authorization.k8s.io/prometheus created
configmap/prom-conf created
deployment.extensions/prometheus-deployment created
service/prometheus-service created

Priksa manawa kabeh wis diwiwiti:

kubectl get pods -l=app=prometheus-server

Ngenteni Prometheus menyang negara Running.

Gunakake kubectl port-forward kanggo ngakses antarmuka panganggo Prometheus (utawa server API) ing http://localhost:9090.

kubectl port-forward service/prometheus-service 9090

Nggunakake Konfigurasi Autoscaling KEDA

Dhawuh kanggo nggawe ScaledObject:

kubectl apply -f keda-prometheus-scaledobject.yaml

Priksa log operator KEDA:

KEDA_POD_NAME=$(kubectl get pods -n keda 
-o=jsonpath='{.items[0].metadata.name}')
kubectl logs $KEDA_POD_NAME -n keda

Asil katon kaya iki:

time="2019-10-15T09:38:28Z" level=info msg="Watching ScaledObject:
default/prometheus-scaledobject"
time="2019-10-15T09:38:28Z" level=info msg="Created HPA with 
namespace default and name keda-hpa-go-prom-app"

Priksa ing ngisor aplikasi. Siji conto kudu mlaku amarga minReplicaCount padha karo 1:

kubectl get pods -l=app=go-prom-app

Verifikasi manawa sumber daya HPA kasil digawe:

kubectl get hpa

Sampeyan kudu ndeleng kaya:

NAME                   REFERENCE                TARGETS     MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-go-prom-app   Deployment/go-prom-app   0/3 (avg)   1         10        1          45s

Priksa kesehatan: akses aplikasi

Kanggo ngakses titik pungkasan REST aplikasi kita, jalanake:

kubectl port-forward service/go-prom-app-service 8080

Sampeyan saiki bisa ngakses aplikasi Go nggunakake alamat kasebut http://localhost:8080. Kanggo nindakake iki, jalanake printah:

curl http://localhost:8080/test

Asil katon kaya iki:

Accessed on 2019-10-21 11:29:10.560385986 +0000 UTC 
m=+406004.817901246
Access count 1

Ing titik iki uga mriksa Redis. Sampeyan bakal weruh sing tombol access_count tambah dadi 1:

kubectl exec -it redis-server-master-0 -- redis-cli get access_count
//output
"1"

Priksa manawa nilai metrik kasebut http_requests padha:

curl http://localhost:8080/metrics | grep http_requests
//output
# HELP http_requests number of http requests
# TYPE http_requests counter
http_requests 1

Nggawe Muatan

Kita bakal nggunakake hey - utilitas kanggo ngasilake beban:

curl -o hey https://storage.googleapis.com/hey-release/hey_darwin_amd64 
&& chmod a+x hey

Sampeyan uga bisa ngundhuh sarana kanggo Linux utawa Windows.

Jalanake:

./hey http://localhost:8080/test

Kanthi gawan, sarana ngirim 200 panjalukan. Sampeyan bisa verifikasi iki nggunakake metrik Prometheus uga Redis.

curl http://localhost:8080/metrics | grep http_requests
//output
# HELP http_requests number of http requests
# TYPE http_requests counter
http_requests 201
kubectl exec -it redis-server-master-0 -- redis-cli get access_count
//output
201

Validasi nilai metrik nyata (dibalekake dening pitakon PromQL):

curl -g 
'http://localhost:9090/api/v1/query?query=sum(rate(http_requests[2m]))'
//output
{"status":"success","data":{"resultType":"vector","result":[{"metric":{},"value":[1571734214.228,"1.686057971014493"]}]}}

Ing kasus iki, asil nyata 1,686057971014493 lan ditampilake ing lapangan value. Iki ora cukup kanggo skala, amarga batesan sing disetel yaiku 3.

More beban!

Ing terminal anyar, monitor jumlah pod aplikasi:

kubectl get pods -l=app=go-prom-app -w

Ayo nambah beban nggunakake printah: 

./hey -n 2000 http://localhost:8080/test

Sawise sawetara wektu, sampeyan bakal weruh HPA nggawe skala penyebaran lan ngluncurake pod anyar. Priksa HPA kanggo mesthekake:

kubectl get hpa
NAME                   REFERENCE                TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
keda-hpa-go-prom-app   Deployment/go-prom-app   1830m/3 (avg)   1         10        6          4m22s

Yen beban ora konsisten, penyebaran bakal dikurangi nganti mung siji pod sing mlaku. Yen sampeyan pengin mriksa metrik nyata (dibalekake dening pitakon PromQL), banjur gunakake printah:

curl -g 
'http://localhost:9090/api/v1/query?query=sum(rate(http_requests[2m]))'

Reresik

//Delete KEDA
kubectl delete namespace keda
//Delete the app, Prometheus server and KEDA scaled object
kubectl delete -f .
//Delete Redis
helm del --purge redis-server

kesimpulan

KEDA ngidini sampeyan nggedhekake penyebaran Kubernetes kanthi otomatis (menyang/saka nol) adhedhasar data saka metrik eksternal. Contone, adhedhasar metrik Prometheus, dawa antrian ing Redis, latensi konsumen ing topik Kafka.

KEDA nggabungake karo sumber eksternal lan uga nyedhiyakake metrik liwat Server Metrik menyang Horizontal Pod Autoscaler.

Good luck!

Apa maneh sing kudu diwaca:

  1. Praktik paling apik lan praktik paling apik kanggo mbukak wadhah lan Kubernetes ing lingkungan produksi.
  2. 90+ alat sing migunani kanggo Kubernetes: panyebaran, manajemen, pemantauan, keamanan lan liya-liyane.
  3. Saluran kita Sekitar Kubernetes ing Telegram.

Source: www.habr.com

Add a comment