Phetolelo ea sengoloa e lokisitsoe bosiung ba pele thupelo e qala .
Mokhoa oa ho boloka litšenyehelo tsa maru ha u sebetsa le Kubernetes? Ha ho na tharollo e le 'ngoe e nepahetseng, empa sehlooho sena se hlalosa lisebelisoa tse' maloa tse tla u thusa ho laola lisebelisoa ka katleho le ho fokotsa litšenyehelo tsa hau tsa cloud computing.
Ke ngotse sengoloa sena ke nahanne ka Kubernetes bakeng sa AWS, empa se tla sebetsa (hoo e batlang e le) ka ho lekana ho bafani ba bang ba maru. Ke nahana hore lihlopha tsa hau li se li ntse li e-na le autoscaling e hlophisitsoeng (). Ho tlosa lisebelisoa le ho theola thepa ea hau ho tla u bolokela chelete feela haeba ho boetse ho fokotsa likepe tsa hau tsa basebetsi (EC2 mohlala).
Sengoliloeng sena se tla akaretsa:
- ho hloekisa lisebelisoa tse sa sebelisoeng ()
- Ho fokotsa boima ba 'mele nakong ea mosebetsi ()
- sebelisa horizontal profiling (HPA),
- ho fokotsa phano e feteletseng ea lisebelisoa (, VPA)
- ho sebelisa mohlala oa Spot
Ho hloekisa lisebelisoa tse sa sebelisoeng
Ho sebetsa sebakeng se potlakileng ho monate. Re batla mekhatlo ea theknoloji ho . Phatlalatso e potlakileng ea software e boetse e bolela ho romelloa ho eketsehileng ho PR, tikoloho ea ponelopele, li-prototypes le litharollo tsa analytics. Tsohle li rometsoe ho Kubernetes. Ke mang ea nang le nako ea ho hloekisa lisebelisoa tsa liteko? Ho bonolo ho lebala ho hlakola teko ea beke. Leru la leru le tla hola qetellong hobane re lebetse ho koala:
(Henning Jacobs:
Bophelo:
(litlhaloso tse qotsitsoeng) Corey Quinn:
Khopolo-taba: Bili ea hau ea AWS ke ts'ebetso ea palo ea basebelisi bao u nang le bona.
'Nete: Bili ea hau ea AWS ke ts'ebetso ea palo ea lienjineri tseo u nang le tsona.
Ivan Kurnosov (ka karabo):
'Nete ea' nete: Bili ea hau ea AWS ke ts'ebetso ea hore na u lebetse lintho tse kae ho thibela / ho hlakola.)
(kube-janitor) e thusa ho hloekisa sehlopha sa hau. Tlhophiso ea janitor e fetoha habonolo bakeng sa ts'ebeliso ea lefats'e le ea lehae:
- Melao e pharalletseng ea Cluster e ka hlalosa nako e telele ea ho phela (TTL) bakeng sa phepelo ea PR/teko.
- Lisebelisoa tsa motho ka mong li ka hlalosoa ka janitor/ttl, ho etsa mohlala ho tlosa spike / prototype ka mor'a matsatsi a 7.
Melao e akaretsang e hlalosoa faeleng ea YAML. Tsela ea eona e fetisoa ka parameter --rules-file ka kube-janitor. Mona ke molao oa mohlala oa ho tlosa libaka tsohle tsa mabitso ka -pr- ka lebitso ka mor'a matsatsi a mabeli:
- id: cleanup-resources-from-pull-requests
resources:
- namespaces
jmespath: "contains(metadata.name, '-pr-')"
ttl: 2dMohlala o latelang o laela tšebeliso ea leibole ea kopo ho Deployment le StatefulSet pods bakeng sa Deployments/StatefulSets tsohle tse ncha ka 2020, empa ka nako e ts'oanang e lumella liteko ho sebetsa ntle le leibole ena bakeng sa beke:
- id: require-application-label
# удалить deployments и statefulsets без метки "application"
resources:
- deployments
- statefulsets
# см. http://jmespath.org/specification.html
jmespath: "!(spec.template.metadata.labels.application) && metadata.creationTimestamp > '2020-01-01'"
ttl: 7dMatha demo e lekanyelitsoeng ea nako bakeng sa metsotso e 30 sehlopheng se tsamaisang kube-janitor:
kubectl run nginx-demo --image=nginx
kubectl annotate deploy nginx-demo janitor/ttl=30mMohloli o mong oa litšenyehelo tse ntseng li hola ke li-persistent volumes (AWS EBS). Ho hlakola Kubernetes StatefulSet ha ho hlakole meqolo ea eona e phehellang (PVC - PersistentVolumeClaim). Li-volume tsa EBS tse sa sebelisoeng li ka bitsa liranta tse makholo ka khoeli. Kubernetes Janitor e na le tšobotsi ea ho hloekisa li-PVC tse sa sebelisoeng. Mohlala, molao ona o tla hlakola li-PVC tsohle tse sa kenngoeng ke pod mme li sa boleloa ke StatefulSet kapa CronJob:
# удалить все PVC, которые не смонтированы и на которые не ссылаются StatefulSets
- id: remove-unused-pvcs
resources:
- persistentvolumeclaims
jmespath: "_context.pvc_is_not_mounted && _context.pvc_is_not_referenced"
ttl: 24hKubernetes Janitor e ka u thusa ho boloka sehlopha sa hau se hloekile le ho thibela butle-butle ho bokella litšenyehelo tsa cloud computing. Bakeng sa litaelo tsa phepelo le tlhophiso, latela .
Ho fokotsa boima ba 'mele nakong ea lihora tsa mosebetsi
Sistimi ea liteko le ea sethala hangata e hlokoa feela ho sebetsa nakong ea lihora tsa khoebo. Lisebelisoa tse ling tsa tlhahiso, joalo ka lisebelisoa tsa back office/admin, le tsona li hloka ho fumaneha ho fokolang feela 'me li ka tingoa bosiu.
(kube-downscaler) e lumella basebelisi le basebelisi ho theola sistimi nakong ea lihora tse sa sebetseng. Li-Deployments le StatefulSets li ka fokotsoa ho fihla ho zero replicas. CronJobs e ka emisoa. Kubernetes Downscaler e ka hlophisoa bakeng sa sehlopha sohle, sebaka se le seng kapa ho feta, kapa lisebelisoa tsa motho ka mong. E ka ba "nako ea ho phomola" kapa "nako ea ho qetela" e ka behoa. Mohlala, ho theola tlase kamoo ho ka khonehang bosiu le mafelo-beke:
image: hjacobs/kube-downscaler:20.4.3
args:
- --interval=30
# не отключать компоненты инфраструктуры
- --exclude-namespaces=kube-system,infra
# не отключать kube-downscaler, а также оставить Postgres Operator, чтобы исключенными БД можно было управлять
- --exclude-deployments=kube-downscaler,postgres-operator
- --default-uptime=Mon-Fri 08:00-20:00 Europe/Berlin
- --include-resources=deployments,statefulsets,stacks,cronjobs
- --deployment-time-annotation=deployment-timeMona ke kemiso ea kluster worker node scaling mafelong a beke:
Ho theola ho tloha ho ~ 13 node ea basebetsi ho ea ho 4 ea basebetsi ka sebele ho etsa phapang e bonahalang bili ea hau ea AWS.
Empa ho thoe'ng haeba ke hloka ho sebetsa nakong ea "downtime" ea sehlopha? Lisebelisoa tse ling li ka qheleloa ka thoko ho ea ho sekala ka ho eketsa theolelo/ho qhelela ka thoko: tlhaloso ea 'nete. Ho romelloa ho ka qheleloa ka thoko ka nakoana ho sebelisoa sehatisi se tlase-tlase-ho fihlela tlhaloso e nang le setempe sa nako se felletseng ka sebopeho sa YYYY-MM-DD HH:MM (UTC). Haeba ho hlokahala, sehlopha sohle se ka theoleloa morao ka ho kenya pod e nang le tlhaloso downscaler/force-uptime, mohlala, ka ho sebelisa nginx e se nang letho:
kubectl run scale-up --image=nginx
kubectl annotate deploy scale-up janitor/ttl=1h # удалить развертывание через час
kubectl annotate pod $(kubectl get pod -l run=scale-up -o jsonpath="{.items[0].metadata.name}") downscaler/force-uptime=trueBona , haeba u thahasella litaelo tsa ho romelloa le likhetho tse eketsehileng.
Sebelisa autoscaling e otlolohileng
Lisebelisoa / lits'ebeletso tse ngata li sebetsana le mokhoa o matla oa mojaro: ka linako tse ling li-pods tsa tsona ha li sebetse, 'me ka linako tse ling li sebetsa ka botlalo. Ho tsamaisa sehlopha se sa khaotseng sa li-pods ho sebetsana le moroalo o moholo oa tlhoro ha ho na litšenyehelo. Kubernetes e tšehetsa autoscaling e otlolohileng ka sesebelisoa (HPA) Tšebeliso ea CPU hangata ke metric e ntle bakeng sa ho lekanya:
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: my-app
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: my-app
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
averageUtilization: 100
type: UtilizationZalando e thehile karolo ea ho hokahanya metrics ea tloaelo habonolo bakeng sa ho lekanya: (kube-metrics-adapter) ke adaptara ea metrics ea sepheo se akaretsang bakeng sa Kubernetes e ka bokellang le ho fana ka metrics ea tloaelo le ea kantle bakeng sa ho otloloha ha pod autoscaling. E tšehetsa sekhahla se ipapisitseng le metrics ea Prometheus, mela ea SQS le litlhophiso tse ling. Mohlala, ho lekanya phetisetso bakeng sa metric ea tloaelo e hlahisitsoeng ke sesebelisoa ka bosona joalo ka JSON ho /metrics , sebelisa:
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: myapp-hpa
annotations:
# metric-config.<metricType>.<metricName>.<collectorName>/<configKey>
metric-config.pods.requests-per-second.json-path/json-key: "$.http_server.rps"
metric-config.pods.requests-per-second.json-path/path: /metrics
metric-config.pods.requests-per-second.json-path/port: "9090"
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: myapp
minReplicas: 1
maxReplicas: 10
metrics:
- type: Pods
pods:
metric:
name: requests-per-second
target:
averageValue: 1k
type: AverageValueHo theha autoscaling e otlolohileng le HPA e lokela ho ba e 'ngoe ea liketso tsa kamehla ho ntlafatsa ts'ebetso ea lits'ebeletso tse se nang naha. Spotify e na le tlhahiso mabapi le boiphihlelo le likhothaletso tsa HPA: .
Ho fokotsa tšebeliso e feteletseng ea lisebelisoa
Mesebetsi ea Kubernetes e totobatsa litlhoko tsa bona tsa CPU / memori ka "likopo tsa lisebelisoa." Lisebelisoa tsa CPU li lekantsoe ka li-cores kapa hangata "millicores," mohlala. 500m e bolela 50% ea vCPU. Lisebelisoa tsa memori li lekanngoa ka li-byte, 'me li ka sebelisa li-suffixes tse tloaelehileng, mohlala. 500Mi, e bolelang 500 megabytes. Likopo tsa lisebelisoa tsa "notlolo" ho li-node tsa basebetsi, ke hore, pod e nang le kopo ea CPU ea 1000m ho node e nang le 4 vCPU e tla siea li-vCPU tse 3 feela li fumaneha ho li-pods tse ling.
Slack (sebaka sa ho phalla) - ke phapang lipakeng tsa lisebelisoa tse kopiloeng le ts'ebeliso ea 'nete. Mohlala, pod e kopang 2 GiB ea mohopolo empa e sebelisa 200 MiB feela e na le ~ 1,8 GiB ea memori e "feteletseng". Ho feta tekano ho bitsa chelete. Khakanyo e mpe ke hore 1 GiB ea memori e feteletseng e bitsa ~ $ 10 ka khoeli.
(kube-resource-report) e bonts'a mehloli e mengata mme e ka u thusa ho tseba bokhoni ba ho boloka:
e bonts'a phaello e kopantsoeng ka kopo le sehlopha. Sena se o lumella ho fumana libaka tseo likopo tsa lisebelisoa li ka fokotsoang ho tsona. Tlaleho ea HTML e hlahisitsoeng e fana feela ka setšoantšo sa tšebeliso ea lisebelisoa. U lokela ho sheba ts'ebeliso ea CPU / memori ha nako e ntse e feta ho fumana likopo tse lekaneng tsa lisebelisoa. Mona ke chate ea Grafana bakeng sa ts'ebeletso e "tloaelehileng" ea CPU-boima: li-pods kaofela li sebelisa li-CPU tse ka tlase ho tse 3 tse kopiloeng:
Ho fokotsa kopo ea CPU ho tloha ho 3000m ho isa ~ 400m ho lokolla lisebelisoa bakeng sa meroalo e meng ea mosebetsi mme ho lumella sehlopha ho ba nyane.
"Karolelano ea tšebeliso ea CPU ea maemo a EC2 hangata e feto-fetoha ho karolo ea palo e le 'ngoe," . Nakong ea EC2 , ho fetola likopo tse ling tsa lisebelisoa tsa Kubernetes faeleng ea YAML ho bonolo ebile ho ka boloka chelete e ngata.
Empa na ehlile re batla hore batho ba fetole boleng ho lifaele tsa YAML? Che, mechini e ka e etsa hamolemonyana! Kubernetes (VPA) e etsa joalo feela: ho fetola likopo tsa lisebelisoa le meeli ho mojaro oa mosebetsi. Mona ke mohlala oa likopo tsa Prometheus CPU (mohala o mosesaane o moputsoa) o fetotsoeng ke VPA ha nako e ntse e ea:
bakeng sa likarolo tsa motheo. Lisebelisoa tse seng tsa bohlokoa le tsona li ka sebelisa VPA.
by Fairwind ke sesebelisoa se etsang VPA bakeng sa thomello ka 'ngoe sebakeng sa mabitso ebe se bonts'a khothaletso ea VPA ka har'a dashboard ea eona. E ka thusa bahlahisi ho theha likopo tse nepahetseng tsa CPU / memori bakeng sa lits'ebetso tsa bona:
Ke ngotse e nyane ka 2019, le haufinyane tjena .
Ho sebelisa EC2 Spot Instances
Qetellong, litšenyehelo tsa AWS EC2 li ka fokotsoa ka ho sebelisa maemo a Spot joalo ka li-node tsa basebetsi ba Kubernetes. . Litheko tsa Spot li fumaneha ka litheolelo tse fihlang ho 90% ha li bapisoa le litheko tse batloang feela. Running Kubernetes ho EC2 Spot ke motsoako o motle: o ka hlakisa mefuta e fapaneng ea mehlala bakeng sa phumaneho e phahameng, ho bolelang hore o ka fumana node e kholoanyane ka theko e tšoanang kapa e tlase, mme matla a eketsehileng a ka sebelisoa ke mojaro oa mosebetsi oa Kubernetes.
U tsamaisa Kubernetes joang ho EC2 Spot? Ho na le likhetho tse 'maloa: sebelisa ts'ebeletso ea mokha oa boraro joalo ka SpotInst (eo hona joale e bitsoang "Spot," u se mpotse hore na hobaneng), kapa eketsa Spot AutoScalingGroup (ASG) sehlopheng sa hau. Mohlala, mona ke snippet ea CloudFormation bakeng sa "Spot ASG e ntlafalitsoeng" e nang le mefuta e mengata ea mehlala:
MySpotAutoScalingGroup:
Properties:
HealthCheckGracePeriod: 300
HealthCheckType: EC2
MixedInstancesPolicy:
InstancesDistribution:
OnDemandPercentageAboveBaseCapacity: 0
SpotAllocationStrategy: capacity-optimized
LaunchTemplate:
LaunchTemplateSpecification:
LaunchTemplateId: !Ref LaunchTemplate
Version: !GetAtt LaunchTemplate.LatestVersionNumber
Overrides:
- InstanceType: "m4.2xlarge"
- InstanceType: "m4.4xlarge"
- InstanceType: "m5.2xlarge"
- InstanceType: "m5.4xlarge"
- InstanceType: "r4.2xlarge"
- InstanceType: "r4.4xlarge"
LaunchTemplate:
LaunchTemplateId: !Ref LaunchTemplate
Version: !GetAtt LaunchTemplate.LatestVersionNumber
MinSize: 0
MaxSize: 100
Tags:
- Key: k8s.io/cluster-autoscaler/node-template/label/aws.amazon.com/spot
PropagateAtLaunch: true
Value: "true"Lintlha tse ling mabapi le ho sebelisa Spot le Kubernetes:
- U hloka ho sebetsana le ho felisoa ha Spot, mohlala, ka ho hula node ha mohlala o emisa
- Zalando e sebelisa semmuso cluster autoscaling le node pool prioritiesl
- Spot Nodes amohela moroalo oa mosebetsi "ngodiso" ho sebetsa sebakeng sa Spot
Kakaretso
Ke tšepa hore u tla fumana tse ling tsa lisebelisoa tse hlahisitsoeng li le molemo ho fokotsa sekoloto sa hau sa maru. U ka fumana boholo ba litaba tsa sengoloa hape ho .
Mekhoa ea hau e metle ea ho boloka litšenyehelo tsa maru ho Kubernetes ke efe? Ke kopa o ntsebise .
Ha e le hantle, li-vCPU tse ka tlase ho 3 li tla lula li sebelisoa ha tlhahiso ea node e fokotsoa ke mehloli ea tsamaiso e bolokiloeng. Kubernetes e khetholla pakeng tsa matla a 'mele a node le lisebelisoa "tse fanoeng" ().
Mohlala oa palo: mohlala o le mong oa m5. o moholo o nang le 8 GiB ea memori ke ~ 84 USD / khoeli (eu-central-1, On-Demand), ke hore ho koala 1/8 ea node ke hoo e ka bang ~ 10 USD / khoeli.
Ho na le mekhoa e meng e mengata ea ho fokotsa sekoloto sa hau sa EC2, joalo ka maemo a boloketsoeng, moralo oa ho boloka chelete, joalo-joalo - Nke ke ka bua ka lihlooho tsena mona, empa ka sebele u lokela ho li sheba!
Source: www.habr.com
