ʻO ke kumu o ka ʻatikala e hoʻolauna i ka mea heluhelu i nā kumu o ka ʻoihana pūnaewele a me ka hoʻokele ʻana i nā kulekele ʻoihana ma Kubernetes, a me ka plugin Calico ʻaoʻao ʻekolu e hoʻonui i nā mana maʻamau. Ma ke ala, e hōʻike ʻia ka maʻalahi o ka hoʻonohonoho ʻana a me kekahi mau hiʻohiʻona me ka hoʻohana ʻana i nā hiʻohiʻona maoli mai kā mākou ʻike hana.
ʻO kahi hoʻolauna wikiwiki i ka mea hoʻohana pūnaewele Kubernetes
ʻAʻole hiki ke noʻonoʻo ʻia kahi pūʻulu Kubernetes me ka ʻole o kahi pūnaewele. Ua hoʻopuka mua mākou i nā mea ma kā lākou kumu: ""A"".
Ma ka pōʻaiapili o kēia ʻatikala, he mea nui e hoʻomaopopo ʻaʻole kuleana ʻo K8 iā ia iho no ka pilina pūnaewele ma waena o nā ipu a me nā nodes: no kēia, nā ʻano like ʻole. CNI plugins (Inoa Pūnaewele Pūnaewele). ʻOi aku e pili ana i kēia manaʻo mākou .
No ka laʻana, ʻo ka mea maʻamau o kēia mau plugins - hāʻawi i ka hoʻohui pūnaewele piha ma waena o nā pūnana puʻupuʻu a pau ma ka hoʻokiʻekiʻe ʻana i nā alahaka ma kēlā me kēia node, e hāʻawi ana i kahi subnet iā ia. Eia naʻe, ʻaʻole maikaʻi ka loaʻa piha a me ka hoʻoponopono ʻole ʻia. No ka hoʻolako ʻana i kekahi ʻano kaʻawale liʻiliʻi i loko o ka puʻupuʻu, pono e komo i ka hoʻonohonoho ʻana o ka pā ahi. Ma ka hihia maʻamau, waiho ʻia ia ma lalo o ka mana o ka CNI like, ʻo ia ke kumu e hiki ai ke unuhi hewa ʻole a nānā ʻole ʻia nā hana a nā ʻaoʻao ʻekolu i nā iptables.
A "ma waho o ka pahu" no ka hoʻonohonoho ʻana i ka hoʻokele kulekele ʻoihana ma kahi hui Kubernetes i hāʻawi ʻia . Hiki ke loaʻa i kēia kumuwaiwai, i puʻunaue ʻia ma luna o nā inoa inoa i koho ʻia, nā lula e hoʻokaʻawale i ke komo ʻana mai kekahi noi a i kekahi. Hiki iā ʻoe ke hoʻonohonoho i ka hiki ke komo ma waena o nā pods kikoʻī, nā kaiapuni (namespaces) a i ʻole nā poloka o nā helu IP:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: test-network-policy
namespace: default
spec:
podSelector:
matchLabels:
role: db
policyTypes:
- Ingress
- Egress
ingress:
- from:
- ipBlock:
cidr: 172.17.0.0/16
except:
- 172.17.1.0/24
- namespaceSelector:
matchLabels:
project: myproject
- podSelector:
matchLabels:
role: frontend
ports:
- protocol: TCP
port: 6379
egress:
- to:
- ipBlock:
cidr: 10.0.0.0/24
ports:
- protocol: TCP
port: 5978ʻAʻole kēia ka laʻana mua loa o hiki i kekahi manawa ke hoʻonāwaliwali i ka makemake e hoʻomaopopo i ke ʻano o ka hana ʻana o nā kulekele pūnaewele. Eia nō naʻe, e hoʻāʻo mākou e hoʻomaopopo i nā kumu kumu a me nā ʻano o ka hoʻoponopono ʻana i nā kahe o ke kaʻa me ka hoʻohana ʻana i nā kulekele pūnaewele...
He mea kūpono aia he 2 mau ʻano o ke kaʻa: ke komo ʻana i ka pod (Ingress) a me ka puka ʻana mai ia (Egress).
ʻOiaʻiʻo, ua māhele ʻia ka politika i kēia mau ʻāpana 2 e pili ana i ke kuhikuhi o ka neʻe.
ʻO ka hiʻohiʻona i makemake ʻia he mea koho; ka mea i pili i ka rula. Hiki i kēia ke lilo i pod (a i ʻole pūʻulu o nā pods) a i ʻole kahi kaiapuni (ʻo ia hoʻi, kahi inoa inoa). He kikoʻī koʻikoʻi: pono nā ʻano ʻelua o kēia mau mea i kahi lepili (i 'ae' ma nā hua'ōlelo Kubernetes) - ʻo ia nā mea a ka poʻe kālai'āina e hana pū me.
Ma waho aʻe o ka helu palena o nā mea koho i hoʻohui ʻia e kekahi ʻano lepili, hiki ke kākau i nā lula e like me "E ʻae / hōʻole i nā mea āpau / kēlā me kēia" i nā ʻano like ʻole. No kēia kumu, hoʻohana ʻia nā hana o ke ʻano:
podSelector: {}
ingress: []
policyTypes:
- Ingress- ma kēia laʻana, ua ālai ʻia nā pods a pau o ke kaiapuni mai ka hele ʻana mai. Hiki ke hoʻokō ʻia ke ʻano like ʻole me ke kūkulu ʻana:
podSelector: {}
ingress:
- {}
policyTypes:
- IngressPela no ka puka ana:
podSelector: {}
policyTypes:
- Egress- e hoʻopau iā ia. A eia ka mea e komo ai:
podSelector: {}
egress:
- {}
policyTypes:
- EgressKe hoʻi nei i ke koho ʻana i kahi plugin CNI no kahi puʻupuʻu, pono e ʻike i kēlā ʻAʻole kākoʻo kēlā me kēia pūnaewele pūnaewele i NetworkPolicy. No ka laʻana, ʻaʻole ʻike ʻo Flannel i ʻōlelo ʻia pehea e hoʻonohonoho ai i nā kulekele pūnaewele, ʻo ia i loko o ka waihona kūhelu. Ua ʻōlelo ʻia kekahi mea ʻē aʻe ma laila - kahi papahana Open Source , ka mea e hoʻonui nui i ka hoʻonohonoho maʻamau o nā Kubernetes API e pili ana i nā kulekele pūnaewele.
E ʻike iā Calico: theory
Hiki ke hoʻohana ʻia ka plugin Calico i ka hui pū ʻana me Flannel (subproject ) a i ʻole kūʻokoʻa, e uhi ana i ka pilina pūnaewele a me ka hiki ke hoʻokele i ka loaʻa.
He aha nā manawa kūpono e hoʻohana ai i ka hopena K8s "boxed" a me ka API set mai Calico e hāʻawi ai?
Eia ka mea i kūkulu ʻia i loko o NetworkPolicy:
- kaupalena 'ia ka poʻe kālai'āina e ke kaiapuni;
- hoʻohana ʻia nā kulekele i nā pods i kaha ʻia me nā lepili;
- Hiki ke hoʻohana ʻia nā lula i nā pods, nā kaiapuni a i ʻole nā subnets;
- Hiki i nā lula ke loaʻa nā protocols, inoa a i ʻole nā kikoʻī awa hōʻailona.
Penei ka hoʻonui ʻana o Calico i kēia mau hana:
- hiki ke hoʻohana i nā kulekele i kekahi mea: pod, ipu, mīkini virtual a i ʻole interface;
- Hiki i nā lula ke komo i kahi hana kūikawā (ka pāpā ʻana, ka ʻae ʻana, ke kau inoa ʻana);
- ʻO ka pahuhopu a i ʻole ke kumu o nā lula hiki ke lilo i awa, kahi ʻano o nā awa, nā protocols, HTTP a i ʻole ICMP mau ʻano, IP a i ʻole subnet (4th a i ʻole 6th hanauna), nā mea koho (nodes, hosts, environments);
- Eia hou, hiki iā ʻoe ke hoʻoponopono i ka hele ʻana o ke kaʻa me ka hoʻohana ʻana i nā hoʻonohonoho DNAT a me nā kulekele hoʻokele.
ʻO ka hana mua ma GitHub i ka lā hoʻopaʻa inoa ʻo Calico a hiki i Iulai 2016, a hoʻokahi makahiki ma hope mai ua lawe ka papahana i kahi kūlana alakaʻi i ka hoʻonohonoho ʻana i ka pilina pūnaewele Kubernetes - ʻike ʻia kēia, no ka laʻana, e nā hualoaʻa. :
Nui nā hoʻonā hoʻokele nui me nā K8, e like me , , a ua hoʻomaka kekahi e paipai iā ia no ka hoʻohana.
No ka hana, maikaʻi nā mea a pau ma ʻaneʻi. I ka hoʻāʻo ʻana i kā lākou huahana, ua hōʻike ka hui hoʻomohala ʻo Calico i ka hana astronomical, e holo ana ma mua o 50000 mau pahu ma 500 mau node kino me ka nui o ka hana ʻana o 20 pahu i kēlā me kēia kekona. ʻAʻohe pilikia i ʻike ʻia me ka scaling. ʻO ia mau hopena aia ma ka hoʻolaha ʻana o ka mana mua. ʻO nā haʻawina kūʻokoʻa e nānā ana i ka throughput a me ka hoʻohana ʻana i nā kumuwaiwai e hōʻoia i ka maikaʻi o ka hana a Calico e like me kā Flannel. :
Ke ulu wikiwiki nei ka papahana, kākoʻo ʻo ia i ka hana ma nā hoʻonā kaulana i mālama ʻia K8s, OpenShift, OpenStack, hiki ke hoʻohana iā Calico i ka wā e hoʻohana ana i kahi hui. , aia nā kuhikuhi e pili ana i ke kūkulu ʻana i nā pūnaewele Service Mesh ( hoʻohana pū ʻia me Istio).
E hoʻomaʻamaʻa me Calico
Ma ka hihia maʻamau o ka hoʻohana ʻana i vanilla Kubernetes, hoʻokomo i ka CNI e iho mai i ka hoʻohana ʻana i ka faila calico.yaml, , ma ka hoohana ana kubectl apply -f.
E like me ke kānāwai, ua kūpono ka mana o ka plugin me nā mana hou o 2-3 o Kubernetes: ʻaʻole i hoʻāʻo ʻia ka hana ma nā mana kahiko a ʻaʻole i hōʻoia ʻia. Wahi a nā mea hoʻomohala, holo ʻo Calico ma Linux kernels ma luna o 3.10 e holo ana i CentOS 7, Ubuntu 16 a i ʻole Debian 8, ma luna o nā iptables a i ʻole IPVS.
Kaʻawale i loko o ke kaiapuni
No ka ʻike maʻamau, e nānā i kahi hihia maʻalahi e hoʻomaopopo i ka ʻokoʻa o nā kulekele ʻoihana ma ka notation Calico mai nā mea maʻamau a pehea e hoʻokaʻawale ai ke ala i ka hana ʻana i nā lula i ko lākou heluhelu a me ka hoʻonohonoho hoʻonohonoho.
Aia he 2 mau noi pūnaewele i hoʻolālā ʻia i ka hui: ma Node.js a me PHP, hoʻohana kekahi o Redis. No ka pale ʻana i ke komo ʻana iā Redis mai PHP, ʻoiai e mālama ana i ka pilina me Node.js, e hoʻopili wale i kēia kulekele:
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: allow-redis-nodejs
spec:
podSelector:
matchLabels:
service: redis
ingress:
- from:
- podSelector:
matchLabels:
service: nodejs
ports:
- protocol: TCP
port: 6379Ua ʻae mākou i ka hele ʻana mai i ke awa Redis mai Node.js. A maopopo ʻaʻole lākou i pāpā i kekahi mea ʻē aʻe. Ke ʻike ʻia ʻo NetworkPolicy, hoʻomaka ka poʻe koho a pau i ʻōlelo ʻia i loko e hoʻokaʻawale ʻia, ke ʻole i kuhikuhi ʻia. Eia naʻe, ʻaʻole pili nā lula kaʻawale i nā mea ʻē aʻe i uhi ʻole ʻia e ka mea koho.
Hoʻohana ka laʻana apiVersion Kubernetes ma waho o ka pahu, akā ʻaʻohe mea e pale iā ʻoe mai ka hoʻohana ʻana . ʻOi aku ka kikoʻī o ka syntax, no laila pono ʻoe e kākau hou i ka lula no ka hihia i luna ma kēia ʻano:
apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
name: allow-redis-nodejs
spec:
selector: service == 'redis'
ingress:
- action: Allow
protocol: TCP
source:
selector: service == 'nodejs'
destination:
ports:
- 6379
ʻO nā hana i ʻōlelo ʻia ma luna no ka ʻae a hōʻole ʻana i nā kaʻa a pau ma o ka NetworkPolicy API maʻamau i loaʻa nā kūkulu me nā pale paʻakikī e hoʻomaopopo a hoʻomanaʻo. I ka hihia o Calico, e hoʻololi i ka loiloi o kahi lula ahi i ka ʻaoʻao, e hoʻololi wale action: Allow maluna o action: Deny.
Kaawale ma ke kaiapuni
E noʻonoʻo i kahi kūlana kahi e hoʻopuka ai kahi noi i nā metric ʻoihana no ka hōʻiliʻili ʻana ma Prometheus a me ka nānā hou ʻana me ka hoʻohana ʻana iā Grafana. Loaʻa paha ka ʻikepili koʻikoʻi i ka hoʻouka ʻana, hiki ke ʻike hou ʻia e ka lehulehu. Hūnā kāua i kēia ʻikepili mai ka nānā ʻana i nā maka:
ʻO Prometheus, ma ke ʻano he lula, ua kau ʻia i kahi ʻoihana lawelawe kaʻawale - ma ka laʻana he inoa inoa e like me kēia:
apiVersion: v1
kind: Namespace
metadata:
labels:
module: prometheus
name: kube-prometheus
kahua metadata.labels ʻaʻole kēia he pōʻino. E like me ka mea i ʻōlelo ʻia ma luna, namespaceSelector (a ʻo kekahi podSelector) hana me nā lepili. No laila, no ka ʻae ʻana i nā metric e lawe ʻia mai nā pods āpau ma kahi awa kikoʻī, pono ʻoe e hoʻohui i kekahi ʻano lepili (a i ʻole e lawe i nā mea i loaʻa), a laila e hoʻopili i kahi hoʻonohonoho e like me:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-metrics-prom
spec:
podSelector: {}
ingress:
- from:
- namespaceSelector:
matchLabels:
module: prometheus
ports:
- protocol: TCP
port: 9100A inā ʻoe e hoʻohana i nā kulekele Calico, e like ka syntax me kēia:
apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
name: allow-metrics-prom
spec:
ingress:
- action: Allow
protocol: TCP
source:
namespaceSelector: module == 'prometheus'
destination:
ports:
- 9100Ma keʻano laulā, ma ka hoʻohui ʻana i kēia mau ʻano kulekele no nā pono kikoʻī, hiki iā ʻoe ke pale aku i ka hana ʻino a i ʻole ka hoʻopilikia ʻole ʻia i ka hana ʻana o nā noi i loko o ka hui.
ʻO ka hoʻomaʻamaʻa maikaʻi loa, e like me ka mea nāna i hana ʻo Calico, ʻo ia ka "Paʻi i nā mea āpau a wehe pono i kāu mea e pono ai", i kākau ʻia ma (ʻo nā mea ʻē aʻe e hahai i kahi ala like - ʻo ia hoʻi, ma ).
Ke hoʻohana nei i nā mea Calico hou
E hoʻomanaʻo wau iā ʻoe ma o ka hoʻonohonoho lōʻihi o Calico API hiki iā ʻoe ke hoʻoponopono i ka loaʻa ʻana o nā nodes, ʻaʻole i kaupalena ʻia i nā pods. Ma ka laʻana e hoʻohana nei GlobalNetworkPolicy ua pani ʻia ka hiki ke hāʻawi i nā noi ICMP i loko o ka puʻupuʻu (no ka laʻana, nā pings mai kahi pod i kahi node, ma waena o nā pods, a i ʻole mai kahi node i kahi IP pod):
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
name: block-icmp
spec:
order: 200
selector: all()
types:
- Ingress
- Egress
ingress:
- action: Deny
protocol: ICMP
egress:
- action: Deny
protocol: ICMP
Ma ka hihia i luna aʻe nei, hiki i nā nodes cluster ke "hiki aku" kekahi i kekahi ma o ICMP. A ua hoʻoholo ʻia kēia pilikia ma ke ʻano GlobalNetworkPolicy, pili i kahi hui HostEndpoint:
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
name: deny-icmp-kube-02
spec:
selector: "role == 'k8s-node'"
order: 0
ingress:
- action: Allow
protocol: ICMP
egress:
- action: Allow
protocol: ICMP
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
name: kube-02-eth0
labels:
role: k8s-node
spec:
interfaceName: eth0
node: kube-02
expectedIPs: ["192.168.2.2"]ʻO ka hihia VPN
ʻO ka mea hope loa, e hāʻawi wau i kahi hiʻohiʻona maoli o ka hoʻohana ʻana i nā hana Calico no ka hihia o ka pili kokoke-cluster, inā ʻaʻole lawa kahi hoʻonohonoho maʻamau o nā kulekele. Hoʻohana nā mea kūʻai aku i kahi tunnel VPN no ke komo ʻana i ka palapala noi pūnaewele, a ua kāohi paʻa ʻia kēia komo ʻana a kaupalena ʻia i kahi papa inoa kikoʻī o nā lawelawe i ʻae ʻia no ka hoʻohana ʻana:
Hoʻopili nā mea kūʻai aku i ka VPN ma o ke awa UDP maʻamau 1194 a, ke hoʻopili ʻia, loaʻa nā ala i nā subnets cluster o nā pods a me nā lawelawe. Hoʻokomo ʻia nā subnets a pau i ʻole e nalowale nā lawelawe i ka wā e hoʻomaka hou ai a me nā hoʻololi ʻōlelo.
He kūlana maʻamau ka awa i ka hoʻonohonoho ʻana, kahi e kau ai i kekahi mau nuances i ke kaʻina hana o ka hoʻonohonoho ʻana i ka noi a hoʻoili iā ia i ka hui Kubernetes. No ka laʻana, ma ka AWS LoadBalancer no UDP i ʻike maoli ʻia i ka hopena o ka makahiki i hala ma kahi papa inoa palena o nā ʻāpana, a ʻaʻole hiki ke hoʻohana ʻia ʻo NodePort ma muli o kona hoʻouna ʻana i nā nodes cluster āpau a ʻaʻole hiki ke hoʻonui i ka helu o nā manawa kikowaena. kumu hoʻomanawanui hewa. Eia hou, pono ʻoe e hoʻololi i ka pae paʻamau o nā awa...
Ma muli o ka ʻimi ʻana ma o nā hoʻonā hiki, ua koho ʻia kēia:
- Hoʻonohonoho ʻia nā Pods me VPN i kēlā me kēia node i loko
hostNetwork, ʻo ia hoʻi, i ka IP maoli. - Hoʻouna ʻia ka lawelawe ma waho ma
ClusterIP. Hoʻokomo kino ʻia kahi awa ma ka node, hiki ke ʻike ʻia mai waho me nā hoʻopaʻa liʻiliʻi (ke kūlana kūlana o kahi helu IP maoli). - ʻO ka hoʻoholo ʻana i ka node kahi o ka pod rose ma waho o ke ʻano o kā mākou moʻolelo. E ʻōlelo wale wau hiki iā ʻoe ke hoʻopaʻa i ka lawelawe i kahi node a kākau paha i kahi lawelawe sidecar liʻiliʻi e nānā i ka IP IP o kēia manawa o ka lawelawe VPN a hoʻoponopono i nā moʻolelo DNS i hoʻopaʻa ʻia me nā mea kūʻai aku - ʻo ka mea i lawa ka noʻonoʻo.
Mai kahi hiʻohiʻona routing, hiki iā mākou ke ʻike kūʻokoʻa i kahi mea kūʻai aku VPN ma o kāna leka uila IP i hoʻopuka ʻia e ka server VPN. Aia ma lalo kahi hiʻohiʻona mua o ka kaupalena ʻana i ke komo ʻana o ka mea kūʻai aku i nā lawelawe, i hōʻike ʻia ma ka Redis i ʻōlelo ʻia ma luna.
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
name: vpnclient-eth0
labels:
role: vpnclient
environment: production
spec:
interfaceName: "*"
node: kube-02
expectedIPs: ["172.176.176.2"]
---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
name: vpn-rules
spec:
selector: "role == 'vpnclient'"
order: 0
applyOnForward: true
preDNAT: true
ingress:
- action: Deny
protocol: TCP
destination:
ports: [6379]
- action: Allow
protocol: UDP
destination:
ports: [53, 67]Maʻaneʻi, ua pāpā loa ʻia ka hoʻopili ʻana i ke awa 6379, akā i ka manawa like ke mālama ʻia ka hana o ka lawelawe DNS, ʻo ka hana ʻana o ka hana pinepine i ka wā e kākau ai i nā lula. No ka mea, e like me ka mea i ʻōlelo mua ʻia, i ka wā e ʻike ʻia ai kahi mea koho, hoʻohana ʻia ke kulekele hōʻole paʻamau iā ia ke ʻole i kuhikuhi ʻia.
Nā hopena
No laila, me ka hoʻohana ʻana i ka API kiʻekiʻe o Calico, hiki iā ʻoe ke hoʻonohonoho maʻalahi a hoʻololi i ke ala ala i loko a puni ka pūʻulu. Ma keʻano laulā, hiki ke nānā aku i kona hoʻohana ʻana e like me ka pana ʻana i nā manu liʻiliʻi me kahi pūkuniahi, a ʻo ka hoʻokō ʻana i kahi pūnaewele L3 me BGP a me IP-IP tunnels ke nānā aku i ka monstrous i kahi hoʻonohonoho Kubernetes maʻalahi ma kahi pūnaewele palahalaha ... .
ʻAʻole hiki ke hoʻokaʻawale i kahi puʻupuʻu e hoʻokō i nā koi palekana i nā manawa a pau, a ʻo kēia kahi e hele mai ai ʻo Calico (a i ʻole ka hopena like) e hoʻopakele. Hoʻohana ʻia nā hiʻohiʻona i hāʻawi ʻia i kēia ʻatikala (me nā hoʻololi liʻiliʻi) i kekahi mau hoʻonohonoho o kā mākou mea kūʻai aku ma AWS.
PS
E heluhelu pū ma kā mākou blog:
- «";
- "He alakaʻi i hōʻike ʻia no ka hui ʻana ma Kubernetes": , ;
- «".
Source: www.habr.com