I-Calico yokunxibelelana kwi-Kubernetes: isingeniso kunye namava amancinci

Injongo yenqaku kukwazisa umfundi kwiziseko zonxibelelwano kunye nokulawula imigaqo-nkqubo yenethiwekhi kwi-Kubernetes, kunye neplagin ye-Calico yesithathu eyandisa amandla asemgangathweni. Endleleni, ukukhululeka kokucwangciswa kunye nezinye iimpawu ziya kuboniswa ngokusebenzisa imizekelo yangempela kumava ethu okusebenza.

Intshayelelo ekhawulezayo kwisixhobo sothungelwano se-Kubernetes

Iqela leKubernetes alinakucingelwa ngaphandle kwenethiwekhi. Sele sipapashe imathiriyeli kwiziseko zazo: “Isikhokelo esinemifanekiso sothungelwano kwi-Kubernetes"Kwaye"Intshayelelo ye-Kubernetes Network Policies kuBasebenzi boKhuseleko».

Kumxholo weli nqaku, kubalulekile ukuqaphela ukuba i-K8s ngokwayo ayinaxanduva kunxibelelwano lwenethiwekhi phakathi kwezikhongozeli kunye neendawo zokuhlala: oku, ezahlukeneyo. Iiplagi zeCNI (Isikhongozeli seNxibelelwano yoNxibelelwano). Okunye malunga nale ngcamango thina nabo bandixelele.

Umzekelo, eyona ixhaphakileyo kwezi plugins Flannel - ibonelela ngokudibanisa ngokupheleleyo kwenethiwekhi phakathi kwazo zonke iindawo zeqela ngokuphakamisa iibhulorho kwindawo nganye, ukunika i-subnet kuyo. Nangona kunjalo, ukufikeleleka okupheleleyo nokungalawulwayo akusoloko kunenzuzo. Ukubonelela ngolunye uhlobo lokuzihlukanisa okuncinci kwi-cluster, kuyimfuneko ukungenelela ekucwangcisweni kwe-firewall. Kwimeko eqhelekileyo, ibekwe phantsi kolawulo lwe-CNI efanayo, yingakho nayiphi na i-third party intervention in iptables inokutolika ngokungalunganga okanye ingahoywa ngokupheleleyo.

Kwaye "ngaphandle kwebhokisi" yokulungelelanisa ulawulo lomgaqo-nkqubo wenethiwekhi kwiqela le-Kubernetes linikezelwa NetworkPolicy API. Lo vimba, usasazwe ngaphezulu kwezithuba zamagama ezikhethiweyo, unokuqulatha imithetho yokwahlula ukufikelela kwisicelo esinye ukuya kwesinye. Ikwakuvumela ukuba uqwalasele ukufikeleleka phakathi kweepod ezithile, iimeko-bume (izithuba zamagama) okanye iibhloko zeedilesi ze-IP:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - ipBlock:
        cidr: 172.17.0.0/16
        except:
        - 172.17.1.0/24
    - namespaceSelector:
        matchLabels:
          project: myproject
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - protocol: TCP
      port: 6379
  egress:
  - to:
    - ipBlock:
        cidr: 10.0.0.0/24
    ports:
    - protocol: TCP
      port: 5978

Lo ayingowona mzekelo wakudala we amaxwebhu asemthethweni inokuthi kanye kwaye iwutyhafise umnqweno wokuqonda ingqiqo yendlela esebenza ngayo imigaqo-nkqubo yenethiwekhi. Nangona kunjalo, siya kuzama ukuqonda imigaqo esisiseko kunye neendlela zokucubungula ukuhamba kwezithuthi kusetyenziswa imigaqo-nkqubo yenethiwekhi ...

Kunengqiqo ukuba kukho iintlobo ezi-2 zetrafikhi: ukungena kwi-pod (Ingress) kunye nokuphuma kuyo (Egress).

Ngokwenyani, ipolitiki yahlulwe yaba ziindidi ezi-2 ezisekelwe kwicala lentshukumo.

Uphawu loyelelwano olulandelayo olufunekayo ngumkhethi; lowo usebenza kuye umgaqo. Oku kunokuba yi-pod (okanye iqela le-pods) okanye indawo (o.k.t. indawo yamagama). Iinkcukacha ezibalulekileyo: zombini iindidi zezi zinto kufuneka ziqulathe ileyibhile (yokurekhoda kwi Kubernetes isigama) - ezi zezo abezopolitiko basebenza nazo.

Ukongeza kwinani eliqingqiweyo labakhethi abadityaniswe luhlobo oluthile lwelebula, kunokwenzeka ukuba ubhale imigaqo efana ne "Vumela / khanyela yonke into / wonke umntu" kwiinguqu ezahlukeneyo. Kule njongo, ulwakhiwo lwefom lusetyenziswa:

  podSelector: {}
  ingress: []
  policyTypes:
  - Ingress

- kulo mzekelo, zonke ii-pods kwindawo zivaliwe kwi-traffic engenayo. Ukuziphatha okuchasayo kunokufezekiswa ngolwakhiwo olulandelayo:

  podSelector: {}
  ingress:
  - {}
  policyTypes:
  - Ingress

Ngokufanayo nakwabaphumayo:

  podSelector: {}
  policyTypes:
  - Egress

- ukuyicima. Kwaye nantsi into yokubandakanya:

  podSelector: {}
  egress:
  - {}
  policyTypes:
  - Egress

Ukubuyela ekukhethweni kwe-plugin ye-CNI ye-cluster, kuyafaneleka ukuba uqaphele oko ayizizo zonke iiplagi zenethiwekhi ezixhasa iNetworkPolicy. Ngokomzekelo, i-Flannel esele ikhankanywe ayiyazi indlela yokuqwalasela imigaqo-nkqubo yenethiwekhi, leyo kuthethwa ngokuthe ngqo kwindawo yokugcina esemthethweni. Enye indlela ikwakhankanyiwe apho - iprojekthi yoMthombo oVulekileyo Calico, eyandisa kakhulu isethi esemgangathweni ye-Kubernetes APIs ngokwemigaqo-nkqubo yenethiwekhi.

Ukwazi uCalico: ithiyori

Iplagi yeCalico ingasetyenziswa ngokudityaniswa neFlannel (subproject umjelokazi) okanye ngokuzimeleyo, egubungela zombini uqhagamshelo lwenethiwekhi kunye nokufumaneka kolawulo lwezakhono.

Ngawaphi amathuba okusebenzisa i-K8s "ibhokisi" isisombululo kunye neseti ye-API evela kwi-Calico ibonelela?

Nantsi into eyakhelwe kwiNethiwekhiPolicy:

• abezopolitiko basikelwe umda yimekobume;
• iipolisi zisetyenziswa kwiipod eziphawulwe ngeelebhile;
• imigaqo ingasetyenziswa kwiipod, iindawo okanye ii-subnets;
• imithetho ingaqulatha iprothokholi, enegama okanye izibuko lemiqondiso yomfuziselo.

Nantsi indlela iCalico eyandisa ngayo le misebenzi:

• imigaqo-nkqubo inokusetyenziswa kuyo nayiphi na into: i-pod, isikhongozeli, umatshini wenyani okanye ujongano;
• imithetho inokuqulatha isenzo esithile (ukuthintela, imvume, ukugawulwa kwemithi);
• ekujoliswe kuyo okanye umthombo wemithetho ingaba yi-port, uluhlu lwamachweba, i-protocol, i-HTTP okanye i-ICMP iimpawu, i-IP okanye i-subnet (isizukulwana se-4 okanye se-6), nabaphi na abakhethi (i-nodes, imikhosi, imimandla);
• Ukongezelela, unokulawula ukuhamba kwetrafikhi usebenzisa izicwangciso ze-DNAT kunye nemigaqo-nkqubo yokuthumela i-traffic.

Eyokuqala yenza kwi-GitHub kwindawo yokugcina i-Calico emva kukaJulayi 2016, kwaye emva konyaka iprojekthi yathatha indawo ehamba phambili ekuququzeleleni uqhagamshelwano lwenethiwekhi ye-Kubernetes - oku kungqinwa, umzekelo, ngeziphumo zophando, eqhutywa yiNew Stack:

Izisombululo ezininzi ezikhulu ezilawulwayo ngee K8s, ezifana Amazon EKS, Azure AKS, Google GKE kwaye abanye baqala ukuyincoma ukuba isetyenziswe.

Ngokuphathelele ukusebenza, yonke into ilungile apha. Ekuvavanyeni imveliso yabo, iqela lophuhliso lweCalico libonise ukusebenza kweenkwenkwezi, liqhuba ngaphezu kwee-container ze-50000 kwii-nodes ze-500 eziphathekayo kunye nesantya sokudala i-container ze-20 ngomzuzwana. Akukho zingxaki zichongiwe ngokukala. Iziphumo ezinjalo zabhengezwa esele ikwisibhengezo senguqulelo yokuqala. Izifundo ezizimeleyo ezigxile ekuphumeni nasekusetyenzisweni kwezixhobo zikwaqinisekisa ukuba ukusebenza kukaCalico kuphantse kwafana nokukaFlannel. Ngokomzekelo:

Iprojekthi iphuhla ngokukhawuleza, ixhasa umsebenzi kwizisombululo ezithandwayo ezilawulwa yi-K8s, i-OpenShift, i-OpenStack, kunokwenzeka ukusebenzisa i-Calico xa uhambisa iqela usebenzisa. khaba, kukho iireferensi kulwakhiwo lothungelwano lweMesh yeNkonzo (Nasi umzekelo isetyenziswe ngokubambisana ne-Istio).

Ziqhelanise neCalico

Kwimeko eqhelekileyo yokusebenzisa i-vanilla Kubernetes, ukufaka i-CNI yehla ekusebenziseni ifayile calico.yaml, ikhutshelwe kwiwebhusayithi esemthethweni, ngokusebenzisa kubectl apply -f.

Njengomthetho, inguqu yangoku ye-plugin ihambelana neenguqu ze-2-3 zakutsha ze-Kubernetes: ukusebenza kwiinguqulelo ezindala akuvavanywanga kwaye akuqinisekiswanga. Ngokutsho kwabaphuhlisi, iCalico isebenza kwii-Linux kernels ngaphezulu kwe-3.10 eqhuba i-CentOS 7, Ubuntu 16 okanye i-Debian 8, ngaphezulu kwee-iptables okanye i-IPVS.

Ukwahluka phakathi kokusingqongileyo

Ukuqonda ngokubanzi, makhe sijonge imeko elula ukuqonda ukuba imigaqo-nkqubo yenethiwekhi kwi-notation ye-Calico ihluke njani kwizinto eziqhelekileyo kunye nendlela yokudala imithetho eyenza lula ukufunda kunye nokuguquguquka koqwalaselo:

Kukho izicelo zewebhu ezi-2 ezifakwe kwiqela: kwi-Node.js kunye ne-PHP, enye yazo isebenzisa i-Redis. Ukuvala ukufikelela kwi-Redis ukusuka kwi-PHP, ngelixa ugcina uqhagamshelo kunye ne-Node.js, sebenzisa lo mgaqo-nkqubo ulandelayo:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-redis-nodejs
spec:
  podSelector:
    matchLabels:
      service: redis
  ingress:
  - from:
    - podSelector:
        matchLabels:
          service: nodejs
    ports:
    - protocol: TCP
      port: 6379

Ngokusisiseko sivumele i-traffic engenayo kwizibuko le-Redis ukusuka kwi-Node.js. Kwaye ngokucacileyo abazange bathintele enye into. Ngokukhawuleza ukuba i-NetworkPolicy ibonakala, bonke abakhethi abakhankanywe kuyo baqala ukuhlukaniswa, ngaphandle kokuba kuchazwe ngenye indlela. Nangona kunjalo, imithetho yokwahlula ayisebenzi kwezinye izinto ezingabandakanywanga ngumkhethi.

Umzekelo usebenzisa apiVersion Kubernetes ngaphandle kwebhokisi, kodwa akukho nto ikuthintela ukuba uyisebenzise umthombo wegama elifanayo ukusuka kuhambiso lweCalico. I-syntax ikhona ineenkcukacha ezithe kratya, ngoko ke kuya kufuneka uphinde ubhale umgaqo wale meko ingentla ngolu hlobo lulandelayo:

apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
  name: allow-redis-nodejs
spec:
  selector: service == 'redis'
  ingress:
  - action: Allow
    protocol: TCP
    source:
      selector: service == 'nodejs'
    destination:
      ports:
      - 6379

Ulwakhiwo olukhankanywe ngasentla lokuvumela okanye ukwala yonke i-traffic nge-NetworkPolicy API eqhelekileyo iqulathe ulwakhiwo olunezibiyeli ekunzima ukuyiqonda nokukhumbula. Kwimeko yeCalico, ukutshintsha ingqiqo yomthetho we-firewall ukuya kwelinye, tshintsha nje action: Allow phezu action: Deny.

Ukwahlulwa ngokwendalo

Ngoku khawufane ucinge imeko apho isicelo sivelisa iimethrikhi zoshishino zokuqokelela kwi-Prometheus kunye nohlalutyo olongezelelweyo usebenzisa iGrafana. Ukulayishwa kungaqulatha idatha ebuthathaka, ephinda ibonwe esidlangalaleni ngokuzenzekelayo. Masifihle le datha kumehlo okukhangela:

I-Prometheus, njengomthetho, ibekwe kwindawo yenkonzo eyahlukileyo - kumzekelo iya kuba yindawo yamagama efana nale:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    module: prometheus
  name: kube-prometheus

Intsimi metadata.labels oku akuzange kwenzeke ngengozi. Njengoko kukhankanyiwe ngasentla, namespaceSelector (njengo podSelector) isebenza ngeeleyibhile. Ke ngoko, ukuvumela iimetrics ukuba zithathwe kuzo zonke iipod kwizibuko elithile, kuya kufuneka udibanise uhlobo oluthile lweleyibhile (okanye uthathe kwezikhoyo), kwaye emva koko usebenzise uqwalaselo olunje:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-metrics-prom
spec:
  podSelector: {}
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          module: prometheus
    ports:
    - protocol: TCP
      port: 9100

Kwaye ukuba usebenzisa iipolisi zeCalico, i-syntax iya kuba ngolu hlobo:

apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
  name: allow-metrics-prom
spec:
  ingress:
  - action: Allow
    protocol: TCP
    source:
      namespaceSelector: module == 'prometheus'
    destination:
      ports:
      - 9100

Ngokubanzi, ngokongeza ezi ntlobo zemigaqo-nkqubo kwiimfuno ezithile, unokukhusela ngokuchasene nokuphazamiseka okunonya okanye ngengozi ekusebenzeni kwezicelo kwiqela.

Olona qheliselo lubalaseleyo, ngokutsho kwabadali beCalico, yindlela "Vala yonke into kwaye uvule ngokucacileyo into oyifunayo", ebhalwe kwi. amaxwebhu asemthethweni (abanye balandela indlela efanayo - ngakumbi, kwi inqaku esele likhankanyiwe).

Ukusebenzisa izinto zeCalico ezongezelelweyo

Makhe ndikukhumbuze ukuba ngeseti eyandisiweyo yeCalico APIs unokulawula ukufumaneka kweenodi, kungaphelelanga kwiipods. Kulo mzekelo ulandelayo usebenzisa GlobalNetworkPolicy ukukwazi ukudlula izicelo ze-ICMP kwiqela livaliwe (umzekelo, i-pings ukusuka kwi-pod ukuya kwi-node, phakathi kwee-pods, okanye ukusuka kwi-node ukuya kwi-IP pod):

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: block-icmp
spec:
  order: 200
  selector: all()
  types:
  - Ingress
  - Egress
  ingress:
  - action: Deny
    protocol: ICMP
  egress:
  - action: Deny
    protocol: ICMP

Kwimeko engentla, kusenokwenzeka ukuba i-cluster nodes "zifikelele" omnye komnye nge-ICMP. Kwaye lo mbandela usonjululwa ngeendlela GlobalNetworkPolicy, ifakwe kwiziko HostEndpoint:

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: deny-icmp-kube-02
spec:
  selector: "role == 'k8s-node'"
  order: 0
  ingress:
  - action: Allow
    protocol: ICMP
  egress:
  - action: Allow
    protocol: ICMP
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: kube-02-eth0
  labels:
    role: k8s-node
spec:
  interfaceName: eth0
  node: kube-02
  expectedIPs: ["192.168.2.2"]

Ityala leVPN

Ekugqibeleni, ndiya kunika umzekelo wangempela wokusebenzisa imisebenzi ye-Calico kwimeko yokusebenzisana kufuphi neqela, xa isethi esemgangathweni yemigaqo-nkqubo ayanele. Ukufikelela kwisicelo sewebhu, abathengi basebenzisa itonela yeVPN, kwaye olu fikelelo lulawulwa ngokuqinileyo kwaye lukhawulelwe kuluhlu oluthile lweenkonzo ezivunyelwe ukusetyenziswa:

Abathengi baxhuma kwi-VPN nge-standard UDP port 1194 kwaye, xa bexhunyiwe, bafumana iindlela kwii-subnet ze-cluster ze-pods kunye neenkonzo. Zonke ii-subnets zityhalwa ukuze zingaphulukani neenkonzo ngexesha lokuqalisa kwakhona kunye notshintsho lwedilesi.

Izibuko kuqwalaselo lisemgangathweni, elibeka ezinye iinuances kwinkqubo yokuqwalasela isicelo kwaye idluliselwe kwiqela leKubernetes. Ngokomzekelo, kwi-AWS LoadBalancer efanayo ye-UDP ibonakala ngokoqobo ekupheleni konyaka ophelileyo kuluhlu olulinganiselwe lwemimandla, kwaye i-NodePort ayinakusetyenziswa ngenxa yokuthunyelwa kwayo kuzo zonke iindawo ze-cluster kwaye akunakwenzeka ukukala inani lemizekelo yeseva iinjongo zokunyamezela iimpazamo. Ukongeza, kuya kufuneka utshintshe uluhlu olungagqibekanga lwamazibuko...

Njengomphumo wokukhangela izisombululo ezinokwenzeka, oku kulandelayo kwakhethwa:

1. Iipods ezineVPN zicwangciswe kwindawo nganye hostNetwork, oko kukuthi, kweyona IP.
2. Inkonzo ithunyelwa ngaphandle ClusterIP. I-port ifakwe ngokwasemzimbeni kwi-node, efikelelekayo ukusuka ngaphandle kunye nokugcinwa okuncinci (ubukho obunemiqathango yedilesi ye-IP yangempela).
3. Ukumisela i-node apho i-pod rose ingaphaya kwendawo yebali lethu. Ndiza kuthetha nje ukuba unokwenza ngokuqinileyo "isikhonkwane" inkonzo kwi-node okanye ubhale inkonzo encinci ye-sidecar eya kubeka iliso kwidilesi ye-IP yangoku yenkonzo ye-VPN kwaye uhlele iirekhodi ze-DNS ezibhaliswe kunye nabaxhasi - nabani na ocinga ngokwaneleyo.

Ukusuka kumbono womzila, sinokubona ngokukodwa umxhasi weVPN ngedilesi ye-IP ekhutshwe ngumncedisi weVPN. Apha ngezantsi ngumzekelo wamandulo wokunqanda ukufikelela komxhasi kwiinkonzo, kuboniswe kwiRedis ekhankanywe ngasentla:

apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: vpnclient-eth0
  labels:
    role: vpnclient
    environment: production
spec:
  interfaceName: "*"
  node: kube-02
  expectedIPs: ["172.176.176.2"]
---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: vpn-rules
spec:
  selector: "role == 'vpnclient'"
  order: 0
  applyOnForward: true
  preDNAT: true
  ingress:
  - action: Deny
    protocol: TCP
    destination:
      ports: [6379]
  - action: Allow
    protocol: UDP
    destination:
      ports: [53, 67]

Apha, ukudibanisa kwi-port 6379 akuvumelekanga ngokungqongqo, kodwa kwangaxeshanye ukusebenza kwenkonzo ye-DNS kuyagcinwa, ukusebenza kwayo kudla ngokubandezeleka xa kusenziwa imithetho. Kuba, njengoko kukhankanyiwe ngaphambili, xa kuvela umkhethi, umgaqo-nkqubo wokukhanyela ongagqibekanga usetyenziswa kuwo ngaphandle kokuba kuchazwe ngenye indlela.

Iziphumo

Ke, usebenzisa iCalico's API ephucukileyo, unokuqwalasela ngokuguquguqukayo kwaye utshintshe ngokuguquguqukayo umzila ngaphakathi kunye nokujikeleza iqela. Ngokubanzi, ukusetyenziswa kwayo kunokujongeka njengokudubula oongqatyana nge-cannon, kunye nokuphumeza inethiwekhi ye-L3 kunye ne-BGP kunye ne-IP-IP itonela ibonakala inqabileyo kufakelo olulula lwe-Kubernetes kwinethiwekhi ecaba ... .

Ukwahlula iqela ukuhlangabezana neemfuno zokhuseleko akunakusoloko kusenzeka, kwaye kulapho uCalico (okanye isisombululo esifanayo) esiza kuhlangula. Imizekelo enikwe kweli nqaku (kunye nohlengahlengiso oluncinci) isetyenziswa kufakelo oluninzi lwabathengi bethu kwi-AWS.

PS

Funda nakwibhlog yethu:

• «Intshayelelo ye-Kubernetes Network Policies kuBasebenzi boKhuseleko";
• "Isikhokelo esineMizobo kwiNethiwekhi eKubernetes": icandelo 1 kunye ne-2 (imodeli yothungelwano, uthungelwano olungaphezulu), icandelo 3 (iinkonzo kunye nokulungiswa kwezithuthi);
• «Isikhongozeli seNxibelelwano yoThungelwano (CNI) - ujongano lwenethiwekhi kunye nomgangatho wezikhongozeli zeLinux».

umthombo: www.habr.com