Calico ee isku xirka Kubernetes: hordhac iyo khibrad yar

Ujeedada maqaalku waa in la soo bandhigo akhristaha aasaaska shabakadaha iyo maareynta siyaasadaha shabakada ee Kubernetes, iyo sidoo kale qaybta saddexaad ee Calico oo kordhisa awoodaha caadiga ah. Jidka dhexdiisa, fududaynta qaabeynta iyo sifooyinka qaarkood ayaa lagu muujin doonaa iyadoo la adeegsanayo tusaalooyin dhab ah oo ka yimid khibradeena hawlgelinta.

Hordhac degdeg ah oo ku saabsan qalabka isku xirka Kubernetes

Kooxda Kubernetes lama qiyaasi karo shabakad la'aanteed. Waxaan horey u daabacnay agab aasaaskooda ah: “Hagaha la sawiran ee isku xidhka Kubernetes"Iyo"Hordhac Xeerarka Shabakadda Kubernetes ee Xirfadlayaasha Amniga".

Marka la eego qodobkan, waxaa muhiim ah in la ogaado in K8s lafteedu aysan mas'uul ka ahayn isku xirka shabakada ee u dhexeeya weelasha iyo noodhka: tan, kala duwan CNI plugins (Container Networking Interface). Wax badan oo ku saabsan fikraddan annaga waxay kaloo ii sheegeen.

Tusaale ahaan, kuwa ugu caansan ee plugins-yadan waa Flannel - Waxay bixisaa isku xidh buuxa oo shabakadeed oo u dhexeeya dhammaan qanjidhada kooxeed iyada oo kor u qaadaysa buundooyinka nood kasta, iyada oo u qoondaynaysa shabakad hoose. Si kastaba ha ahaatee, gelitaan dhamaystiran oo aan sharciyeysnayn mar walba faa'iido ma aha. Si loo hubiyo nooc ka mid ah go'doominta ugu yar ee kutlada, waxaa lagama maarmaan ah in la farageliyo qaabeynta dab-damiska. Xaaladda guud, waxaa la hoosgeliyaa isla CNI-da, taas oo ah sababta faragelinta dhinac saddexaad ee iptables loo fasiri karo si khaldan ama la iska indhatiro gebi ahaanba.

Iyo "ka baxsan sanduuqa" ee abaabulka maamulka siyaasadda shabakada ee kutlada Kubernetes waa la bixiyay NetworkPolicy API. Khayraadkan, oo loo qaybiyay meelo magacyo la doortay, waxa laga yaabaa inay ku jiraan xeerar lagu kala saaro gelitaanka hal codsi iyo mid kale. Waxa kale oo ay kuu ogolaanaysaa inaad isku hagaajiso gelitaannada u dhexeeya boodhadhka gaarka ah, deegaan (meelaha magacyadooda) ama baloogyada cinwaannada IP:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - ipBlock:
        cidr: 172.17.0.0/16
        except:
        - 172.17.1.0/24
    - namespaceSelector:
        matchLabels:
          project: myproject
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - protocol: TCP
      port: 6379
  egress:
  - to:
    - ipBlock:
        cidr: 10.0.0.0/24
    ports:
    - protocol: TCP
      port: 5978

Tani maaha tusaalaha ugu horreeya dukumeenti rasmi ah waxaa laga yaabaa inay hal mar iyo dhammaanba niyad jabiso rabitaanka fahamka macquulka ah ee sida siyaasadaha shabakadu u shaqeeyaan. Si kastaba ha ahaatee, waxaan wali isku dayi doonaa inaan fahamno mabaadi'da aasaasiga ah iyo hababka habaynta socodka taraafikada iyadoo la adeegsanayo siyaasadaha shabakada

Waa macquul in ay jiraan 2 nooc oo taraafig ah: Gelida Booska (Soo-gudista) iyo ka bixidda (Egress).

Runtii siyaasaddu waxay u qaybsantaa 2daa qaybood oo ku salaysan jihada loo socdo.

Sifada soo socota ee loo baahan yahay waa xulashada; kii xeerku ku dhaqmo. Tani waxay noqon kartaa boodh (ama koox kabo ah) ama bay'ad (ie magac magac). Faahfaahin muhiim ah: labada nooc ee walxahan waa inay ku jiraan calaamad (calaamadda Ereyada Kubernetes) - kuwani waa kuwa ay siyaasiyiintu ku shaqeeyaan.

Marka lagu daro tiro xaddidan oo xulasho ah oo ay ku mideysan yihiin nooc ka mid ah calaamadda, waxaa suurtagal ah in la qoro xeerar sida "Oggolow / diido wax kasta / qof walba" oo kala duwan. Ujeedadaas awgeed, dhismayaasha foomka ayaa loo isticmaalaa:

  podSelector: {}
  ingress: []
  policyTypes:
  - Ingress

- Tusaalahan, dhammaan boodhadhka deegaanka waa laga xannibay taraafikada soo socota. Dhaqanka ka soo horjeeda waxaa lagu gaari karaa dhismaha soo socda:

  podSelector: {}
  ingress:
  - {}
  policyTypes:
  - Ingress

Sidoo kale bixista:

  podSelector: {}
  policyTypes:
  - Egress

- in la damiyo. Oo halkan waxa ku jira:

  podSelector: {}
  egress:
  - {}
  policyTypes:
  - Egress

Ku soo noqoshada doorashada plugin CNI ee koox-kooxeed, waxaa xusid mudan taas Shabakad kastaa ma taageerto NetworkPolicy. Tusaale ahaan, Flannel hore loo sheegay ma garanayo sida loo habeeyo siyaasadaha shabakada, taas oo si toos ah baa loo yidhi ee kaydka rasmiga ah. Beddel kale ayaa sidoo kale halkaas lagu sheegay - mashruuca Isha Furan Calico, kaas oo si weyn u balaadhinaya heerka caadiga ah ee Kubernetes APIs marka loo eego siyaasadaha shabakada.

Barashada Calico: aragti

Qalabka Calico waxaa loo isticmaali karaa is dhexgalka Flannel (mashruuc-hoosaad Canal) ama si madaxbanaan, daboolaya isku xidhka shabakada iyo awoodaha maaraynta helitaanka labadaba.

Waa maxay fursadaha ay bixiyaan isticmaalka K8s "sanduuq" xalka iyo API dhigay ee Calico?

Waa kuwan waxa lagu dhisay NetworkPolicy:

• siyaasiyiinta waxa xaddidaya deegaanka;
• siyaasadaha waxaa lagu dabaqaa galalka lagu calaamadeeyay;
• xeerarka waxaa lagu dabaqi karaa pods, deegaan ama subnets;
• xeerarku waxay ka koobnaan karaan borotokool, magac ama calaamad deked qeexan.

Waa kuwan sida Calico u ballaariyo hawlahan:

• siyaasadaha waxaa lagu dabaqi karaa shay kasta: pod, weel, mashiinka farsamada ama interface;
• Xeerarku waxay ka koobnaan karaan ficil gaar ah (mamnuuc, ogolaansho, gooyn);
• bartilmaameedka ama isha xeerarku waxay noqon karaan deked, dekedo kala duwan, borotokool, HTTP ama ICMP sifooyin, IP ama subnet (jiilka 4aad ama 6aad), doorasha kasta (nodes, martigeliyaha, bay'ada);
• Intaa waxaa dheer, waxaad nidaamin kartaa marinka taraafikada adoo isticmaalaya habaynta DNAT iyo siyaasadaha gudbinta taraafikada.

Midka ugu horreeya wuxuu ku sameeyaa GitHub ee kaydka Calico taariikhda dib u soo noqoshada July 2016, iyo sanad ka dib mashruucu wuxuu qaatay jagada hogaaminta ee abaabulka isku xirka shabakada Kubernetes - tani waxaa caddaynaya, tusaale ahaan, natiijooyinka sahanka, waxaa qabtay The New Stack:

Xalal badan oo waaweyn oo la maareeyey oo leh K8s, sida Amazon EKS, Azure AKS, Google GKE iyo kuwo kale waxay bilaabeen inay ku taliyaan in la isticmaalo.

Sida waxqabadka, wax walba waa ku fiican yihiin halkan. Markay tijaabinayaan badeecadooda, kooxda horumarinta Calico waxay soo bandhigeen waxqabadka astronomical, iyagoo ku ordaya in ka badan 50000 oo weel oo ku yaal 500 qanjidhada jirka oo leh heerka abuurista 20 weel ilbiriqsi kasta. Wax dhibaato ah laguma aqoonsan miisaanka. Natiijooyinka noocan oo kale ah ayaa lagu dhawaaqay mar horeba lagu dhawaaqay nuqulkii ugu horreeyay. Daraasado madax-bannaan oo diiradda saaraya wax-soo-saarka iyo isticmaalka kheyraadka ayaa sidoo kale xaqiijinaya waxqabadka Calico inuu ku dhow yahay sida Flannel. Tusaale ahaan:

Mashruucu si dhakhso leh ayuu u horumarayaa, wuxuu taageeraa shaqada xalalka caanka ah ee ay maamusho K8s, OpenShift, OpenStack, waa suurtagal in la isticmaalo Calico marka la dirayo koox isticmaalaya kops, waxaa jira tixraacyo ku saabsan dhismaha shabakadaha Adeegga Mesh (waa kan tusaale loo isticmaalo iyadoo lala kaashanayo Istio).

Ku celceli Calico

Xaaladda guud ee isticmaalka vanilj Kubernetes, ku rakibida CNI waxay hoos ugu dhacdaa isticmaalka faylka calico.yaml, lagala soo baxay website-ka rasmiga ah, iyadoo la isticmaalayo kubectl apply -f.

Sida caadiga ah, nooca hadda ee plugin wuxuu la jaan qaadayaa 2-3 nooc ee ugu dambeeyay ee Kubernetes: hawlgalka noocyadii hore lama tijaabin lamana dammaanad qaadayo. Sida laga soo xigtay horumariyayaashu, Calico wuxuu ku shaqeeyaa kernels Linux oo ka sarreeya 3.10 oo ku shaqeeya CentOS 7, Ubuntu 16 ama Debian 8, dusha sare ee iptables ama IPVS.

Go'doominta deegaanka dhexdiisa

Si loo fahmo guud, aan eegno kiis fudud si aan u fahamno sida siyaasadaha shabakada ee qoraalka Calico ay uga duwan yihiin kuwa caadiga ah iyo sida habka loo abuuro xeerar u fududeeyo akhrintooda iyo dabacsanaanta qaabeynta:

Waxaa jira 2 codsiyo shabakadeed oo la geeyay kooxda: gudaha Node.js iyo PHP, mid ka mid ah taas oo adeegsata Redis. Si aad u xannibto gelitaanka Redis ee PHP, iyadoo la ilaalinayo isku xidhka Node.js, kaliya dabaq siyaasaddan soo socota:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-redis-nodejs
spec:
  podSelector:
    matchLabels:
      service: redis
  ingress:
  - from:
    - podSelector:
        matchLabels:
          service: nodejs
    ports:
    - protocol: TCP
      port: 6379

Asal ahaan waxaan u oggolaanay taraafikada ka soo socota dekedda Redis ee ka imanaya Node.js. Waxayna si cad u diideen wax kale. Isla marka NetworkPolicy soo baxdo, dhammaan xulashooyinka lagu sheegay waxay bilaabayaan inay go'doomiyaan, haddii aan si kale loo cayimin. Si kastaba ha ahaatee, xeerarka go'doominta laguma dabaqo shay kale oo aanu khusayn cidda dooratay.

Tusaalaha ayaa isticmaala apiVersion Kubernetes oo ka baxay sanduuqa, laakiin ma jiraan wax kaa diidaya inaad isticmaasho kheyraadka isla magaca laga keenay gaarsiinta Calico. Erayga halkaa ka jira aad buu u faahfaahsan yahay, marka waxaad u baahan doontaa inaad dib u qorto xeerka kiiska kore qaabkan soo socda:

apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
  name: allow-redis-nodejs
spec:
  selector: service == 'redis'
  ingress:
  - action: Allow
    protocol: TCP
    source:
      selector: service == 'nodejs'
    destination:
      ports:
      - 6379

Dhismayaasha aan kor ku soo xusnay ee oggolaanshaha ama diidmada dhammaan taraafikada iyada oo loo marayo NetworkPolicy API ee caadiga ah waxay ka kooban yihiin dhismo leh jaantusyo ay adagtahay in la fahmo lana xasuusto. Xaaladda Calico, si loo beddelo macquulka ah ee xeerka dab-damiska oo ka soo horjeeda, kaliya beddel action: Allow on action: Deny.

Go'doomin deegaan

Hadda qiyaas xaalad uu codsigu ka soo saaro cabbiro ganacsi oo loogu ururinayo Prometheus iyo falanqayn dheeraad ah oo la isticmaalayo Grafana. Soo raritaanka waxaa ku jiri kara xog xasaasi ah, taasoo mar kale si guud loo arki karo si caadi ah. Aan ka qarino xogtan indhaha soo jiidaya:

Prometheus, sida caadiga ah, waxaa lagu meeleeyaa deegaan adeeg gaar ah - tusaale ahaan waxay noqon doontaa magac magac sidan oo kale ah:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    module: prometheus
  name: kube-prometheus

field metadata.labels Tani waxay noqotay mid aan shil ahayn. Sida kor ku xusan, namespaceSelector (sidoo kale podSelector) ku shaqeeya calaamado. Sidaa darteed, si aad u oggolaato in cabbirada laga soo qaado dhammaan boodhadhka ku yaal deked gaar ah, waa inaad ku darto nooc ka mid ah calaamad (ama ka soo qaad kuwa jira), ka bacdina adeegso qaabayn sida:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-metrics-prom
spec:
  podSelector: {}
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          module: prometheus
    ports:
    - protocol: TCP
      port: 9100

Oo haddii aad isticmaasho siyaasadaha Calico, syntax waxay noqon doontaa sidan:

apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
  name: allow-metrics-prom
spec:
  ingress:
  - action: Allow
    protocol: TCP
    source:
      namespaceSelector: module == 'prometheus'
    destination:
      ports:
      - 9100

Guud ahaan, marka lagu daro siyaasadaha noocaan ah ee baahiyaha gaarka ah, waxaad ka ilaalin kartaa faragelinta xaasidnimada ama shilalka ah ee hawlgalka codsiyada kutlada.

Habka ugu fiican, sida ay qabaan hal-abuurayaasha Calico, waa "wax walba xannib oo si cad u fur waxaad u baahan tahay" habka, oo lagu diiwaangeliyay dukumeenti rasmi ah (kuwa kale waxay raacaan hab la mid ah - gaar ahaan, gudaha maqaal hore loo sheegay).

Isticmaalka Walxaha Calico Dheeraadka ah

Aan ku xasuusiyo in iyada oo loo marayo set-ka dheer ee Calico API-yada waxaad nidaamin kartaa helitaanka qanjidhada, oo aan ku xaddidnayn pods. Tusaalaha soo socda adigoo isticmaalaya GlobalNetworkPolicy Awoodda lagu gudbin karo codsiyada ICMP ee kutlada waa xiran tahay (tusaale, pings-ka ka soo gaddiga ilaa qanjirada, inta u dhaxaysa pods, ama ka soo baxa noode ilaa IP pod):

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: block-icmp
spec:
  order: 200
  selector: all()
  types:
  - Ingress
  - Egress
  ingress:
  - action: Deny
    protocol: ICMP
  egress:
  - action: Deny
    protocol: ICMP

Xaaladda sare, weli waa suurtogal in qanjidhada kooxuhu ay "la xiriiraan" midba midka kale iyada oo loo marayo ICMP. Arrintaasna qaab lagu xalliyo GlobalNetworkPolicy, lagu dabaqay cid HostEndpoint:

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: deny-icmp-kube-02
spec:
  selector: "role == 'k8s-node'"
  order: 0
  ingress:
  - action: Allow
    protocol: ICMP
  egress:
  - action: Allow
    protocol: ICMP
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: kube-02-eth0
  labels:
    role: k8s-node
spec:
  interfaceName: eth0
  node: kube-02
  expectedIPs: ["192.168.2.2"]

Kiiska VPN

Ugu dambeyntii, waxaan ku siin doonaa tusaale dhab ah oo ku saabsan isticmaalka hawlaha Calico ee kiiska isdhexgalka u dhow, marka nidaamka caadiga ah ee siyaasadaha aysan ku filneyn. Si loo galo arjiga shabakadda, macaamiishu waxay adeegsadaan tunnel VPN, gelitaankan si adag ayaa loo kontroolo oo xaddidan yahay liis gaar ah oo adeegyada loo oggol yahay in la isticmaalo:

Macaamiishu waxay ku xidhaan VPN-ka iyagoo sii maraya dekedda caadiga ah ee UDP 1194 oo, marka lagu xidho, waxay helayaan dariiqyada isku xidhka hoose ee qaybaha iyo adeegyada. Dhammaan shabakadaha hoose waa la riixaa si aysan u lumin adeegyada marka dib loo bilaabo oo la beddelo ciwaannada.

Dekadda ku jirta qaabeynta ayaa ah mid caadi ah, taas oo ku soo rogtay xoogaa nuances habka habaynta codsiga iyo u wareejinta kooxda Kubernetes. Tusaale ahaan, isla AWS LoadBalancer ee UDP ayaa si dhab ah u soo muuqday dhamaadkii sannadkii hore liis xaddidan oo gobollo ah, iyo NodePort lama isticmaali karo sababtoo ah u gudbinteeda dhammaan qanjidhada kooxeed mana suurtogal ah in la cabbiro tirada tusaalooyinka server-ka ujeedooyinka dulqaadka qaladka. Intaa waxaa dheer, waa inaad beddeshaa tirada dekedaha ee caadiga ah...

Natiijadii raadinta xalalka suurtagalka ah, waxa la doortay:

1. Boodhadhka leh VPN waxaa loo qorsheeyay nood kasta hostNetwork, taas oo ah, IP-ga dhabta ah.
2. Adeegga waxaa lagu dhejiyaa dibadda ClusterIP. Deked jir ahaan ayaa lagu rakibay noodhka, taas oo laga heli karo dibadda iyada oo boos celin yar leh (joogitaanka shuruuda ee ciwaanka IP-ga dhabta ah).
3. Go'aaminta noodhka ay boodhka ka soo baxday waa ka baxsan baaxadda sheekadeena. Kaliya waxaan dhihi doonaa inaad si adag u "cidi karto" adeegga qanjidhada ama waxaad ku qori kartaa adeeg yar oo dhinac ah kaas oo la socon doona cinwaanka IP-ga ee hadda adeega VPN oo aad tafatiran karto diiwaannada DNS ee ka diiwaangashan macaamiisha - qof kasta oo leh male ku filan.

Marka loo eego dhinaca dariiqa, waxaanu si gaar ah ugu aqoonsan karnaa macmiilka VPN ciwaanka IP-ga ee uu soo saaray server-ka VPN. Hoos waxaa ku yaal tusaale hore oo xaddidaya gelitaanka macmiilka noocaas ah adeegyada, oo lagu muujiyey Redis-ka sare lagu sheegay:

apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: vpnclient-eth0
  labels:
    role: vpnclient
    environment: production
spec:
  interfaceName: "*"
  node: kube-02
  expectedIPs: ["172.176.176.2"]
---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: vpn-rules
spec:
  selector: "role == 'vpnclient'"
  order: 0
  applyOnForward: true
  preDNAT: true
  ingress:
  - action: Deny
    protocol: TCP
    destination:
      ports: [6379]
  - action: Allow
    protocol: UDP
    destination:
      ports: [53, 67]

Halkan, isku xirka dekedda 6379 si adag ayaa loo mamnuucay, laakiin isla mar ahaantaana hawlgalka adeegga DNS waa la ilaaliyaa, shaqeynta taas oo inta badan la dhibtoonaysa marka la samaynayo xeerar. Sababtoo ah, sidaan hore u soo sheegnay, marka doorashadu soo baxdo, nidaamka diidmada caadiga ah ayaa lagu dabaqaa haddii aan si kale loo cayimin.

Natiijooyinka

Markaa, addoo isticmaalaya API-ga horumarsan ee Calico, waxaad si dabacsanaan leh u habeyn kartaa oo aad si firfircoon u beddeli kartaa marin-u-socodka gudaha iyo hareeraha kooxda. Guud ahaan, isticmaalkeedu wuxuu u ekaan karaa sida shimbiraha toogashada leh madfac, iyo hirgelinta shabakad L3 ah oo leh BGP iyo tunnel IP-IP waxay u egtahay mid bahalnimo ah rakibida Kubernetes fudud ee shabakad siman ... Si kastaba ha ahaatee, haddii kale qalabku wuxuu u muuqdaa mid waxtar leh oo faa'iido leh. .

Go'doominta kooxdu si ay u buuxiso shuruudaha amniga waxaa laga yaabaa inaysan had iyo jeer suurtagal ahayn, waana halka Calico (ama xal la mid ah) uu yimaado samatabbixinta. Tusaalooyinka lagu bixiyay maqaalkan (oo leh wax-ka-beddelo yar-yar) ayaa loo adeegsadaa dhowr qalab oo macaamiisheena AWS ah.

PS

Sidoo kale ka akhri boggayaga:

• «Hordhac Xeerarka Shabakadda Kubernetes ee Xirfadlayaasha Amniga";
• "Hagaha Sharaxan ee Isku-xidhka Kubernetes": qaybaha 1 iyo 2 (qaabka shabakada, shabakadaha dulsaar), qaybta 3 (adeegyada iyo habaynta gaadiidka);
• «Interface Networking Container (CNI) - interface network iyo heerka weelasha Linux".

Source: www.habr.com