Calico ye networking muKubernetes: sumo uye chiitiko chidiki

Chinangwa chechinyorwa ndechekuzivisa muverengi kune izvo zvekutanga zvetiweki uye kutonga network marongero muKubernetes, pamwe neyechitatu-bato Calico plugin iyo inowedzera kugona kwakajairwa. Panzira, kureruka kwekugadzirisa uye mamwe maitiro acharatidzwa uchishandisa mienzaniso chaiyo kubva kune yedu yekushandisa ruzivo.

Yekukurumidza sumo yeKubernetes networking appliance

A Kubernetes cluster haigone kufungidzirwa pasina network. Takatoburitsa zvinyorwa pamusoro pezvazviri: “Gwaro rine mufananidzo kune network muKubernetes"Uye"Nhanganyaya yeKubernetes Network Policies yeChengetedzo Nyanzvi".

Muchirevo chechinyorwa chino, zvakakosha kuziva kuti K8s pachayo haina mhosva yekubatanidza network pakati pemidziyo nemanodhi: nekuda kweizvi, zvakasiyana. CNI plugins (Container Networking Interface). Zvimwe pamusoro pepfungwa iyi isu vakandiudzawo.

Semuenzaniso, iyo yakajairika yeiyi plugins ndeye Flannel - inopa yakazara network yekubatanidza pakati pese masumbu node nekusimudza mabhiriji pane imwe neimwe node, ichipa subnet kwairi. Zvisinei, kuwanika kwakakwana uye kusina murairo hakusi kunobatsira nguva dzose. Kuti upe imwe mhando yekusarudzika kushoma musumbu, zvinodikanwa kupindira mukugadziriswa kwefirewall. Muzviitiko zvakawanda, inoiswa pasi pekutonga kweCNI imwechete, ndicho chikonzero chero chechitatu-party interventions mu iptables inogona kududzirwa zvisizvo kana kuregererwa zvachose.

Uye "kunze kwebhokisi" yekuronga manejimendi mutemo manejimendi muKubernetes cluster inopihwa Purogiramu inonzi NetworkPolicy. Ichi chishandiso, chakagoverwa pamusoro penzvimbo dzakasarudzwa, chinogona kunge chine mitemo yekusiyanisa kuwana kubva kune imwe application kuenda kune imwe. Iyo zvakare inobvumidza iwe kugadzirisa kuwanikwa pakati pemapodhi chaiwo, nharaunda (mazita enzvimbo) kana zvivharo zve IP kero:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - ipBlock:
        cidr: 172.17.0.0/16
        except:
        - 172.17.1.0/24
    - namespaceSelector:
        matchLabels:
          project: myproject
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - protocol: TCP
      port: 6379
  egress:
  - to:
    - ipBlock:
        cidr: 10.0.0.0/24
    ports:
    - protocol: TCP
      port: 5978

Uyu hausi iwo muenzaniso wekare we zvinyorwa zvepamutemo inogona kamwe uye zvachose kuodza mwoyo chishuwo chekunzwisisa pfungwa yekuti network network inoshanda sei. Nekudaro, isu ticharamba tichiedza kunzwisisa misimboti uye nzira dzekugadzirisa kuyerera kwetraffic tichishandisa network network ...

Zvine musoro kuti kune 2 mhando dzemotokari: kupinda pod (Ingress) uye kubuda kubva mairi (Egress).

Chaizvoizvo, zvematongerwo enyika zvakakamurwa muzvikamu zviviri izvi zvichienderana negwara rekufamba.

Chinotevera chinodiwa hunhu is a selector; uyo mutemo unoshanda kwaari. Iyi inogona kunge iri podhi (kana boka remapodhi) kana nharaunda (kureva nzvimbo yezita). Tsanangudzo yakakosha: marudzi ese ezviro izvi anofanira kunge aine label (chitaridzi muKubernetes terminology) - aya ndiwo anoshanda nawo vezvematongerwo enyika.

Pamusoro penhamba inogumira yevanosarudza vakabatana neimwe mhando yezita, zvinokwanisika kunyora mitemo senge "Bvumira / ramba zvese / munhu wese" mukusiyana kwakasiyana. Nechinangwa ichi, zvivakwa zvefomu zvinoshandiswa:

  podSelector: {}
  ingress: []
  policyTypes:
  - Ingress

- mumuenzaniso uyu, mapodhi ese ari munharaunda akavharwa kubva kune inouya traffic. Maitiro akapesana anogona kuwanikwa nekuvakwa kunotevera:

  podSelector: {}
  ingress:
  - {}
  policyTypes:
  - Ingress

Zvakangofanana kune anobuda:

  podSelector: {}
  policyTypes:
  - Egress

- kuidzima. Uye hezvino izvo zvekusanganisira:

  podSelector: {}
  egress:
  - {}
  policyTypes:
  - Egress

Kudzokera kusarudzo yeCNI plugin yeboka, zvakakosha kucherechedza izvozvo haisi yega yega network plugin inotsigira NetworkPolicy. Semuenzaniso, iyo yatotaurwa Flannel haizive maitiro ekugadzirisa network network, iyo zvinonzi zvakananga mudura repamutemo. Imwe nzira inotaurwawo ipapo - Open Source project Calico, iyo inowedzera zvakanyanya yakajairwa seti yeKubernetes APIs maererano netiweki marongero.

Kusvika pakuziva Calico: dzidziso

Iyo Calico plugin inogona kushandiswa mukubatanidzwa neFlannel (subproject mugero) kana yakazvimirira, inovhara zvese zviri zviviri network yekubatanidza uye kuwanikwa manejimendi kugona.

Ndeipi mikana yekushandisa iyo K8s "boxed" mhinduro uye API yakaiswa kubva kuCalico inopa?

Hezvino izvo zvakavakwa muNetworkPolicy:

• vezvematongerwo enyika vanoganhurirwa nemhoteredzo;
• mitemo inoshandiswa kumapodhi akanyorwa nemavara;
• mitemo inogona kushandiswa kune pods, zvakatipoteredza kana subnets;
• Mitemo inogona kuve nemaprotocol, ane mazita kana ekufananidzira maratidziro echiteshi.

Heano maitiro Calico anowedzera aya mabasa:

• mitemo inogona kushandiswa kune chero chinhu: pod, mudziyo, chaiwo muchina kana interface;
• mitemo inogona kuva nechiito chaicho (kurambidza, mvumo, kutema miti);
• chinangwa kana chitubu chemitemo chinogona kunge chiri chiteshi, ruzhinji rwezviteshi, maprotocol, HTTP kana ICMP hunhu, IP kana subnet (4th kana 6th chizvarwa), chero vasarudzo (node, mauto, nharaunda);
• Pamusoro pezvo, unogona kudzora mafambiro etraffic uchishandisa DNAT marongero uye marongero ekutumira traffic.

Yekutanga inozvipira paGitHub muCalico repository yakadzokera munaChikunguru 2016, uye gore rakatevera chirongwa chakatora chinzvimbo chekutungamira mukuronga Kubernetes network yekubatanidza - izvi zvinoratidzwa, semuenzaniso, nemhedzisiro yeongororo. yakaitwa neThe New Stack:

Mazhinji mahombe anogadziriswa mhinduro neK8s, senge Amazon EKS, Azure AKS, Google GKE uye vamwe vakatanga kuikurudzira kuti ishandiswe.

Kana zviri zvekuita, zvese zvakanaka pano. Mukuyedza chigadzirwa chavo, timu yekusimudzira yeCalico yakaratidza kuita kwezvemuchadenga, ichimhanyisa midziyo inodarika zviuru makumi mashanu pa50000 node dzemuviri ine mwero wekugadzira wemidziyo makumi maviri pasekondi. Hapana matambudziko akaonekwa nekuyera. Migumisiro yakadaro zvakaziviswa kare pakuziviswa kweshanduro yekutanga. Zvidzidzo zvakazvimirira zvinotarisa pakushandisa uye kushandisa zviwanikwa zvinosimbisawo kuita kwaCalico kwakada kufanana nekwaFlannel. Somuenzaniso:

Iyo purojekiti iri kukura nekukurumidza, inotsigira basa mune dzakakurumbira mhinduro dzakagadziriswa K8s, OpenShift, OpenStack, zvinogoneka kushandisa Calico paunenge uchiendesa cluster uchishandisa. kops, pane mareferensi ekuvakwa kweService Mesh network (heino muenzaniso yakashandiswa pamwe chete neIstio).

Dzidzira neCalico

Mune yakajairika kesi yekushandisa vanilla Kubernetes, kuisa CNI kunodzika pakushandisa iyo faira calico.yaml, yakatorwa kubva pawebhusaiti yepamutemo, nekushandisa kubectl apply -f.

Sezvo mutemo, iyo yazvino vhezheni ye plugin inoenderana neazvino 2-3 shanduro yeKubernetes: kushanda mushanduro dzekare hakuna kuedzwa uye haina kuvimbiswa. Zvinoenderana nevagadziri, Calico inomhanya paLinux kernels pamusoro pe3.10 inomhanya CentOS 7, Ubuntu 16 kana Debian 8, pamusoro pe iptables kana IPVS.

Kuzviparadzanisa nevamwe mukati mezvakatipoteredza

Kuti tinzwisise zvakazara, ngatitarisei nyaya yakapusa kuti tinzwisise kuti network marongero ari muCalico notation anosiyana sei kubva kune akajairwa uye kuti nzira yekugadzira mitemo inorerutsa kuverenga kwavo uye kugadzirisa kuchinjika:

Kune maviri ewebhu maapplication akaiswa musumbu: muNode.js uye PHP, imwe yacho inoshandisa Redis. Kuvhara kupinda kweRedis kubva kuPP, uchichengetedza kubatana neNode.js, ingo shandisa iyo inotevera mutemo:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-redis-nodejs
spec:
  podSelector:
    matchLabels:
      service: redis
  ingress:
  - from:
    - podSelector:
        matchLabels:
          service: nodejs
    ports:
    - protocol: TCP
      port: 6379

Chaizvoizvo isu takabvumira traffic inouya kuchiteshi cheRedis kubva kuNode.js. Uye zvakajeka havana kurambidza chimwe chinhu. Panongoonekwa NetworkPolicy, vese vanosarudza vanotaurwa mairi vanotanga kuve vega, kunze kwekunge zvataurwa neimwe nzira. Zvisinei, mitemo yekuzviparadzanisa haishande kune zvimwe zvinhu zvisina kuvharwa nemusarudzo.

Muenzaniso unoshandiswa apiVersion Kubernetes kunze kwebhokisi, asi hapana chinokutadzisa kuishandisa resource yezita rimwe chete kubva kuCalico delivery. Iyo syntax iripo yakatsanangurwa, saka iwe unozofanirwa kunyora zvakare mutemo wenyaya iri pamusoro nenzira inotevera:

apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
  name: allow-redis-nodejs
spec:
  selector: service == 'redis'
  ingress:
  - action: Allow
    protocol: TCP
    source:
      selector: service == 'nodejs'
    destination:
      ports:
      - 6379

Izvo zvakataurwa pamusoro zvinovaka zvekubvumidza kana kuramba zvese traffic kuburikidza neyakajairwa NetworkPolicy API ine zvivakwa zvine maparentheses izvo zvakaoma kunzwisisa nekurangarira. Panyaya yeCalico, kushandura pfungwa yemutemo we firewall kune zvakapesana, ingochinja action: Allow pamusoro action: Deny.

Kuparadzaniswa nenharaunda

Zvino fungidzira mamiriro apo application inogadzira bhizinesi metrics yekuunganidza muPrometheus uye nekumwe kuongorora uchishandisa Grafana. Iyo yekukwirisa inogona kunge iine data rakajeka, iro rinoonekwa zvakare pachena nechero. Ngativanzei iyi data kubva pakuona maziso:

Prometheus, sekutonga, inoiswa munzvimbo yakaparadzana yebasa - mumuenzaniso ichave nzvimbo yezita seizvi:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    module: prometheus
  name: kube-prometheus

munda metadata.labels izvi zvakazoitika kuti hazvina kuitika. Sezvataurwa pamusoro apa, namespaceSelector (naizvozvowo podSelector) inoshanda nemavara. Naizvozvo, kubvumidza metrics kutorwa kubva kune ese mapodhi pane chaiyo chiteshi, iwe uchafanirwa kuwedzera imwe mhando yezita (kana kutora kubva kune iripo), wobva waisa gadziriso senge:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-metrics-prom
spec:
  podSelector: {}
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          module: prometheus
    ports:
    - protocol: TCP
      port: 9100

Uye kana ukashandisa Calico marongero, iyo syntax ichave seizvi:

apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
  name: allow-metrics-prom
spec:
  ingress:
  - action: Allow
    protocol: TCP
    source:
      namespaceSelector: module == 'prometheus'
    destination:
      ports:
      - 9100

Kazhinji, nekuwedzera aya marudzi emitemo yezvido chaizvo, unogona kudzivirira kubva kune zvakaipa kana netsaona kukanganiswa mukushanda kwemaapplication musumbu.

Maitiro akanakisa, sekureva kwevagadziri veCalico, ndiyo "Vimba zvese uye uvhure pachena zvaunoda" nzira, yakanyorwa mukati. zvinyorwa zvepamutemo (vamwe vanotevera nzira yakafanana - kunyanya, in yatotaurwa nyaya).

Kushandisa Zvimwe Calico Zvinhu

Rega ndikuyeuchidze kuti kuburikidza neakatambanudzwa seti yeCalico APIs unogona kudzora kuwanikwa kwemanodhi, kwete kugumira kumapodhi. Mumuenzaniso unotevera kushandisa GlobalNetworkPolicy kugona kupfuudza zvikumbiro zveICMP musumbu kwakavharwa (semuenzaniso, pings kubva papodhi kuenda kune node, pakati pemapodhi, kana kubva node kuenda kuIP pod):

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: block-icmp
spec:
  order: 200
  selector: all()
  types:
  - Ingress
  - Egress
  ingress:
  - action: Deny
    protocol: ICMP
  egress:
  - action: Deny
    protocol: ICMP

Munyaya iri pamusoro apa, zvichiri kugoneka kuti masumbu masumbu "asvike" kune mumwe nemumwe kuburikidza neICMP. Uye nyaya iyi inogadziriswa nenzira GlobalNetworkPolicy, inoshandiswa kusangano HostEndpoint:

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: deny-icmp-kube-02
spec:
  selector: "role == 'k8s-node'"
  order: 0
  ingress:
  - action: Allow
    protocol: ICMP
  egress:
  - action: Allow
    protocol: ICMP
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: kube-02-eth0
  labels:
    role: k8s-node
spec:
  interfaceName: eth0
  node: kube-02
  expectedIPs: ["192.168.2.2"]

Iyo VPN Case

Pakupedzisira, ini ndichapa muenzaniso chaiwo wekushandisa Calico mabasa enyaya yepedyo-boka rekudyidzana, kana yakajairwa seti yemitemo isina kukwana. Kuti uwane iyo webhu application, vatengi vanoshandisa VPN tunnel, uye kuwana uku kunodzorwa zvakasimba uye kunogumira kune yakatarwa runyorwa rwesevhisi inotenderwa kushandiswa:

Vatengi vanobatana neVPN kuburikidza neyakajairwa UDP port 1194 uye, kana yakabatana, inogashira nzira dzinoenda kune sumbu subnets yemapodhi nemasevhisi. Ese subnets anosundirwa kuti asarasikirwe masevhisi panguva yekutangazve uye shanduko yekero.

Iyo chiteshi mukugadzirisa ndeyeyero, iyo inoisa mamwe nuances pamaitiro ekugadzirisa iyo application uye nekuiendesa kuKubernetes cluster. Semuenzaniso, mune imwecheteyo AWS LoadBalancer yeUDP yakaonekwa chaizvo mukupera kwegore rapfuura mune yakaderera runyorwa rwematunhu, uye NodePort haigone kushandiswa nekuda kwekutumira kwayo kune ese masumbu node uye hazvigoneke kuyera huwandu hwemaseva ezviitiko zve. kukanganisa kushivirira zvinangwa. Uyezve, iwe uchafanirwa kushandura iyo default renji yemadoko ...

Nekuda kwekutsvaga kuburikidza nemhinduro dzinobvira, zvinotevera zvakasarudzwa:

1. Mapodhi ane VPN akarongwa pane node mukati hostNetwork, ndiko kuti, kuIP chaiyo.
2. Iyo sevhisi inotumirwa kunze kuburikidza ClusterIP. Chiteshi chakaiswa mumuviri pane node, iyo inowanikwa kubva kunze nekuchengetedzwa kudiki (kuvepo kwemamiriro eiyo chaiyo IP kero).
3. Kusarudza iyo node iyo pod yakasimuka inopfuura chiyero chenyaya yedu. Ini ndinongotaura kuti iwe unogona zvakasimba "kuroverera" sevhisi kune node kana kunyora diki padivi sevhisi iyo inoongorora yazvino IP kero yeVPN sevhisi uye kugadzirisa marekodhi eDNS akanyoreswa nevatengi - chero ane fungidziro yakakwana.

Kubva pakuona kwenzira, isu tinokwanisa kuziva mutengi weVPN nekero yayo yeIP yakapihwa neVPN server. Pazasi pane muenzaniso wekutanga wekudzora kuwana kwemutengi akadaro kumasevhisi, anoratidzwa pane yataurwa pamusoro Redis:

apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: vpnclient-eth0
  labels:
    role: vpnclient
    environment: production
spec:
  interfaceName: "*"
  node: kube-02
  expectedIPs: ["172.176.176.2"]
---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: vpn-rules
spec:
  selector: "role == 'vpnclient'"
  order: 0
  applyOnForward: true
  preDNAT: true
  ingress:
  - action: Deny
    protocol: TCP
    destination:
      ports: [6379]
  - action: Allow
    protocol: UDP
    destination:
      ports: [53, 67]

Pano, kubatanidza kune port 6379 kunorambidzwa zvachose, asi panguva imwechete kushanda kweDNS sevhisi kunochengetedzwa, kushanda kwaro kunowanzotambura pakudhirowa mitemo. Nekuti, sezvakambotaurwa, kana anosarudza aonekwa, iyo yekuramba mutemo yekuramba inoiswa kwairi kunze kwekunge yatsanangurwa neimwe nzira.

Migumisiro

Saka, uchishandisa Calico's advanced API, unogona kuchinjika kugadzirisa uye zvine simba kuchinja routing mukati nekutenderedza sumbu. Kazhinji, kushandiswa kwayo kunogona kuita sekupfura shiri duku nekanoni, uye kushandisa L3 network ine BGP uye IP-IP tunnels inotaridzika inotyisa mukugadzirisa Kubernetes kugadzika pane flat network ... Zvisinei, kana zvisizvo chishandiso chinotaridzika chaizvo uye chinobatsira. .

Kuparadzanisa sumbu kuti usangane nezvinodiwa zvekuchengetedza kunogona kusagoneka nguva dzose, uye apa ndipo panouya Calico (kana mhinduro yakafanana) kuzonunura. Mienzaniso yakapihwa muchinyorwa chino (ine zvidiki zvigadziriso) inoshandiswa mukumisikidzwa kwakati wandei kwevatengi vedu muAWS.

PS

Verenga zvakare pablog yedu:

• «Nhanganyaya yeKubernetes Network Policies yeChengetedzo Nyanzvi";
• "An Illustrated Guide to Networking muKubernetes": zvikamu 1 uye 2 (network model, overlay network), Chikamu 3 (masevhisi uye kugadzirisa traffic);
• «Container Networking Interface (CNI) - network interface uye yakajairwa yeLinux midziyo".

Source: www.habr.com