Calico pamaneti ku Kubernetes: mawu oyamba ndi chidziwitso chaching'ono

Cholinga cha nkhaniyi ndikudziwitsa owerenga zoyambira pamaneti ndi kuyang'anira ma network ku Kubernetes, komanso pulogalamu yachitatu ya Calico yomwe imakulitsa kuthekera kokhazikika. Panjira, kumasuka kwa kasinthidwe ndi zina zidzawonetsedwa pogwiritsa ntchito zitsanzo zenizeni kuchokera ku zochitika zathu zogwirira ntchito.

Kuyambitsa mwachangu kwa Kubernetes networking appliance

Gulu la Kubernetes silingaganizidwe popanda netiweki. Tasindikiza kale zolemba pazoyambira zawo: "Chiwongolero chowonetsera ma network ku Kubernetes"Ndipo"Chiyambi cha Kubernetes Network Policies for Security Professionals".

Pankhani ya nkhaniyi, ndikofunikira kuzindikira kuti ma K8s pawokha alibe udindo wolumikizana ndi ma netiweki pakati pa zotengera ndi ma node: pa izi, zosiyanasiyana. Mapulogalamu a CNI (Chotengera Networking Interface). Zambiri za lingaliro ili adandiuzanso.

Mwachitsanzo, ambiri mwa mapulaginiwa ndi Flannel - imapereka kulumikizana kwathunthu kwa netiweki pakati pa magulu onse amagulu pokweza milatho pa node iliyonse, ndikugawa subnet kwa iyo. Komabe, kupezeka kwathunthu ndi kosagwirizana sikuli kopindulitsa nthawi zonse. Kupereka mtundu wina wa kudzipatula pang'ono mu tsango, m'pofunika kulowerera kasinthidwe a firewall. Nthawi zambiri, zimayikidwa pansi pa ulamuliro wa CNI yomweyo, chifukwa chake njira iliyonse yachitatu mu iptables ikhoza kutanthauziridwa molakwika kapena kunyalanyazidwa palimodzi.

Ndipo "kunja kwa bokosi" pokonzekera kasamalidwe ka ndondomeko za maukonde mu gulu la Kubernetes amaperekedwa NetworkPolicy API. Chida ichi, chogawidwa pamalo osankhidwa, chikhoza kukhala ndi malamulo osiyanitsa mwayi wopezeka kuchokera ku pulogalamu ina kupita ku ina. Zimakupatsaninso mwayi wosintha kupezeka pakati pa ma pod, madera (malo amazina) kapena midadada ya ma adilesi a IP:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - ipBlock:
        cidr: 172.17.0.0/16
        except:
        - 172.17.1.0/24
    - namespaceSelector:
        matchLabels:
          project: myproject
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - protocol: TCP
      port: 6379
  egress:
  - to:
    - ipBlock:
        cidr: 10.0.0.0/24
    ports:
    - protocol: TCP
      port: 5978

Ichi si chitsanzo akale kwambiri zolemba zovomerezeka mwina kamodzi kokha kufooketsa chikhumbo chofuna kumvetsetsa malingaliro a momwe ma network amagwirira ntchito. Komabe, tidzayesetsabe kumvetsetsa mfundo zazikuluzikulu ndi njira zoyendetsera kayendetsedwe ka magalimoto pogwiritsa ntchito ndondomeko za intaneti ...

Ndizomveka kuti pali mitundu iwiri yamagalimoto: kulowa mu pod (Ingress) ndikutulukamo (Egress).

Kwenikweni, ndale zimagawidwa m'magulu a 2 awa kutengera kayendetsedwe ka kayendetsedwe kake.

Chotsatira chofunikira ndi chosankha; kwa amene lamuloli likugwira ntchito. Izi zitha kukhala poto (kapena gulu la makoko) kapena chilengedwe (ie malo a mayina). Tsatanetsatane wofunikira: mitundu yonse iwiri ya zinthu izi iyenera kukhala ndi chizindikiro (chizindikiro mu Kubernetes terminology) - awa ndi omwe andale amagwira nawo ntchito.

Kuwonjezera pa chiwerengero chochepa cha osankhidwa ophatikizidwa ndi mtundu wina wa chizindikiro, ndizotheka kulemba malamulo monga "Lolani / kukana chirichonse / aliyense" mosiyana. Pachifukwa ichi, mapangidwe a fomu amagwiritsidwa ntchito:

  podSelector: {}
  ingress: []
  policyTypes:
  - Ingress

- mu chitsanzo ichi, ma pod onse m'chilengedwe atsekedwa kuti asalowe. Khalidwe losiyana likhoza kutheka ndi kumanga motere:

  podSelector: {}
  ingress:
  - {}
  policyTypes:
  - Ingress

Mofananamo kwa otuluka:

  podSelector: {}
  policyTypes:
  - Egress

- kuzimitsa. Ndipo izi ndi zomwe muyenera kuphatikiza:

  podSelector: {}
  egress:
  - {}
  policyTypes:
  - Egress

Kubwerera ku chisankho cha pulogalamu yowonjezera ya CNI pamagulu, ndizofunika kudziwa si pulogalamu yowonjezera ya netiweki yomwe imathandizira NetworkPolicy. Mwachitsanzo, Flannel yotchulidwa kale sadziwa momwe angakhazikitsire ndondomeko za intaneti, zomwe zikunenedwa mwachindunji m'malo ovomerezeka. Njira ina imatchulidwanso pamenepo - polojekiti ya Open Source Kalico, yomwe imakulitsa kwambiri seti yokhazikika ya Kubernetes APIs malinga ndi ndondomeko za intaneti.

Kudziwa Calico: chiphunzitso

Pulagi ya Calico itha kugwiritsidwa ntchito kuphatikiza ndi Flannel (subproject ngalande) kapena paokha, kukhudza kulumikizidwa kwa netiweki ndi kuthekera kowongolera kupezeka.

Kodi kugwiritsa ntchito njira ya K8s "boxed" ndi API yochokera ku Calico kumapereka?

Nazi zomwe zidapangidwa mu NetworkPolicy:

• andale amachepetsedwa ndi chilengedwe;
• ndondomeko zikugwiritsidwa ntchito pazitsulo zolembedwa ndi zilembo;
• malamulo angagwiritsidwe ntchito popanga, malo kapena subnets;
• malamulo amatha kukhala ndi ma protocol, otchulidwa kapena madoko ophiphiritsa.

Umu ndi momwe Calico imakulitsira ntchito izi:

• ndondomeko zingagwiritsidwe ntchito pa chinthu chilichonse: pod, chidebe, makina enieni kapena mawonekedwe;
• malamulo akhoza kukhala ndi zochita zinazake (kuletsa, chilolezo, kudula mitengo);
• chandamale kapena magwero a malamulo akhoza kukhala doko, madoko osiyanasiyana, ma protocol, HTTP kapena ICMP zikhumbo, IP kapena subnet (4th kapena 6th generation), osankhidwa aliwonse (node, makamu, chilengedwe);
• Kuphatikiza apo, mutha kuyang'anira kuchuluka kwa magalimoto pogwiritsa ntchito makonda a DNAT ndi ndondomeko zotumizira anthu.

Woyamba adachita pa GitHub m'malo osungiramo Calico kuyambira Julayi 2016, ndipo patatha chaka chimodzi polojekitiyi idatenga udindo wotsogola pakukonzekera kulumikizidwa kwa netiweki ya Kubernetes - izi zikuwonetsedwa, mwachitsanzo, ndi zotsatira za kafukufukuyu, yolembedwa ndi The New Stack:

Mayankho ambiri oyendetsedwa ndi K8s, monga Amazon EKS, Azure AKS, Google GKE ndipo ena anayamba kulilimbikitsa kuti ligwiritsidwe ntchito.

Ponena za magwiridwe antchito, zonse ndizabwino apa. Poyesa malonda awo, gulu lachitukuko la Calico lidawonetsa momwe zinthu zakuthambo zimagwirira ntchito, zikuyendetsa zotengera 50000 pama node 500 okhala ndi zotengera 20 pamphindikati. Palibe zovuta zomwe zidadziwika pakukulitsa. Zotsatira zake zinalengezedwa kale pa kulengeza kwa Baibulo loyamba. Maphunziro odziyimira pawokha omwe amayang'ana kwambiri pakugwiritsa ntchito komanso kugwiritsa ntchito zida amatsimikiziranso kuti Calico akuchita bwino kwambiri ngati Flannel. Mwachitsanzo:

Ntchitoyi ikukula mwachangu kwambiri, imathandizira ntchito pamayankho otchuka omwe amayendetsedwa ndi K8s, OpenShift, OpenStack, ndizotheka kugwiritsa ntchito Calico potumiza gulu pogwiritsa ntchito kops, pali zonena zomanga ma network a Service Mesh (nachi chitsanzo amagwiritsidwa ntchito molumikizana ndi Istio).

Yesani ndi Calico

Nthawi zambiri kugwiritsa ntchito vanila Kubernetes, kukhazikitsa CNI kumatsikira kugwiritsa ntchito fayilo calico.yaml, dawunilodi kuchokera patsamba lovomerezeka, pogwiritsa ntchito kubectl apply -f.

Monga lamulo, mtundu waposachedwa wa plugin umagwirizana ndi mitundu yaposachedwa ya 2-3 ya Kubernetes: kugwira ntchito m'matembenuzidwe akale sikuyesedwa ndipo sikutsimikiziridwa. Malinga ndi omwe akupanga, Calico imayendetsa ma kernels a Linux pamwamba pa 3.10 omwe akuthamanga CentOS 7, Ubuntu 16 kapena Debian 8, pamwamba pa iptables kapena IPVS.

Kudzipatula mkati mwa chilengedwe

Kuti timvetsetse bwino, tiyeni tiwone nkhani yosavuta kuti timvetsetse momwe ma netiweki amachitidwe mu Calico notation amasiyanirana ndi omwe ali oyenera komanso momwe njira yopangira malamulo imathandizira kuwerengeka kwawo komanso kusinthasintha kwa kasinthidwe:

Pali mapulogalamu awiri a pa intaneti omwe ayikidwa mugululi: mu Node.js ndi PHP, imodzi mwazomwe zimagwiritsa ntchito Redis. Kuletsa kulowa kwa Redis kuchokera ku PHP, ndikusunga kulumikizana ndi Node.js, ingogwiritsani ntchito mfundo zotsatirazi:

kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-redis-nodejs
spec:
  podSelector:
    matchLabels:
      service: redis
  ingress:
  - from:
    - podSelector:
        matchLabels:
          service: nodejs
    ports:
    - protocol: TCP
      port: 6379

Kwenikweni tidalola magalimoto obwera kudoko la Redis kuchokera ku Node.js. Ndipo mwachiwonekere sanaletse china chirichonse. NetworkPolicy ikangowonekera, osankhidwa onse omwe atchulidwamo amayamba kudzipatula, pokhapokha atanenedwa mwanjira ina. Komabe, malamulo odzipatula sagwira ntchito kuzinthu zina zomwe sizinapangidwe ndi wosankha.

Chitsanzo amagwiritsa apiVersion Kubernetes kunja kwa bokosi, koma palibe chomwe chimakulepheretsani kugwiritsa ntchito gwero la dzina lomwelo kuchokera kumayendedwe a Calico. Kalembedwe kameneka kali ndi zambiri, kotero muyenera kulembanso lamulo la nkhani yomwe ili pamwambayi motere:

apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
  name: allow-redis-nodejs
spec:
  selector: service == 'redis'
  ingress:
  - action: Allow
    protocol: TCP
    source:
      selector: service == 'nodejs'
    destination:
      ports:
      - 6379

Zomwe tatchulazi zimapangidwira kulola kapena kukana magalimoto onse kudzera mu NetworkPolicy API yokhazikika imakhala ndi zomanga zomwe zimakhala zovuta kuzimvetsa ndikuzikumbukira. Pankhani ya Calico, kusintha malingaliro a lamulo la firewall kuti likhale losiyana, ingosintha action: Allow pa action: Deny.

Kudzipatula ndi chilengedwe

Tsopano lingalirani momwe ntchito imapanga ma metric abizinesi kuti asonkhanitse ku Prometheus ndikuwunikanso pogwiritsa ntchito Grafana. Zomwe zidakwezedwa zitha kukhala ndi data yachinsinsi, yomwe imatha kuwonedwanso ndi anthu mwachisawawa. Tiyeni tibise deta iyi kuti asayang'ane:

Prometheus, monga lamulo, imayikidwa m'malo ogwirira ntchito osiyana - mwachitsanzo, idzakhala malo monga awa:

apiVersion: v1
kind: Namespace
metadata:
  labels:
    module: prometheus
  name: kube-prometheus

m'munda metadata.labels izi sizinangochitika mwangozi. Monga tafotokozera pamwambapa, namespaceSelector (komanso podSelector) imagwira ntchito ndi zilembo. Chifukwa chake, kuti mulole kuti ma metric achotsedwe pamadontho onse padoko linalake, muyenera kuwonjezera zilembo zamtundu wina (kapena kutenga zomwe zilipo kale), ndiyeno gwiritsani ntchito kasinthidwe ngati:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-metrics-prom
spec:
  podSelector: {}
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          module: prometheus
    ports:
    - protocol: TCP
      port: 9100

Ndipo ngati mugwiritsa ntchito mfundo za Calico, mawuwo azikhala motere:

apiVersion: crd.projectcalico.org/v1
kind: NetworkPolicy
metadata:
  name: allow-metrics-prom
spec:
  ingress:
  - action: Allow
    protocol: TCP
    source:
      namespaceSelector: module == 'prometheus'
    destination:
      ports:
      - 9100

Nthawi zambiri, powonjezera mitundu iyi ya ndondomeko pazofuna zinazake, mutha kuteteza ku kusokonezedwa koyipa kapena mwangozi pakugwiritsa ntchito mapulogalamu mumagulu.

Njira yabwino kwambiri, malinga ndi omwe adapanga Calico, ndi njira ya "Lekani chilichonse ndikutsegula zomwe mukufuna", zolembedwa mu. zolemba zovomerezeka (ena amatsatira njira yofananira - makamaka, in nkhani yomwe yatchulidwa kale).

Kugwiritsa Ntchito Zowonjezera Calico Zinthu

Ndiroleni ndikukumbutseni kuti kudzera mu seti yowonjezera ya Calico APIs mutha kuyang'anira kupezeka kwa node, osangokhala ndi ma pod. Mu chitsanzo chotsatira ntchito GlobalNetworkPolicy Kuthekera kopereka zopempha za ICMP mgululi kumatsekedwa (mwachitsanzo, ma pings kuchokera ku pod kupita ku mfundo, pakati pa ma pod, kapena kuchokera ku node kupita ku IP pod):

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: block-icmp
spec:
  order: 200
  selector: all()
  types:
  - Ingress
  - Egress
  ingress:
  - action: Deny
    protocol: ICMP
  egress:
  - action: Deny
    protocol: ICMP

Pankhani yomwe ili pamwambapa, ndizothekabe kuti ma cluster node "afikirane" wina ndi mnzake kudzera pa ICMP. Ndipo nkhaniyi imathetsedwa mwa njira GlobalNetworkPolicy, yogwiritsidwa ntchito ku bungwe HostEndpoint:

apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: deny-icmp-kube-02
spec:
  selector: "role == 'k8s-node'"
  order: 0
  ingress:
  - action: Allow
    protocol: ICMP
  egress:
  - action: Allow
    protocol: ICMP
---
apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: kube-02-eth0
  labels:
    role: k8s-node
spec:
  interfaceName: eth0
  node: kube-02
  expectedIPs: ["192.168.2.2"]

Mlandu wa VPN

Potsirizira pake, ndipereka chitsanzo chenichenicho chogwiritsira ntchito ntchito za Calico pazochitika zamagulu oyandikana nawo, pamene ndondomeko yokhazikika sikwanira. Kuti mupeze pulogalamu yapaintaneti, makasitomala amagwiritsa ntchito njira ya VPN, ndipo mwayiwu umayendetsedwa mwamphamvu ndipo umangotengera mndandanda wazinthu zomwe zimaloledwa kugwiritsidwa ntchito:

Makasitomala amalumikizana ndi VPN kudzera pa doko lodziwika bwino la UDP 1194 ndipo, akalumikizidwa, amalandira njira zopita kumagulu am'magulu a ma pod ndi ntchito. Ma subnet onse amakankhidwa kuti asataye ntchito pakuyambiranso ndikusintha maadiresi.

Doko mu kasinthidwe ndi lokhazikika, lomwe limayika ma nuances ena pakukonzekera kugwiritsa ntchito ndikusamutsira ku gulu la Kubernetes. Mwachitsanzo, mu AWS LoadBalancer yemweyo wa UDP adawonekera kwenikweni kumapeto kwa chaka chatha pamndandanda wocheperako wa zigawo, ndipo NodePort siingagwiritsidwe ntchito chifukwa chotumizira pamagulu onse amgulu ndipo ndizosatheka kukulitsa kuchuluka kwa ma seva a zolakwa kulolera zolinga. Komanso, muyenera kusintha mtundu wa madoko ...

Chifukwa cha kufufuza njira zothetsera mavuto, zotsatirazi zinasankhidwa:

1. Ma Pod okhala ndi VPN amakonzedwa pa node iliyonse hostNetwork, ndiye kuti, ku IP yeniyeni.
2. Ntchitoyi imatumizidwa kunja ClusterIP. Doko limayikidwa pamfundoyi, yomwe imapezeka kunja ndikusungitsa pang'ono (kukhalapo kwa adilesi yeniyeni ya IP).
3. Kuzindikira node yomwe duwa la rose silingathe kupitilira nkhani yathu. Ndingonena kuti mutha "kukhomerera" mwamphamvu ntchitoyo ku node kapena kulemba ntchito yaying'ono yam'mbali yomwe ingayang'anire ma adilesi a IP apano a ntchito ya VPN ndikusintha zolemba za DNS zolembetsedwa ndi makasitomala - aliyense amene ali ndi malingaliro okwanira.

Kuchokera pamayendedwe, titha kuzindikira mwapadera kasitomala wa VPN ndi adilesi yake ya IP yoperekedwa ndi seva ya VPN. Pansipa pali chitsanzo choyambirira choletsa mwayi wamakasitomala wotereyu kuzithandizo, zowonetsedwa pa Redis yomwe yatchulidwa pamwambapa:

apiVersion: crd.projectcalico.org/v1
kind: HostEndpoint
metadata:
  name: vpnclient-eth0
  labels:
    role: vpnclient
    environment: production
spec:
  interfaceName: "*"
  node: kube-02
  expectedIPs: ["172.176.176.2"]
---
apiVersion: crd.projectcalico.org/v1
kind: GlobalNetworkPolicy
metadata:
  name: vpn-rules
spec:
  selector: "role == 'vpnclient'"
  order: 0
  applyOnForward: true
  preDNAT: true
  ingress:
  - action: Deny
    protocol: TCP
    destination:
      ports: [6379]
  - action: Allow
    protocol: UDP
    destination:
      ports: [53, 67]

Apa, kulumikiza ku doko 6379 ndikoletsedwa kotheratu, koma nthawi yomweyo ntchito ya DNS imasungidwa, yomwe nthawi zambiri imavutika polemba malamulo. Chifukwa, monga tanenera kale, pamene wosankha akuwonekera, ndondomeko yotsutsa yokhazikika imagwiritsidwa ntchito pokhapokha ngati itatchulidwa.

Zotsatira

Chifukwa chake, pogwiritsa ntchito Calico's advanced API, mutha kusintha mosinthika ndikusintha mayendedwe mkati ndi kuzungulira gululo. Nthawi zambiri, kugwiritsa ntchito kwake kumatha kuwoneka ngati mpheta zowombera ndi cannon, ndikukhazikitsa maukonde a L3 okhala ndi ma tunnel a BGP ndi IP-IP amawoneka owopsa pakuyika kwa Kubernetes pamaneti osakhazikika ... .

Kupatula gulu kuti likwaniritse zofunikira zachitetezo sikungakhale kotheka nthawi zonse, ndipo apa ndipamene Calico (kapena njira yofananira) imabwera kudzapulumutsa. Zitsanzo zomwe zaperekedwa m'nkhaniyi (ndi zosintha zazing'ono) zimagwiritsidwa ntchito pakuyika zingapo zamakasitomala athu mu AWS.

PS

Werenganinso pa blog yathu:

• «Chiyambi cha Kubernetes Network Policies for Security Professionals";
• "An Illustrated Guide to Networking in Kubernetes": Gawo 1 ndi 2 (chitsanzo cha netiweki, maukonde ophatikizika), Gawo 3 (ntchito ndi kukonza magalimoto);
• «Container Networking Interface (CNI) - mawonekedwe a netiweki ndi muyezo wa zida za Linux".

Source: www.habr.com