Kusukela empilweni noKubernetes: Iseva ye-HTTP ayizange ithande kanjani abaseSpain

Ummeleli weklayenti lethu, isitaki sakhe sohlelo lokusebenza sihlala efwini leMicrosoft (Azure), ubhekane nenkinga: muva nje, ezinye izicelo ezivela kwamanye amaklayenti avela eYurophu ziqale ukuphela ngephutha 400 (Isicelo esibi). Zonke izinhlelo zokusebenza zibhalwe ku-.NET, zisetshenziswe ku-Kubernetes...

Enye yezinhlelo zokusebenza yi-API, lapho yonke ithrafikhi ifika ekugcineni. Le thrafikhi ilalelwa yiseva ye-HTTP UKestrel, ilungiselelwe iklayenti le-NET futhi isingathwe ku-pod. Ngokulungisa iphutha, sibe nenhlanhla ngomqondo wokuthi bekunomsebenzisi othile okhiqize kabusha inkinga ngokungashintshi. Kodwa-ke, yonke into yayiyinkimbinkimbi ngochungechunge lwethrafikhi:

Iphutha ku-Ingress libukeke kanje:

{
   "number_fields":{
      "status":400,
      "request_time":0.001,
      "bytes_sent":465,
      "upstream_response_time":0,
      "upstream_retries":0,
      "bytes_received":2328
   },
   "stream":"stdout",
   "string_fields":{
      "ingress":"app",
      "protocol":"HTTP/1.1",
      "request_id":"f9ab8540407208a119463975afda90bc",
      "path":"/api/sign-in",
      "nginx_upstream_status":"400",
      "service":"app",
      "namespace":"production",
      "location":"/front",
      "scheme":"https",
      "method":"POST",
      "nginx_upstream_response_time":"0.000",
      "nginx_upstream_bytes_received":"120",
      "vhost":"api.app.example.com",
      "host":"api.app.example.com",
      "user":"",
      "address":"83.41.81.250",
      "nginx_upstream_addr":"10.240.0.110:80",
      "referrer":"https://api.app.example.com/auth/login?long_encrypted_header",
      "service_port":"http",
      "user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36",
      "time":"2019-03-06T18:29:16+00:00",
      "content_kind":"cache-headers-not-present",
      "request_query":""
   },
   "timestamp":"2019-03-06 18:29:16",
   "labels":{
      "app":"nginx",
      "pod-template-generation":"6",
      "controller-revision-hash":"1682636041"
   },
   "namespace":"kube-nginx-ingress",
   "nsec":6726612,
   "source":"kubernetes",
   "host":"k8s-node-55555-0",
   "pod_name":"nginx-v2hcb",
   "container_name":"nginx",
   "boolean_fields":{}
}

Ngesikhathi esifanayo, uKestrel wanikeza:

HTTP/1.1 400 Bad Request
Connection: close
Date: Wed, 06 Mar 2019 12:34:20 GMT
Server: Kestrel
Content-Length: 0

Noma ngabe i-verbosity ephezulu, iphutha le-Kestrel laliqukethe ngokwedlulele ulwazi oluncane oluwusizo:

{
   "number_fields":{"ThreadId":76},
   "stream":"stdout",
   "string_fields":{
      "EventId":"{"Id"=>17, "Name"=>"ConnectionBadRequest"}",
      "SourceContext":"Microsoft.AspNetCore.Server.Kestrel",
      "ConnectionId":"0HLL2VJSST5KV",
      "@mt":"Connection id "{ConnectionId}" bad request data: "{message}"",
      "@t":"2019-03-07T13:06:48.1449083Z",
      "@x":"Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Malformed request: invalid headers.n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.TryParseRequest(ReadResult result, Boolean& endConnection)n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<ProcessRequestsAsync>d__185`1.MoveNext()",
      "message":"Malformed request: invalid headers."
   },
   "timestamp":"2019-03-07 13:06:48",
   "labels":{
      "pod-template-hash":"2368795483",
      "service":"app"
   },
   "namespace":"production",
   "nsec":145341848,
   "source":"kubernetes",
   "host":"k8s-node-55555-1",
   "pod_name":"app-67bdcf98d7-mhktx",
   "container_name":"app",
   "boolean_fields":{}
}

Kungabonakala sengathi i-tcpdump kuphela ezosiza ukuxazulula le nkinga... kodwa ngizophinda mayelana nochungechunge lwethrafikhi:

Uphenyo

Ngokusobala, kungcono ukulalela ithrafikhi kuleyo node ethile, lapho u-Kubernetes efake i-pod: umthamo wokulahla uzoba kangangokuthi kuzokwazi ukuthola okungenani okuthile ngokushesha. Futhi ngempela, lapho kuhlolwa, uhlaka olulandelayo lwaqashelwa:

GET /back/user HTTP/1.1
Host: api.app.example.com
X-Request-ID: 27ceb14972da8c21a8f92904b3eff1e5
X-Real-IP: 83.41.81.250
X-Forwarded-For: 83.41.81.250
X-Forwarded-Host: api.app.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Original-URI: /front/back/user
X-Scheme: https
X-Original-Forwarded-For: 83.41.81.250
X-Nginx-Geo-Client-Country: Spain
X-Nginx-Geo-Client-City: M.laga
Accept-Encoding: gzip
CF-IPCountry: ES
CF-RAY: 4b345cfd1c4ac691-MAD
CF-Visitor: {"scheme":"https"}
pragma: no-cache
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36
referer: https://app.example.com/auth/login
accept-language: en-US,en;q=0.9,en-GB;q=0.8,pl;q=0.7
cookie: many_encrypted_cookies; .AspNetCore.Identity.Application=something_encrypted; 
CF-Connecting-IP: 83.41.81.250
True-Client-IP: 83.41.81.250
CDN-Loop: cloudflare

HTTP/1.1 400 Bad Request
Connection: close
Date: Wed, 06 Mar 2019 12:34:20 GMT
Server: Kestrel
Content-Length: 0

Lapho kubhekisiswa kahle indawo yokulahla imfucuza, igama laqashelwa M.laga. Kulula ukuqagela ukuthi alikho idolobha laseM.laga eSpain (kodwa likhona Málaga). Sithatha lo mbono, sibheke ama-Ingress configs, lapho sabona khona lowo ofakwe enyangeni edlule (ngesicelo seklayenti) amazwibela "angenabungozi".:

    ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header X-Nginx-Geo-Client-Country $geoip_country_name;
      proxy_set_header X-Nginx-Geo-Client-City $geoip_city;

Ngemva kokukhubaza ukudluliselwa kwalezi zihloko, yonke into ihambe kahle! (Kusheshe kwaba sobala ukuthi uhlelo lokusebenza ngokwalo alusadingi lezi zihloko.)

Manje ake sibheke inkinga ngokujwayelekile. Ingenziwa kabusha kalula ngaphakathi kohlelo lokusebenza ngokwenza isicelo se-telnet ku localhost:80:

GET /back/user HTTP/1.1
Host: api.app.example.com
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
Cookie: test=Desiree

... uyabuya 401 Unauthorized, njengoba bekulindelekile. Kwenzekani uma senza:

GET /back/user HTTP/1.1
Host: api.app.example.com
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
Cookie: test=Désirée

?

Izobuya 400 Bad request - kulogi yohlelo lokusebenza sizothola iphutha esivele silijwayele:

{
   "@t":"2019-03-31T12:59:54.3746446Z",
   "@mt":"Connection id "{ConnectionId}" bad request data: "{message}"",
   "@x":"Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Malformed request: invalid headers.n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.TryParseRequest(ReadResult result, Boolean& endConnection)n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<ProcessRequestsAsync>d__185`1.MoveNext()",
   "ConnectionId":"0HLLLR1J974L9",
   "message":"Malformed request: invalid headers.",
   "EventId":{
      "Id":17,
      "Name":"ConnectionBadRequest"
   },
   "SourceContext":"Microsoft.AspNetCore.Server.Kestrel",
   "ThreadId":71
}

Imiphumela

Ngokukhethekile Kestrel ngeke cubungula kahle izihloko ze-HTTP ezinezinhlamvu ezifanele ku-UTF-8, eziqukethwe emagameni wenani elikhulu lamadolobha.

Isici esingeziwe esimweni sethu ukuthi iklayenti okwamanje ayihleli ukushintsha ukusetshenziswa kwe-Kestrel kuhlelo lokusebenza. Nokho, izinkinga ku-AspNetCore ngokwayo (No.4318, No.7707) bathi ngeke kusize lokhu...

Ukufingqa: inothi ayisekho ngezinkinga ezithile ze-Kestrel noma i-UTF-8 (ngo-2019?!), kodwa mayelana neqiniso lokuthi ingqondo kanye nokufunda okungaguquki Zonke izinyathelo ozithathayo ngenkathi ucinga izinkinga zizothela izithelo maduze. Ngikufisela inhlanhla!

PS

Funda futhi kubhulogi yethu:

• «Izimbungulu zesistimu yokuzijabulisa eyi-6 ekusebenzeni kwe-Kubernetes [nesixazululo sazo]";
• «Amathiphu namasu we-Kubernetes: Amakhasi ephutha enziwe aba ngewakho ku-NGINX Ingress";
• «Ukubuka konke nokuqhathaniswa kwezilawuli ze-Ingress ze-Kubernetes";
• «Ukuqapha ama-pings phakathi kwama-node e-Kubernetes - iresiphi yethu";
• «Izimo ezi-3 ezingajwayelekile mayelana nesistimu engaphansi yenethiwekhi ye-Linux".

Source: www.habr.com