Kubva kuhupenyu naKubernetes: Sevha yeHTTP haina kufarira maSpanish

Mumiriri wemutengi wedu, ane stack yekunyorera inogara muMicrosoft gore (Azure), akagadzirisa dambudziko: nguva pfupi yadarika, zvimwe zvikumbiro kubva kune vamwe vatengi kubva kuEurope zvakatanga kupera nekukanganisa 400 (Chikumbiro Chakaipa) Zvese zvinoshandiswa zvakanyorwa mu.NET, zvakaiswa muKubernetes...

Imwe yemashandisirwo ndeye API, kuburikidza iyo traffic yese inozouya. Iyi traffic inoteererwa neHTTP server kestrel, yakagadziridzwa ne.NET client uye yakagarwa mune pod. Nekugadzirisa, takaita rombo rakanaka mupfungwa yekuti paive nemushandisi aigara achigadzira dambudziko. Nekudaro, zvese zvaive zvakaomeswa neiyo traffic cheni:

Iko kukanganisa muIngress kwakaita seizvi:

{
   "number_fields":{
      "status":400,
      "request_time":0.001,
      "bytes_sent":465,
      "upstream_response_time":0,
      "upstream_retries":0,
      "bytes_received":2328
   },
   "stream":"stdout",
   "string_fields":{
      "ingress":"app",
      "protocol":"HTTP/1.1",
      "request_id":"f9ab8540407208a119463975afda90bc",
      "path":"/api/sign-in",
      "nginx_upstream_status":"400",
      "service":"app",
      "namespace":"production",
      "location":"/front",
      "scheme":"https",
      "method":"POST",
      "nginx_upstream_response_time":"0.000",
      "nginx_upstream_bytes_received":"120",
      "vhost":"api.app.example.com",
      "host":"api.app.example.com",
      "user":"",
      "address":"83.41.81.250",
      "nginx_upstream_addr":"10.240.0.110:80",
      "referrer":"https://api.app.example.com/auth/login?long_encrypted_header",
      "service_port":"http",
      "user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36",
      "time":"2019-03-06T18:29:16+00:00",
      "content_kind":"cache-headers-not-present",
      "request_query":""
   },
   "timestamp":"2019-03-06 18:29:16",
   "labels":{
      "app":"nginx",
      "pod-template-generation":"6",
      "controller-revision-hash":"1682636041"
   },
   "namespace":"kube-nginx-ingress",
   "nsec":6726612,
   "source":"kubernetes",
   "host":"k8s-node-55555-0",
   "pod_name":"nginx-v2hcb",
   "container_name":"nginx",
   "boolean_fields":{}
}

Panguva imwecheteyo, Kestrel akapa:

HTTP/1.1 400 Bad Request
Connection: close
Date: Wed, 06 Mar 2019 12:34:20 GMT
Server: Kestrel
Content-Length: 0

Kunyangwe iine verbosity yakanyanya, iyo Kestrel kukanganisa yaive yakanyanyisa ruzivo rushoma runobatsira:

{
   "number_fields":{"ThreadId":76},
   "stream":"stdout",
   "string_fields":{
      "EventId":"{"Id"=>17, "Name"=>"ConnectionBadRequest"}",
      "SourceContext":"Microsoft.AspNetCore.Server.Kestrel",
      "ConnectionId":"0HLL2VJSST5KV",
      "@mt":"Connection id "{ConnectionId}" bad request data: "{message}"",
      "@t":"2019-03-07T13:06:48.1449083Z",
      "@x":"Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Malformed request: invalid headers.n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.TryParseRequest(ReadResult result, Boolean& endConnection)n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<ProcessRequestsAsync>d__185`1.MoveNext()",
      "message":"Malformed request: invalid headers."
   },
   "timestamp":"2019-03-07 13:06:48",
   "labels":{
      "pod-template-hash":"2368795483",
      "service":"app"
   },
   "namespace":"production",
   "nsec":145341848,
   "source":"kubernetes",
   "host":"k8s-node-55555-1",
   "pod_name":"app-67bdcf98d7-mhktx",
   "container_name":"app",
   "boolean_fields":{}
}

Zvinoita sekunge tcpdump chete ndiyo ichabatsira kugadzirisa dambudziko iri ... asi ini ndichadzokorora nezve traffic chain:

Kuferefeta

Zviripachena, zviri nani kuteerera traffic pane iyo node chaiyo, uko Kubernetes akatumira pod: huwandu hwekuraswa huchave hwakadaro zvekuti zvinokwanisika kuwana chero chinhu nekukurumidza. Uye zvechokwadi, pakuiongorora, iyo inotevera furemu yakaonekwa:

GET /back/user HTTP/1.1
Host: api.app.example.com
X-Request-ID: 27ceb14972da8c21a8f92904b3eff1e5
X-Real-IP: 83.41.81.250
X-Forwarded-For: 83.41.81.250
X-Forwarded-Host: api.app.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Original-URI: /front/back/user
X-Scheme: https
X-Original-Forwarded-For: 83.41.81.250
X-Nginx-Geo-Client-Country: Spain
X-Nginx-Geo-Client-City: M.laga
Accept-Encoding: gzip
CF-IPCountry: ES
CF-RAY: 4b345cfd1c4ac691-MAD
CF-Visitor: {"scheme":"https"}
pragma: no-cache
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36
referer: https://app.example.com/auth/login
accept-language: en-US,en;q=0.9,en-GB;q=0.8,pl;q=0.7
cookie: many_encrypted_cookies; .AspNetCore.Identity.Application=something_encrypted; 
CF-Connecting-IP: 83.41.81.250
True-Client-IP: 83.41.81.250
CDN-Loop: cloudflare

HTTP/1.1 400 Bad Request
Connection: close
Date: Wed, 06 Mar 2019 12:34:20 GMT
Server: Kestrel
Content-Length: 0

Pakunyatsotariswa kwerasira, shoko rakaonekwa M.laga. Zviri nyore kufungidzira kuti hakuna guta reM.laga muSpain (asi ririko Málaga) Tichibata pane iyi pfungwa, takatarisa iyo Ingress configs, kwatakaona iyo yakaiswa mwedzi wapfuura (pakukumbira kwemutengi) "isina ngozi" snippet:

    ingress.kubernetes.io/configuration-snippet: |
      proxy_set_header X-Nginx-Geo-Client-Country $geoip_country_name;
      proxy_set_header X-Nginx-Geo-Client-City $geoip_city;

Mushure mekudzima kuendeswa mberi kweiyi misoro, zvese zvakave zvakanaka! (Nenguva isipi zvakava pachena kuti iyo application pachayo yakanga isingachade iyi misoro.)

Zvino ngatitarisei dambudziko zvakanyanya. Inogona kudhindwa zvakare nyore mukati mekushandisa nekuita chikumbiro che telnet kune localhost:80:

GET /back/user HTTP/1.1
Host: api.app.example.com
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
Cookie: test=Desiree

... anodzoka 401 Unauthorized, sezvaitarisirwa. Chii chinoitika kana tikaita:

GET /back/user HTTP/1.1
Host: api.app.example.com
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
Cookie: test=Désirée

?

Achadzoka 400 Bad request - mugwaro rekushandisa tinogashira chikanganiso chatinoziva kwatiri:

{
   "@t":"2019-03-31T12:59:54.3746446Z",
   "@mt":"Connection id "{ConnectionId}" bad request data: "{message}"",
   "@x":"Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Malformed request: invalid headers.n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.TryParseRequest(ReadResult result, Boolean& endConnection)n   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<ProcessRequestsAsync>d__185`1.MoveNext()",
   "ConnectionId":"0HLLLR1J974L9",
   "message":"Malformed request: invalid headers.",
   "EventId":{
      "Id":17,
      "Name":"ConnectionBadRequest"
   },
   "SourceContext":"Microsoft.AspNetCore.Server.Kestrel",
   "ThreadId":71
}

Migumisiro

Kunyanya Kestrel handikwanise gadzirisa nemazvo misoro yeHTTP ine mavara akakwana muUTF-8, ari mumazita ehuwandu hwakakura hwemaguta.

Chimwezve chinhu munyaya yedu ndechekuti mutengi haaronge kushandura Kestrel mukushandisa. Nekudaro, nyaya muAspNetCore pachayo (Nha .4318, Nha .7707) vanoti izvi hazvibatsire...

Kupfupisa: iyo noti haisisiri nezve chaiwo matambudziko eKestrel kana UTF-8 (muna 2019?!), Asi nezve chokwadi chekuti kungwarira uye kudzidza kwakafanana Nhanho yega yega yaunotora uchitsvaga matambudziko inozobereka zvibereko. Rombo rakanaka!

PS

Verenga zvakare pablog yedu:

• «6 inovaraidza system bugs mukushanda kweKubernetes [uye mhinduro yavo]";
• «Kubernetes matipi & matipi: tsika yekukanganisa mapeji muNGINX Ingress";
• «Kutarisisa uye kuenzanisa kweIngress controllers yeKubernetes";
• «Kutarisa pings pakati peKubernetes node - yedu resipi";
• «3 kesi dzisina kujairika nezve Linux network subsystem".

Source: www.habr.com