E Vertrieder vun eisem Client, deem säin Applikatiounsstack an der Microsoft Cloud (Azure) wunnt, huet e Problem adresséiert: viru kuerzem hunn e puer Ufroe vun e puer Clienten aus Europa ugefaang mam Feeler 400 (). All Applikatiounen sinn an .NET geschriwwe ginn, a Kubernetes ofgesat ...
Eng vun den Applikatiounen ass d'API, duerch déi all Traffic schlussendlech kënnt. Dëse Traffic gëtt vum HTTP-Server nogelauschtert , konfiguréiert vum .NET Client an an engem Pod gehost. Mat Debugging ware mir Gléck am Sënn datt et e spezifesche Benotzer war deen de Problem konsequent reproduzéiert huet. Allerdéngs war alles komplizéiert vun der Verkéierskette:
De Feeler am Ingress huet esou ausgesinn:
{
"number_fields":{
"status":400,
"request_time":0.001,
"bytes_sent":465,
"upstream_response_time":0,
"upstream_retries":0,
"bytes_received":2328
},
"stream":"stdout",
"string_fields":{
"ingress":"app",
"protocol":"HTTP/1.1",
"request_id":"f9ab8540407208a119463975afda90bc",
"path":"/api/sign-in",
"nginx_upstream_status":"400",
"service":"app",
"namespace":"production",
"location":"/front",
"scheme":"https",
"method":"POST",
"nginx_upstream_response_time":"0.000",
"nginx_upstream_bytes_received":"120",
"vhost":"api.app.example.com",
"host":"api.app.example.com",
"user":"",
"address":"83.41.81.250",
"nginx_upstream_addr":"10.240.0.110:80",
"referrer":"https://api.app.example.com/auth/login?long_encrypted_header",
"service_port":"http",
"user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36",
"time":"2019-03-06T18:29:16+00:00",
"content_kind":"cache-headers-not-present",
"request_query":""
},
"timestamp":"2019-03-06 18:29:16",
"labels":{
"app":"nginx",
"pod-template-generation":"6",
"controller-revision-hash":"1682636041"
},
"namespace":"kube-nginx-ingress",
"nsec":6726612,
"source":"kubernetes",
"host":"k8s-node-55555-0",
"pod_name":"nginx-v2hcb",
"container_name":"nginx",
"boolean_fields":{}
}Zur selwechter Zäit huet Kestrel:
HTTP/1.1 400 Bad Request
Connection: close
Date: Wed, 06 Mar 2019 12:34:20 GMT
Server: Kestrel
Content-Length: 0Och mat maximaler Verbositéit huet de Kestrel Feeler extrem enthale wéineg nëtzlech Informatiounen:
{
"number_fields":{"ThreadId":76},
"stream":"stdout",
"string_fields":{
"EventId":"{"Id"=>17, "Name"=>"ConnectionBadRequest"}",
"SourceContext":"Microsoft.AspNetCore.Server.Kestrel",
"ConnectionId":"0HLL2VJSST5KV",
"@mt":"Connection id "{ConnectionId}" bad request data: "{message}"",
"@t":"2019-03-07T13:06:48.1449083Z",
"@x":"Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Malformed request: invalid headers.n at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.TryParseRequest(ReadResult result, Boolean& endConnection)n at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<ProcessRequestsAsync>d__185`1.MoveNext()",
"message":"Malformed request: invalid headers."
},
"timestamp":"2019-03-07 13:06:48",
"labels":{
"pod-template-hash":"2368795483",
"service":"app"
},
"namespace":"production",
"nsec":145341848,
"source":"kubernetes",
"host":"k8s-node-55555-1",
"pod_name":"app-67bdcf98d7-mhktx",
"container_name":"app",
"boolean_fields":{}
}Et géif schéngen datt nëmmen tcpdump hëlleft dëse Problem ze léisen ... awer ech widderhuelen iwwer d'Traffic Kette:
Ënnersichung
Natierlech ass et besser de Traffic ze lauschteren op deem spezifesche Node, wou Kubernetes e Pod agesat huet: de Volume vum Dump wäert esou sinn datt et méiglech ass op d'mannst eppes zimlech séier ze fannen. An tatsächlech, wann Dir et iwwerpréift, gouf de folgende Frame gemierkt:
GET /back/user HTTP/1.1
Host: api.app.example.com
X-Request-ID: 27ceb14972da8c21a8f92904b3eff1e5
X-Real-IP: 83.41.81.250
X-Forwarded-For: 83.41.81.250
X-Forwarded-Host: api.app.example.com
X-Forwarded-Port: 443
X-Forwarded-Proto: https
X-Original-URI: /front/back/user
X-Scheme: https
X-Original-Forwarded-For: 83.41.81.250
X-Nginx-Geo-Client-Country: Spain
X-Nginx-Geo-Client-City: M.laga
Accept-Encoding: gzip
CF-IPCountry: ES
CF-RAY: 4b345cfd1c4ac691-MAD
CF-Visitor: {"scheme":"https"}
pragma: no-cache
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36
referer: https://app.example.com/auth/login
accept-language: en-US,en;q=0.9,en-GB;q=0.8,pl;q=0.7
cookie: many_encrypted_cookies; .AspNetCore.Identity.Application=something_encrypted;
CF-Connecting-IP: 83.41.81.250
True-Client-IP: 83.41.81.250
CDN-Loop: cloudflare
HTTP/1.1 400 Bad Request
Connection: close
Date: Wed, 06 Mar 2019 12:34:20 GMT
Server: Kestrel
Content-Length: 0
Bei enger méi genauer Inspektioun vum Deponie gouf d'Wuert opgefall M.laga. Et ass einfach ze roden datt et keng M.laga Stad a Spuenien gëtt (awer et gëtt ). Op dëser Iddi erfaasst, hu mir d'Ingress Konfiguratioune gekuckt, wou mir déi gesinn, déi virun engem Mount agefouert goufen (op Ufro vum Client) "harmlos" Snippet:
ingress.kubernetes.io/configuration-snippet: |
proxy_set_header X-Nginx-Geo-Client-Country $geoip_country_name;
proxy_set_header X-Nginx-Geo-Client-City $geoip_city;Nodeems de Forward vun dësen Header deaktivéiert gouf, ass alles gutt ginn! (Et gouf séier kloer datt d'Applikatioun selwer dës Header net méi brauch.)
Loosst eis elo de Problem kucken méi allgemeng. Et kann einfach an der Applikatioun reproduzéiert ginn andeems Dir eng Telnet Ufro mécht localhost:80:
GET /back/user HTTP/1.1
Host: api.app.example.com
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
Cookie: test=Desiree
... zréck 401 Unauthorized, wéi erwaart. Wat geschitt wa mir maachen:
GET /back/user HTTP/1.1
Host: api.app.example.com
cache-control: no-cache
accept: application/json, text/plain, */*
origin: https://app.example.com
Cookie: test=Désirée?
Wäert zréck 400 Bad request - am Applikatiounsprotokoll kréie mir e Feeler deen eis scho vertraut ass:
{
"@t":"2019-03-31T12:59:54.3746446Z",
"@mt":"Connection id "{ConnectionId}" bad request data: "{message}"",
"@x":"Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Malformed request: invalid headers.n at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.Http1Connection.TryParseRequest(ReadResult result, Boolean& endConnection)n at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.<ProcessRequestsAsync>d__185`1.MoveNext()",
"ConnectionId":"0HLLLR1J974L9",
"message":"Malformed request: invalid headers.",
"EventId":{
"Id":17,
"Name":"ConnectionBadRequest"
},
"SourceContext":"Microsoft.AspNetCore.Server.Kestrel",
"ThreadId":71
}Resultater
Speziell Kestrel korrekt veraarbecht HTTP-Header mat de korrekten Zeechen an UTF-8, déi an den Nimm vun enger zimlech grousser Zuel vu Stied enthale sinn.
En zousätzleche Faktor an eisem Fall ass datt de Client de Moment net plangt d'Ëmsetzung vu Kestrel an der Applikatioun z'änneren. Wéi och ëmmer, Themen an AspNetCore selwer (, ) se soen dat dat net hëlleft...
Fir ze resuméieren: d'Notiz geet net méi iwwer déi spezifesch Probleemer vu Kestrel oder UTF-8 (am 2019?!), mee iwwer d'Tatsaach datt mindfulness a konsequent Studie All Schrëtt, deen Dir maacht wärend Dir no Probleemer sicht, wäert fréier oder spéider Fruucht droen. Vill Gléck!
PS
Liest och op eisem Blog:
- «";
- «";
- «";
- «";
- «".
Source: will.com