{"id":80532,"date":"2020-05-07T01:42:17","date_gmt":"2020-05-06T23:42:17","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/problemy-s-dns-v-kubernetes-publichnyj-postmortem"},"modified":"2020-05-07T01:42:17","modified_gmt":"2020-05-06T23:42:17","slug":"problemy-s-dns-v-kubernetes-publichnyj-postmortem","status":"publish","type":"post","link":"https:\/\/prohoster.info\/bs\/blog\/administrirovanie\/problemy-s-dns-v-kubernetes-publichnyj-postmortem","title":{"rendered":"Problemi sa DNS-om u Kubernetesu. Javna postmortem analiza","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><i><b>Napomena prev.:<\/b> ovo je prijevod javnog postmortema iz in\u017eenjerskog bloga kompanije <noindex><a rel=\"nofollow\" href=\"https:\/\/preply.com\/\">Preply<\/a><\/noindex>. U njemu se opisuje problem s conntrack u Kubernetes klasteru, koji je doveo do djelomi\u010dne nedostupnosti nekih produkcijskih usluga.<\/i><\/p>\n<p>Ovaj \u010dlanak mo\u017ee biti koristan onima koji \u017eele saznati ne\u0161to vi\u0161e o postmortemima ili sprije\u010diti neke potencijalne probleme s DNS-om u budu\u0107nosti.<\/p>\n<p><img decoding=\"async\" alt=\"Problemi sa DNS-om u Kubernetesu. Javna postmortem analiza\" src=\"\/wp-content\/uploads\/2020\/05\/0e841b0231f11171d52d82030cd1df21.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>To nije DNS<br \/>\nNe mo\u017ee biti da je to DNS<br \/>\nTo je bio DNS<\/i><br \/>\n<noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2>Malo o postmortemima i procesima u Preply<\/h2>\n<p><\/p>\n<blockquote><p>U postmortemu se opisuje kvar u radu ili neki doga\u0111aj u produkciji. Postmortem uklju\u010duje hronologiju doga\u0111aja, opis uticaja na korisnike, korijenski uzrok, akcije i nau\u010dene lekcije.<\/p>\n<p><i><noindex><a rel=\"nofollow\" href=\"http:\/\/shop.oreilly.com\/product\/0636920063964.do\">Seeking SRE<\/a><\/noindex><\/i><\/p><\/blockquote>\n<p>\nNa nedjeljnim sastancima uz pizzu, u krugu tehni\u010dkog tima, dijelimo razne informacije. Jedan od najva\u017enijih dijelova takvih sastanaka su postmortemi, koji naj\u010de\u0161\u0107e dolaze uz prezentaciju sa slajdovima i dubljom analizom nastalog incidenta. Iako ne \u2018apludiramo\u2019 nakon postmortema, trudimo se razvijati kulturu 'bez okrivljavanja' (<noindex><a rel=\"nofollow\" href=\"https:\/\/codeascraft.com\/2012\/05\/22\/blameless-postmortems\/\">blameless cluture<\/a><\/noindex>). Vjerujemo da pisanje i predstavljanje postmortema mo\u017ee pomo\u0107i nama (i ne samo nama) u spre\u010davanju sli\u010dnih incidenata u budu\u0107nosti, upravo zato ih i dijelimo.<\/p>\n<blockquote><p>Osobe uklju\u010dene u incident moraju osje\u0107ati da mogu detaljno govoriti o njemu, ne boje\u0107i se kazne ili odmazde. Nema optu\u017eivanja! Pisanje postmortema nije kazna, ve\u0107 mogu\u0107nost u\u010denja za cijelu kompaniju.<\/p>\n<p><i><noindex><a rel=\"nofollow\" href=\"https:\/\/devblog.axway.com\/dev-insights\/keep-calms-devops-s-sharing\/\">Keep CALMS &amp; DevOps: S je za Dijeljenje<\/a><\/noindex><\/i><\/p><\/blockquote>\n<p><\/p>\n<h2>Problemi s DNS-om u Kubernetesu. Postmortem<\/h2>\n<p>\n<b>Datum:<\/b> 28.02.2020<\/p>\n<p><b>Autori:<\/b> Amet U., Andrej S., Igor K., Aleksei P.<\/p>\n<p><b>Status:<\/b> Zavr\u0161eno<\/p>\n<p><b>Ukratko:<\/b> Djelomi\u010dna nedostupnost DNS-a (26 min) za neke usluge u Kubernetes klasteru<\/p>\n<p><b>Uticaj:<\/b> 15000 doga\u0111aja izgubljeno za usluge A, B i C<\/p>\n<p><b>Korijenski uzrok:<\/b> Kube-proxy nije mogao ispravno ukloniti staru stavku iz tabele conntrack, zbog \u010dega su se neke usluge i dalje poku\u0161avale povezati na nepostoje\u0107e podove<\/p>\n<pre><code class=\"bash\">E0228 20:13:53.795782       1 proxier.go:610] Failed to delete kube-system\/kube-dns:dns endpoint connections, error: error deleting conntrack entries for UDP peer {100.64.0.10, 100.110.33.231}, error: conntrack command returned: ...<\/code><\/pre>\n<p>\n<b>Okida\u010d:<\/b> Zbog niskog optere\u0107enja unutar Kubernetes klastera, CoreDNS-autoscaler smanjio je broj podova u deploymentu sa tri na dva<\/p>\n<p><b>Re\u0161enje:<\/b> Novo implementiranje aplikacije izazvalo je kreiranje novih \u010dvorova, CoreDNS-autoscaler je dodao vi\u0161e podova za podr\u0161ku klasteru, \u0161to je izazvalo prepisivanje conntrack tabele<\/p>\n<p><b>Otkrivanje:<\/b> Prometheus monitoring je otkrio veliki broj 5xx gre\u0161aka za servise A, B i C i inicirao poziv za de\u017eurne in\u017eenjere<\/p>\n<p><img decoding=\"async\" alt=\"Problemi sa DNS-om u Kubernetesu. Javna postmortem analiza\" src=\"\/wp-content\/uploads\/2020\/05\/14a2216ce7564f05c6dc3e7cb1c3dbb0.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>5xx gre\u0161ke u Kibani<\/i><\/p>\n<h3>Akcije<\/h3>\n<p><\/p>\n<p>Akcija<br \/>\nTip<br \/>\nOdgovorna osoba<br \/>\nZadatak<\/p>\n<p>Isklju\u010diti autoscaler za CoreDNS<br \/>\nspre\u010di.<br \/>\nAmet U.<br \/>\nDEVOPS-695<\/p>\n<p>Postaviti ke\u0161iraju\u0107i DNS server<br \/>\nsmanji.<br \/>\nMaks V.<br \/>\nDEVOPS-665<\/p>\n<p>Postaviti monitoring conntrack-a<br \/>\nspre\u010di.<br \/>\nAmet U.<br \/>\nDEVOPS-674<\/p>\n<p><\/p>\n<h3>Izvu\u010dene lekcije<\/h3>\n<p>\n<b>\u0160ta je dobro pro\u0161lo:<\/b><\/p>\n<ul>\n<li>Monitoring je radio efikasno. Reakcija je bila brza i organizovana<\/li>\n<li>Nismo nai\u0161li na nikakve limite na \u010dvorovima<\/li>\n<\/ul>\n<p><b>\u0160ta nije bilo u redu:<\/b><\/p>\n<ul>\n<li>Jo\u0161 uvek nepoznata stvarna prvobitna uzrok, izgleda kao <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/kubernetes\/kubernetes\/issues\/66651\">specifi\u010dna gre\u0161ka<\/a><\/noindex> u conntrack-u<\/li>\n<li>Sve akcije ispravljaju samo posledice, ne prvobitni uzrok (gre\u0161ka)<\/li>\n<li>Znali smo da \u0107emo \u0440\u0430\u043d\u043e ili kasno imati problema s DNS-om, ali nismo prioritizovali zadatke<\/li>\n<\/ul>\n<p>\n<b>Gde smo imali sre\u0107e:<\/b><\/p>\n<ul>\n<li>Novo implementiranje je aktiviralo CoreDNS-autoscaler, koji je prepisao conntrack tabelu<\/li>\n<li>Ovaj bug je zahvatio samo deo servisa<\/li>\n<\/ul>\n<p><\/p>\n<h3>Hronologija (EET)<\/h3>\n<p><\/p>\n<p>Vreme<br \/>\nAkcija<\/p>\n<p>22:13<br \/>\nCoreDNS-autoscaler je smanjio broj podova sa tri na dva<\/p>\n<p>22:18<br \/>\nDe\u017eurni in\u017eenjeri su po\u010deli dobijati pozive od sistema monitoringa<\/p>\n<p>22:21<br \/>\nDe\u017eurni in\u017eenjeri su po\u010deli istra\u017eivati uzrok gre\u0161aka<\/p>\n<p>22:39<br \/>\nDe\u017eurni in\u017eenjeri su po\u010deli vra\u0107ati jednu od poslednjih verzija servisa na prethodnu verziju<\/p>\n<p>22:40<br \/>\n5xx gre\u0161ke su prestale da se pojavljuju, situacija se stabilizovala<\/p>\n<p><\/p>\n<ul>\n<li><b>Vreme do otkri\u0107a:<\/b> 4 min<\/li>\n<li><b>Vreme do preduzimanja akcija:<\/b> 21 min<\/li>\n<li><b>Vreme do ispravke:<\/b> 1 min<\/li>\n<\/ul>\n<p><\/p>\n<h3>Dodatne informacije<\/h3>\n<p><\/p>\n<ul>\n<li>Logovi CoreDNS:\n<pre><code class=\"bash\">I0228 20:13:53.507780       1 event.go:221] Event(v1.ObjectReference{Kind:\"Deployment\", Namespace:\"kube-system\", Name:\"coredns\", UID:\"2493eb55-3dc0-11ea-b3a2-02bb48f8c230\", APIVersion:\"apps\/v1\", ResourceVersion:\"132690686\", FieldPath:\"\"}): type: 'Normal' reason: 'ScalingReplicaSet' Smanjena replika seta coredns-6cbb6646c9 na 2<\/code><\/pre>\n<\/li>\n<li>Linkovi na Kibanu (izrezano), Grafana (izrezano)<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.projectcalico.org\/when-linux-conntrack-is-no-longer-your-friend\/\">Gde Linux conntrack vi\u0161e nije va\u0161 prijatelj<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/kubernetes.io\/blog\/2019\/03\/29\/kube-proxy-subtleties-debugging-an-intermittent-connection-reset\/\">Kube-proxy suptilnosti: debagovanje povremenog resetovanja veze<\/a><\/noindex><\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.weave.works\/blog\/racy-conntrack-and-dns-lookup-timeouts\">Neodre\u0111en conntrack i DNS lookup timeouti<\/a><\/noindex><\/li>\n<\/ul>\n<p>\nDa bi se minimizirala upotreba CPU-a, Linux kernel koristi ne\u0161to \u0161to se zove conntrack. Ukratko, to je alat koji sadr\u017ei spisak NAT zapisa, koji se \u010duvaju u posebnoj tabeli. Kada slede\u0107i paket stigne iz istog poda u isti pod kao pre, krajnja IP adresa ne\u0107e biti izra\u010dunata ponovo, ve\u0107 \u0107e biti uzeta iz conntrack tabele.<br \/>\n<img decoding=\"async\" alt=\"Problemi sa DNS-om u Kubernetesu. Javna postmortem analiza\" src=\"\/wp-content\/uploads\/2020\/05\/895db4057ee0912e16e017c6e3134162.png\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<i>Kako conntrack funkcioni\u0161e<\/i><\/p>\n<h2>Zaklju\u010dci<\/h2>\n<p>\nOvo je bio primjer jednog od na\u0161ih postmrtnih izvje\u0161taja s korisnim linkovima. Konkretno u ovom \u010dlanku dijelimo informacije koje mogu biti korisne drugim kompanijama. Zbog toga se ne bojimo praviti gre\u0161ke i zbog toga smo jedan od na\u0161ih postmrtnih izvje\u0161taja u\u010dinili javnim. Evo jo\u0161 nekoliko zanimljivih javnih postmrtnih izvje\u0161taja:<\/p>\n<ul>\n<li>GitLab: <noindex><a rel=\"nofollow\" href=\"https:\/\/about.gitlab.com\/blog\/2017\/02\/10\/postmortem-of-database-outage-of-january-31\/\">Postmortem prekida rada baze 31. januara<\/a><\/noindex><\/li>\n<li>Dropbox: <noindex><a rel=\"nofollow\" href=\"https:\/\/dropbox.tech\/infrastructure\/outage-post-mortem\">Postmortem prekida rada<\/a><\/noindex><\/li>\n<li>Spotify: <noindex><a rel=\"nofollow\" href=\"https:\/\/labs.spotify.com\/2017\/03\/31\/spotifys-lovehate-relationship-with-dns\/\">Spotifyjeva ljubav\/nena\u0161enje prema DNS-u<\/a><\/noindex><\/li>\n<li>Mnoge druge iz <noindex><a rel=\"nofollow\" href=\"https:\/\/gist.github.com\/cheapRoc\/d9e73fe05480330c1e36410cbdf0e867\">ovog gista<\/a><\/noindex> i repozitorija <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/hjacobs\/kubernetes-failure-stories\">Pri\u010de o neuspjesima u Kubernetesu<\/a><\/noindex><\/li>\n<li>Tako\u0111e <noindex><a rel=\"nofollow\" href=\"https:\/\/landing.google.com\/sre\/sre-book\/chapters\/postmortem\/\">primer<\/a><\/noindex> javnog postmortema sa SRE knjigom<\/li>\n<\/ul>\n<p>Izvor: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/500346\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u044d\u0442\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u043f\u0443\u0431\u043b\u0438\u0447\u043d\u043e\u0433\u043e \u043f\u043e\u0441\u0442\u043c\u043e\u0440\u0442\u0435\u043c\u0430 \u0438\u0437 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u043d\u043e\u0433\u043e \u0431\u043b\u043e\u0433\u0430 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Preply. \u0412 \u043d\u0435\u043c \u043e\u043f\u0438\u0441\u044b\u0432\u0430\u0435\u0442\u0441\u044f \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u0430 \u0441 conntrack \u0432 Kubernetes-\u043a\u043b\u0430\u0441\u0442\u0435\u0440\u0435, \u043a\u043e\u0442\u043e\u0440\u0430\u044f \u043f\u0440\u0438\u0432\u0435\u043b\u0430 \u043a \u0447\u0430\u0441\u0442\u0438\u0447\u043d\u043e\u043c\u0443 \u043f\u0440\u043e\u0441\u0442\u043e\u044e \u043d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0445 \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432 \u043f\u0440\u043e\u0434\u0430\u043a\u0448\u043d\u0430. \u0414\u0430\u043d\u043d\u0430\u044f \u0441\u0442\u0430\u0442\u044c\u044f \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c \u043f\u043e\u043b\u0435\u0437\u043d\u043e\u0439 \u0442\u0435\u043c, \u043a\u0442\u043e \u0445\u043e\u0447\u0435\u0442 \u0443\u0437\u043d\u0430\u0442\u044c \u043d\u0435\u043c\u043d\u043e\u0433\u043e \u0431\u043e\u043b\u044c\u0448\u0435 \u043e \u043f\u043e\u0441\u0442\u043c\u043e\u0440\u0442\u0435\u043c\u0430\u0445 \u0438\u043b\u0438 \u043f\u0440\u0435\u0434\u043e\u0442\u0432\u0440\u0430\u0442\u0438\u0442\u044c \u043d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043f\u043e\u0442\u0435\u043d\u0446\u0438\u0430\u043b\u044c\u043d\u044b\u0435 \u043f\u0440\u043e\u0431\u043b\u0435\u043c\u044b \u0441 DNS \u0432 \u0431\u0443\u0434\u0443\u0449\u0435\u043c. \u042d\u0442\u043e \u043d\u0435 DNS \u041d\u0435 \u043c\u043e\u0436\u0435\u0442 \u0431\u044b\u0442\u044c [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":80533,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-80532","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.5.1 - aioseo.com -->\n\t<meta name=\"description\" content=\"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u044d\u0442\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u043f\u0443\u0431\u043b\u0438\u0447\u043d\u043e\u0433\u043e \u043f\u043e\u0441\u0442\u043c\u043e\u0440\u0442\u0435\u043c\u0430 \u0438\u0437 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u043d\u043e\u0433\u043e \u0431\u043b\u043e\u0433\u0430 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Preply.\" \/>\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/bs\/blog\/administrirovanie\/problemy-s-dns-v-kubernetes-publichnyj-postmortem\" \/>\n\t\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.5.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"bs_BA\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u041f\u0440\u043e\u0431\u043b\u0435\u043c\u044b \u0441 DNS \u0432 Kubernetes. \u041f\u0443\u0431\u043b\u0438\u0447\u043d\u044b\u0439 \u043f\u043e\u0441\u0442\u043c\u043e\u0440\u0442\u0435\u043c | ProHoster\" \/>\n\t\t<meta property=\"og:description\" content=\"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u044d\u0442\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u043f\u0443\u0431\u043b\u0438\u0447\u043d\u043e\u0433\u043e \u043f\u043e\u0441\u0442\u043c\u043e\u0440\u0442\u0435\u043c\u0430 \u0438\u0437 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u043d\u043e\u0433\u043e \u0431\u043b\u043e\u0433\u0430 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Preply.\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/bs\/blog\/administrirovanie\/problemy-s-dns-v-kubernetes-publichnyj-postmortem\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-05-06T23:42:17+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-05-06T23:42:17+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Problemi sa DNS-om u Kubernetesu. Javni postmortem | ProHoster","description":"Napomena: ovo je prevod javnog postmortema iz in\u017eenjerskog bloga kompanije Preply.","canonical_url":"https:\/\/prohoster.info\/bs\/blog\/administrirovanie\/problemy-s-dns-v-kubernetes-publichnyj-postmortem","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"bs_BA","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u041f\u0440\u043e\u0431\u043b\u0435\u043c\u044b \u0441 DNS \u0432 Kubernetes. \u041f\u0443\u0431\u043b\u0438\u0447\u043d\u044b\u0439 \u043f\u043e\u0441\u0442\u043c\u043e\u0440\u0442\u0435\u043c | ProHoster","og:description":"\u041f\u0440\u0438\u043c. \u043f\u0435\u0440\u0435\u0432.: \u044d\u0442\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u043f\u0443\u0431\u043b\u0438\u0447\u043d\u043e\u0433\u043e \u043f\u043e\u0441\u0442\u043c\u043e\u0440\u0442\u0435\u043c\u0430 \u0438\u0437 \u0438\u043d\u0436\u0435\u043d\u0435\u0440\u043d\u043e\u0433\u043e \u0431\u043b\u043e\u0433\u0430 \u043a\u043e\u043c\u043f\u0430\u043d\u0438\u0438 Preply.","og:url":"https:\/\/prohoster.info\/bs\/blog\/administrirovanie\/problemy-s-dns-v-kubernetes-publichnyj-postmortem","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-05-06T23:42:17+00:00","article:modified_time":"2020-05-06T23:42:17+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"80532","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 16:16:27","updated":"2022-09-28 05:25:18","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/posts\/80532","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/comments?post=80532"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/posts\/80532\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/media\/80533"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/media?parent=80532"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/categories?post=80532"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/bs\/wp-json\/wp\/v2\/tags?post=80532"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}