{"id":96515,"date":"2020-10-12T19:43:02","date_gmt":"2020-10-12T17:43:02","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/odyssey-roadmap-chto-eshhyo-my-hotim-ot-pulera-soedinenij-andrej-borodin-2019"},"modified":"2020-10-12T19:43:02","modified_gmt":"2020-10-12T17:43:02","slug":"odyssey-roadmap-chto-eshhyo-my-hotim-ot-pulera-soedinenij-andrej-borodin-2019","status":"publish","type":"post","link":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/odyssey-roadmap-chto-eshhyo-my-hotim-ot-pulera-soedinenij-andrej-borodin-2019","title":{"rendered":"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/6c445191705a4c96939cd094bc457a53.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen raport, Andrei Borodin va explica cum au luat \u00een considerare experien\u021ba scal\u0103rii PgBouncer \u00een proiectarea connection pooler-ului <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex\/odyssey\">Odiseea<\/a><\/noindex>, cum l-au implementat \u00een produc\u021bie. De asemenea, vom discuta despre ce func\u021bii ale connection pooler-ului ne-ar pl\u0103cea s\u0103 vedem \u00een versiunile noi: este important pentru noi s\u0103 nu ne satisfacem doar nevoile, ci s\u0103 dezvolt\u0103m comunitatea utilizatorilor <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex\/odyssey\">Odiseea<\/a><\/noindex>.<\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<p>Video:<\/p>\n<p>\n<center><div class=\"youtube-placeholder\" data-id=\"OUx40ABHFZY\" onclick=\"loadVideo(this)\">\r\n        <img decoding=\"async\" src=\"https:\/\/img.youtube.com\/vi\/OUx40ABHFZY\/hqdefault.jpg\" alt=\"Reda\u021bi video\" loading=\"lazy\" width=\"480\" height=\"360\" style=\"width:100%;height:auto;\">\r\n        <div class=\"play-button\"><\/div>\r\n    <\/div><\/center><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/dd2ea50b72e3d9c4e795aecbf9afdd25.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Salut tuturor! M\u0103 numesc Andrei. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/0a2f1880005bec1f61b72fb23b555adc.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>La Yandex, m\u0103 ocup cu dezvoltarea bazelor de date open source. \u0218i ast\u0103zi avem un subiect despre connection pooler. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/c18e7c937423d2514faa12e820e78e70.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dac\u0103 \u0219ti\u021bi cum se nume\u0219te connection pooler \u00een rom\u00e2n\u0103, v\u0103 rog s\u0103-mi spune\u021bi. Vreau foarte mult s\u0103 g\u0103sesc un termen tehnic bun care s\u0103 se consolideze \u00een literatura de specialitate. <\/p>\n<p><\/p>\n<p>Tema este destul de complex\u0103, deoarece \u00een multe baze de date connection pooler-ul este \u00eencorporat \u0219i nu trebuie s\u0103-l cuno\u0219ti. Sigur, exist\u0103 unele set\u0103ri peste tot, dar \u00een Postgres nu func\u021bioneaz\u0103 a\u0219a. \u0218i \u00een paralel (la HighLoad++ 2019) are loc raportul lui Nikolai Samohvalov despre configurarea interog\u0103rilor \u00een Postgres. \u0218i din c\u00e2te \u00een\u021beleg, aici au venit oamenii care au configurat interog\u0103rile perfect, iar ace\u0219tia se confrunt\u0103 cu probleme sistemice mai rare legate de re\u021bea, utilizarea resurselor. \u0218i pe alocuri a fost destul de complicat pentru c\u0103 problemele nu sunt evidente.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/89c660dd530575e2938e6bf312b83e95.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>La Yandex exist\u0103 Postgres. \u00cen Yandex.Cloud tr\u0103iesc multe servicii ale Yandex. \u0218i avem c\u00e2teva petabytes de date, care genereaz\u0103 nu mai pu\u021bin de un milion de interog\u0103ri pe secund\u0103 \u00een Postgres. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/288bfa37b9dcd60acd22ba48f0ef8a39.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i oferim un cluster destul de standard pentru toate serviciile \u2013 aceasta este nodul principal, cu dou\u0103 replici obi\u0219nuite (una sincron\u0103 \u0219i una asincron\u0103), backup, scalarea interog\u0103rilor de citire pe replic\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/c249ca55bb387e25a5560324024cc523.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Fiecare nod al cluster-ului este un Postgres, pe care, pe l\u00e2ng\u0103 Postgres \u0219i sistemele de monitorizare, este instalat \u0219i connection pooler. Connection pooler-ul este folosit pentru fencing \u0219i pentru scopul s\u0103u principal. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/0e29be55cc048007db21c80bbb08a9d4.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Care este scopul principal al connection pooler-ului? <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/e2fbbd440c2781573cb35deeef358f20.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen Postgres, este adoptat un model de proces \u00een lucru cu baza de date. Aceasta \u00eenseamn\u0103 c\u0103 o conexiune \u2013 este un singur proces, un singur backend Postgres. \u0218i \u00een acest backend exist\u0103 multe tipuri diferite de cache-uri, pe care este costisitor s\u0103 le faci diferite pentru diverse conexiuni. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/587498063b1af8fac4a845961c654bc2.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen plus, \u00een codul Postgres exist\u0103 un array numit procArray. Acesta con\u021bine date esen\u021biale despre conexiunile de re\u021bea. Aproape toate algoritmii de procesare a procArray-ului au complexitate liniar\u0103, parcurg\u00e2nd \u00eentregul array al conexiunilor de re\u021bea. Este un ciclu destul de rapid, dar cu un num\u0103r mare de conexiuni de re\u021bea \u00een intrare, totul devine pu\u021bin mai costisitor. \u0218i, c\u00e2nd totul devine pu\u021bin mai scump, se poate ajunge la o pre\u021b foarte mare pentru un num\u0103r mare de conexiuni. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/b534a38d0bfae7a65fc4258155a50280.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Exist\u0103 3 abord\u0103ri posibile:<\/p>\n<p><\/p>\n<ul>\n<li>Pe partea aplica\u021biei. <\/li>\n<li>Pe partea bazei de date. <\/li>\n<li>\u0218i \u00eentre ele, adic\u0103 toate combina\u021biile posibile. <\/li>\n<\/ul>\n<p><\/p>\n<p>Din p\u0103cate, pooler-ul \u00eencorporat este \u00een prezent \u00een dezvoltare. Prietenii de la PostgreSQL Professional se ocup\u0103 \u00een principal de acest lucru. C\u00e2nd va ap\u0103rea, este greu de prezis. \u0218i, de fapt, alegerea arhitectului nostru are la dispozi\u021bie dou\u0103 solu\u021bii. Este pool-ul pe partea aplica\u021biei \u0219i pool-ul proxy. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/702527a09e9fc86dd39b7fd69dcf1fc9.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Pool-ul pe partea aplica\u021biei este cea mai simpl\u0103 op\u021biune. Aproape toate driverele client ofer\u0103 o modalitate de a reprezenta milioanele de conexiuni din cod drept c\u00e2teva zeci de conexiuni \u00een baza de date. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/7b66b9e20d22f472ca5ac39f6b9e0ba4.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Problema apare atunci c\u00e2nd, la un moment dat, vrei s\u0103 scalaze backend-ul, vrei s\u0103-l desf\u0103\u0219ori pe mai multe ma\u0219ini virtuale. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/f216fd7a8c01c2b07891e2242c48b06a.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Apoi \u00ee\u021bi dai seama c\u0103 mai ai c\u00e2teva zone de disponibilitate, c\u00e2teva centre de date. Iar abordarea cu pool-ul pe partea clientului duce la numere mari. Mari - adic\u0103 aproximativ 10.000 de conexiuni. Acesta este cap\u0103tul la care poate func\u021biona normal. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/e5bc5f284fd8ff568bbe16c3895b7ac6.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen ceea ce prive\u0219te proxy poolers, exist\u0103 dou\u0103 poolers care fac multe lucruri. Acestea nu sunt doar poolers. Sunt poolers + o func\u021bionalitate excelent\u0103. Acestea sunt <noindex><a rel=\"nofollow\" href=\"https:\/\/www.pgpool.net\/\">Pgpool<\/a><\/noindex> \u0219i <noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/CrunchyData\/crunchy-proxy\">Crunchy-Proxy<\/a><\/noindex>. <\/p>\n<p><\/p>\n<p>Dar, din p\u0103cate, aceast\u0103 func\u021bionalitate suplimentar\u0103 nu este necesar\u0103 pentru to\u021bi. \u0218i duce la faptul c\u0103 poolers suport\u0103 doar pooling de sesiune, deci un client de intrare, un client de ie\u0219ire \u00een baza de date. <\/p>\n<p><\/p>\n<p>Pentru nevoile noastre, aceasta nu se potrive\u0219te foarte bine, a\u0219a c\u0103 folosim PgBouncer, care implementeaz\u0103 pooling de tranzac\u021bii, adic\u0103 conexiunile serverului se potrivesc conexiunilor client doar pe durata tranzac\u021biei. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/e5fedbe14656d2ac2cde8d1878a950e2.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i sub sarcina noastr\u0103 - este adev\u0103rat. <strong>Dar exist\u0103 c\u00e2teva probleme<\/strong>.<img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/61fd36cadcd9948d624dafa711777307.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Problemele apar atunci c\u00e2nd vrei s\u0103 diagnostichezi o sesiune, pentru c\u0103 toate conexiunile de intrare sunt locale. Toate provin din loopback \u0219i cumva devine dificil s\u0103 urm\u0103re\u0219ti sesiunea. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/36108c4956c410a1b1f286a88bc393d6.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Desigur, pute\u021bi utiliza application_name_add_host. Aceasta este metoda de pe partea Bouncer pentru a ad\u0103uga o adres\u0103 IP \u00een application_name. Dar application_name este setat printr-o conexiune suplimentar\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/b4cec483aa776f48db7ad3966c3b78e4.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Pe acest grafic, linia galben\u0103 reprezint\u0103 cererile reale, iar linia albastr\u0103 reprezint\u0103 cererile care ajung \u00een baza de date. Iar aceast\u0103 diferen\u021b\u0103 este exact setarea application_name, care este necesar\u0103 doar pentru urm\u0103rire, dar care nu este deloc gratuit\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/c02ea73a6f1a2a4cd6af6c621f321ab4.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen plus, \u00een Bouncer nu se poate limita un pool, adic\u0103 num\u0103rul de conexiuni la baza de date pentru un anumit utilizator, pentru o anumit\u0103 baz\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/3431a463e04106066e2799aead66f571.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>La ce duce asta? Ave\u021bi un serviciu suprasolicitat, scris \u00een C++, \u0219i undeva \u00een apropiere un serviciu mic pe Node, care nu face nimic grav cu baza, dar driverul s\u0103u se duce de cap. Deschide 20.000 de conexiuni, iar tot restul a\u0219teapt\u0103. Chiar ave\u021bi un cod normal. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/d7fa321714dc017b017945dffcf79bc6.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Am scris, desigur, un mic patch pentru Bouncer, care a ad\u0103ugat aceast\u0103 setare, adic\u0103 limitarea clien\u021bilor la pool. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/d58884fc5ac0f90ff8442f126972157a.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ar fi putut fi f\u0103cut pe partea Postgres, adic\u0103 rolurile \u00een baza de date limitate \u00een func\u021bie de num\u0103rul de conexiuni.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/8d6ff70db72964ed4eae5eaa47309931.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dar atunci pierde\u021bi capacitatea de a \u00een\u021belege de ce nu ave\u021bi conexiuni cu serverul. PgBouncer nu transmite eroarea de conexiune, el returneaz\u0103 \u00eentotdeauna aceea\u0219i informa\u021bie. \u0218i nu pute\u021bi \u00een\u021belege: poate c\u0103 v-a\u021bi schimbat parola, poate c\u0103 baza a c\u0103zut, poate c\u0103 ceva nu este \u00een regul\u0103. Dar nu exist\u0103 nicio diagnoz\u0103. Dac\u0103 nu se poate stabili sesiunea, nu ve\u021bi \u0219ti de ce nu se poate face. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/cb3048344a94f2953bf403485d28e69f.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>La un moment dat, v\u0103 uita\u021bi la graficele aplica\u021biei \u0219i vede\u021bi c\u0103 aplica\u021bia nu func\u021bioneaz\u0103.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/e67c6680bb9a7f41242af9f9e4920916.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Vede\u021bi \u00een top \u0219i observa\u021bi c\u0103 Bouncer este uniprodus. Acesta este un moment de cotitur\u0103 \u00een via\u021ba serviciului. \u00cen\u021belege\u021bi c\u0103 v-a\u021bi preg\u0103tit pentru scalarea bazei de date \u00een decurs de un an \u0219i jum\u0103tate, dar trebuie s\u0103 scala\u021bi pooler-ul. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/4c140c4ba6db19cb9939c1148638fb73.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Am ajuns la concluzia c\u0103 avem nevoie de mai mul\u021bi PgBouncer. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/d369f572a66c16a8917e13393324a644.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/lwn.net\/Articles\/542629\/\">https:\/\/lwn.net\/Articles\/542629\/<\/a><\/noindex><\/p>\n<p><\/p>\n<p>Pu\u021bin am patch-uit Bouncer. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/97755489a25ebfe38efd56ff897bd8fa.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i am f\u0103cut astfel \u00eenc\u00e2t s\u0103 poat\u0103 fi ridicate mai multe Bouncers cu reutilizarea portului TCP. Iar sistemul de operare redistribuie automat conexiunile TCP de intrare \u00eentre ele \u00eentr-un mod round-robin.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/ac30d3f067de2a8cd66ad2325c3a3b58.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Aceasta este transparent pentru clien\u021bi, adic\u0103 totul arat\u0103 ca \u0219i cum a\u021bi avea un singur Bouncer, dar ave\u021bi o fragmentare a conexiunilor idle \u00eentre Bouncers-urile \u00een func\u021biune. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/53825f075db00b026c12013a610be456.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i, la un moment dat, s-ar putea s\u0103 observa\u021bi c\u0103 ace\u0219ti 3 Bouncers \u00ee\u0219i consum\u0103 fiecare nucleul 100%. Ave\u021bi nevoie de o mul\u021bime de Bouncers. De ce? <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/66c17883a043e157cd7ceab2c2a4e326.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Pentru c\u0103 ave\u021bi TLS. Ave\u021bi o conexiune criptat\u0103. \u0218i dac\u0103 testa\u021bi Postgres cu TLS \u0219i f\u0103r\u0103 TLS, ve\u021bi observa c\u0103 num\u0103rul de conexiuni stabilite scade cu aproape dou\u0103 ordine de m\u0103rime odat\u0103 cu activarea cript\u0103rii, deoarece handshake-ul TLS consum\u0103 resursele CPU-ului. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/6101121ebe136af16e5eb1a484c65d66.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i \u00een v\u00e2rf pute\u021bi observa o mul\u021bime de func\u021bii criptografice care sunt executate \u00een timpul valului de conexiuni \u00een incoming. Deoarece primary-ul nostru poate comuta \u00eentre zone de disponibilitate, valul de conexiuni \u00een incoming este o situa\u021bie destul de tipic\u0103. Adic\u0103, dintr-un motiv oarecare, vechiul primary a fost inaccesibil, \u00eentreaga \u00eenc\u0103rc\u0103tur\u0103 a fost trimis\u0103 c\u0103tre un alt centru de date. Toate acestea vor veni simultan s\u0103 se salute cu TLS. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/c238ac2f511568590dc275e01c192c00.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i un num\u0103r mare de handshake-uri TLS poate s\u0103 nu se salute deja cu Bouncer, ci s\u0103-l str\u00e2ng\u0103 de g\u00e2t. Din cauza timeout-ului, valul de conexiuni \u00een incoming poate deveni persistent. Dac\u0103 ave\u021bi retry \u00een baz\u0103 f\u0103r\u0103 exponential backoff, ele nu vor veni din nou \u0219i din nou ca un val coerent. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/f42d0b210174d4d172254603ae6b834a.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Iat\u0103 un exemplu de 16 PgBouncer, care \u00eencarc\u0103 16 nuclee la 100%.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/8d048d3774c09e5b6a3fb3c7bff62084.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Am ajuns la PgBouncer \u00een cascad\u0103. Aceasta este cea mai bun\u0103 configura\u021bie pe care o putem ob\u021bine pe sarcina noastr\u0103 cu Bouncer. Bouncers externe ne servesc pentru handshake-ul TCP, iar Bouncers interne servesc pentru pooling-ul real, astfel \u00eenc\u00e2t s\u0103 nu fragment\u0103m prea mult conexiunile externe.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/3a4ee4ea46d5fe8479a5a7222ee134da.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen aceast\u0103 configura\u021bie, un restart lin este posibil. Pute\u021bi reporni toate aceste 18 Bouncers unul c\u00e2te unul. Dar men\u021binerea unei astfel de configura\u021bii este destul de complicat\u0103. Administratorii de sistem, DevOps \u0219i persoanele care sunt cu adev\u0103rat responsabile pentru acest server nu vor fi foarte \u00eenc\u00e2nta\u021bi de aceast\u0103 schem\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/5291fe67dbe02036f3ff8e98bdd84d1b.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>S-ar p\u0103rea c\u0103 putem promova toate dezvolt\u0103rile noastre \u00een open source, dar Bouncer nu este bine suportat. De exemplu, posibilitatea de a rula mai multe PgBouncers pe acela\u0219i port a fost angajat\u0103 acum o lun\u0103. Iar pull request-ul pentru aceast\u0103 func\u021bie a fost f\u0103cut acum c\u00e2\u021biva ani. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/2059d7f49649d136249103db4aa3ec19.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/www.postgresql.org\/docs\/current\/libpq-cancel.html\">https:\/\/www.postgresql.org\/docs\/current\/libpq-cancel.html<\/a><\/noindex><\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/pgbouncer\/pgbouncer\/pull\/79\">https:\/\/github.com\/pgbouncer\/pgbouncer\/pull\/79<\/a><\/noindex><\/p>\n<p><\/p>\n<p>Sau un alt exemplu. \u00cen Postgres, pute\u021bi anula o interogare \u00een curs de execu\u021bie trimi\u021b\u00e2nd un secret printr-o alt\u0103 conexiune f\u0103r\u0103 autentificare suplimentar\u0103. Dar unii clien\u021bi trimit pur \u0219i simplu un TCP-reset, adic\u0103 rup conexiunea de re\u021bea. Ce va face Bouncer \u00een aceast\u0103 situa\u021bie? Nu va face nimic. Va continua s\u0103 execute interogarea. Dac\u0103 a\u021bi primit un num\u0103r imens de conexiuni, care prin cereri mici au suprasolicitat baza, atunci doar ruperea conexiunii cu Bouncer nu va fi suficient\u0103; va trebui, de asemenea, s\u0103 \u00eenchide\u021bi acele interog\u0103ri care sunt active \u00een baz\u0103. <\/p>\n<p><\/p>\n<p>Aceasta a fost corectat\u0103, dar aceast\u0103 problem\u0103 \u00eenc\u0103 nu a fost fuzionat\u0103 \u00een upstream-ul Bouncer-ului. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/a9695b6d9e640a5543dc73429eb0050b.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i astfel am ajuns la concluzia c\u0103 avem nevoie de propriul nostru connection pooler, care va evolua, va fi corectat, \u00een care putem rezolva rapid problemele \u0219i care, desigur, trebuie s\u0103 fie multithread. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/e755cd9b06a004f0eced1dc12fd3a6fb.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Am stabilit multithreading-ul ca o sarcin\u0103 principal\u0103. Trebuie s\u0103 suport\u0103m bine valul de conexiuni TLS \u00een curs de incoming. <\/p>\n<p><\/p>\n<p>Pentru aceasta a fost necesar s\u0103 dezvolt\u0103m o bibliotec\u0103 separat\u0103, numit\u0103 Machinarium, care este destinat\u0103 descrierii st\u0103rilor ma\u0219inii conexiunii de re\u021bea ca un cod secven\u021bial. Dac\u0103 te ui\u021bi \u00een codul surs\u0103 libpq, vei vedea apeluri destul de complexe care \u00ee\u021bi pot returna un rezultat \u0219i \u00ee\u021bi pot spune: \u201eSun\u0103-m\u0103 mai t\u00e2rziu. Acum am IO, dar, c\u00e2nd IO va fi finalizat, voi avea sarcini pentru procesor\u201d. \u0218i aceasta este o schem\u0103 pe mai multe niveluri. Interac\u021biunea de re\u021bea este de obicei descris\u0103 ca o ma\u0219in\u0103 a st\u0103rilor. O multitudine de reguli de tip \u201eDac\u0103 am primit anterior un antet de pachet de dimensiune N, atunci acum a\u0219tept N octe\u021bi\u201d, \u201eDac\u0103 am trimis un pachet SYNC, atunci acum a\u0219tept un pachet cu metadatele rezultatului\u201d. Se ob\u021bine un cod destul de complicat \u0219i contraintuitiv, ca \u0219i cum un labirint ar fi fost transformat \u00eentr-o desf\u0103\u0219urare liniar\u0103. Am f\u0103cut astfel \u00eenc\u00e2t, \u00een loc de a avea o ma\u0219in\u0103 a st\u0103rilor, programatorul s\u0103 descrie calea principal\u0103 de interac\u021biune sub form\u0103 de cod imperativ obi\u0219nuit. Pur \u0219i simplu, \u00een acest cod imperativ trebuie s\u0103 inser\u0103m locuri unde secven\u021ba de execu\u021bie trebuie \u00eentrerupt\u0103 a\u0219tept\u00e2nd date de la re\u021bea, transmit\u00e2nd contextul de execu\u021bie c\u0103tre o alt\u0103 corutin\u0103 (thread verde). Aceast\u0103 abordare este similar\u0103 cu aceea \u00een care cel mai a\u0219teptat traseu din labirint este notat consecutiv, iar apoi ad\u0103ug\u0103m ramifica\u021bii. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/a6bfb4b11961c7db3a7c7184674e3bc2.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen concluzie, avem un singur fir care face TCP accept \u0219i transfer\u0103, prin round-robin, mai multe conexiuni TPC lucr\u0103torilor. <\/p>\n<p><\/p>\n<p>\u00cen plus, fiecare conexiune a clientului lucreaz\u0103 \u00eentotdeauna pe un singur procesor. Acest lucru permite o utilizare prietenoas\u0103 a cache-ului. <\/p>\n<p><\/p>\n<p>De asemenea, am \u00eembun\u0103t\u0103\u021bit pu\u021bin colectarea pachetelor mici \u00eentr-un singur pachet pentru a desc\u0103rca stiva TCP a sistemului.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/7d740c23baecdd2f993c83cd54a0a8e1.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen plus, am \u00eembun\u0103t\u0103\u021bit pooling-ul tranzac\u021biilor, astfel \u00eenc\u00e2t Odyssey, \u00een configura\u021bia sa, poate trimite CANCEL \u0219i ROLLBACK \u00een cazul pierderii conexiunii de re\u021bea, adic\u0103 dac\u0103 nimeni nu a\u0219teapt\u0103 cererea, Odyssey va spune bazei s\u0103 nu \u00eencerce s\u0103 execute o cerere care ar putea consuma resurse pre\u021bioase.<\/p>\n<p><\/p>\n<p>De asemenea, \u00eencerc\u0103m s\u0103 men\u021binem conexiunile pentru acela\u0219i client. Aceasta permite evitarea reinstal\u0103rii application_name_add_host. Dac\u0103 este posibil, nu avem o reinstalare suplimentar\u0103 a parametrilor necesari pentru diagnosticare. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/afb2fb7fb16d65c9ea73a8978d1eb83c.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ac\u021bion\u0103m \u00een interesul Yandex.Cloud. Dac\u0103 utiliza\u021bi PostgreSQL gestionat \u0219i ave\u021bi un connection pooler configurat, pute\u021bi crea replicare logic\u0103 extern\u0103, adic\u0103 pute\u021bi p\u0103r\u0103si sistemul nostru dac\u0103 dori\u021bi, folosind replicarea logic\u0103. Bouncer nu va transmite fluxul de replicare logic\u0103 extern. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/c53defc92ebd8f3bec57ce64d97b9f72.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Acesta este un exemplu de configurare a replic\u0103rii logice. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/a37d9d1abf288d17fa646b642e8b62ab.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>De asemenea, avem suport pentru replicarea fizic\u0103 extern. \u00cen Cloud, acest lucru nu este posibil, deoarece clusterele ar oferi prea multe informa\u021bii despre ele \u00eensele. Dar \u00een instala\u021biile voastre, dac\u0103 ave\u021bi nevoie de replicare fizic\u0103 prin connection pooler \u00een Odyssey, acest lucru este posibil. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/572934ce5b08115e57ab4b55d2153c6d.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Odyssey are o monitorizare complet compatibil\u0103 cu PgBouncer. Avem aceea\u0219i consol\u0103, care execut\u0103 aproape toate acelea\u0219i comenzi. Dac\u0103 lipse\u0219te ceva, trimite\u021bi un pull request sau, m\u0103car, o problem\u0103 pe GitHub, \u0219i vom ad\u0103uga comenzile necesare. Dar func\u021bionalitatea de baz\u0103 a consolei PgBouncer este deja disponibil\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/451dd798fb3523a7a174ebfe00a14bc5.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i, bine\u00een\u021beles, avem redirec\u021bionarea erorilor. Vom returna eroarea raportat\u0103 de baz\u0103. Ve\u021bi primi informa\u021bii despre motivul pentru care nu v\u0103 conecta\u021bi la baz\u0103, nu doar c\u0103 nu reu\u0219i\u021bi. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/14b4cb0efa33a2658dfc528349a5382f.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Aceast\u0103 op\u021biune poate fi dezactivat\u0103 \u00een cazul \u00een care dori\u021bi o compatibilitate 100% cu PgBouncer. Ne putem comporta la fel ca Bouncer, doar ca precau\u021bie. <\/p>\n<p><\/p>\n<p><strong>Dezvoltare<\/strong><\/p>\n<p><\/p>\n<p>C\u00e2teva cuvinte despre codul surs\u0103 Odyssey. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/6055d5ec9676c331a83592916a6a292f.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex\/odyssey\/pull\/66\">https:\/\/github.com\/yandex\/odyssey\/pull\/66<\/a><\/noindex><\/p>\n<p><\/p>\n<p>De exemplu, exist\u0103 comenzi de tipul \u201ePauz\u0103 \/Reluare\u201d. Acestea sunt folosite de obicei pentru a actualiza baza de date. Dac\u0103 trebuie s\u0103 actualiza\u021bi Postgres, \u00eel pute\u021bi pune pe pauz\u0103 \u00een connection pooler, s\u0103 face\u021bi pg_upgrade, dup\u0103 care s\u0103 relua\u021bi. Iar din partea clientului, va p\u0103rea c\u0103 baza a \u00eencetinit pur \u0219i simplu. Aceast\u0103 func\u021bionalitate ne-a fost adus\u0103 de oamenii din comunitate. Deocamdat\u0103 nu a fost \u00eembinat\u0103, dar \u00een cur\u00e2nd totul va fi. (Deja \u00eembinat\u0103)<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/d2f7c57e0ae3e466ec2a701bbdb8eb3f.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex\/odyssey\/pull\/73\">https:\/\/github.com\/yandex\/odyssey\/pull\/73<\/a><\/noindex> \u2014 deja \u00eembinat\u0103<\/p>\n<p><\/p>\n<p>\u00cen plus, una dintre noile func\u021bii din PgBouncer este suportul pentru autentificarea SCRAM, pe care ne-a adus-o de asemenea o persoan\u0103 care nu lucreaz\u0103 la Yandex.Cloud. Ambele \u2013 sunt func\u021bionalit\u0103\u021bi complexe \u0219i importante. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/f39346a543c88d17728233d05c2dff19.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Prin urmare, vreau s\u0103 v\u0103 povestesc din ce este realizat Odyssey, poate c\u0103 \u0219i voi vre\u021bi s\u0103 scrie\u021bi un pic de cod. <\/p>\n<p><\/p>\n<p>Ave\u021bi baza de cod Odyssey, care se bazeaz\u0103 pe dou\u0103 biblioteci principale. Biblioteca Kiwi \u2013 este o implementare a protocolului de mesaje Postgres. Adic\u0103, proto 3 nativ la Postgres \u2013 sunt mesajele standard cu care frontend-urile \u0219i backend-urile pot schimba informa\u021bii. Acestea sunt implementate \u00een biblioteca Kiwi.<\/p>\n<p><\/p>\n<p>Biblioteca Machinarium \u2013 este o bibliotec\u0103 pentru implementarea firelor de execu\u021bie. Un mic fragment din acest Machinarium este scris \u00een assembler. Dar, nu v\u0103 speria\u021bi, sunt doar 15 linii. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/cd18f250af50eabd53536b2c0d7f12b5.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Arhitectura Odyssey. Exist\u0103 o ma\u0219in\u0103 principal\u0103 \u00een care sunt rulate coroutines. \u00cen aceast\u0103 ma\u0219in\u0103 se realizeaz\u0103 acceptarea conexiunilor TCP \u00een a\u0219teptare \u0219i distribuirea acestora \u00eentre workers. <\/p>\n<p><\/p>\n<p>\u00cen interiorul unui worker poate func\u021biona un handler pentru mai mul\u021bi clien\u021bi. De asemenea, \u00een firul principal ruleaz\u0103 consola \u0219i gestionarea sarcinilor crone pentru eliminarea conexiunilor care nu mai sunt necesare \u00een pool.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/7072fb20f406f2ee5788542860ccdf50.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Pentru testarea Odyssey se utilizeaz\u0103 setul standard de teste Postgres. Pur \u0219i simplu rul\u0103m install-check prin Bouncer \u0219i prin Odyssey, ob\u021binem un div nul. Exist\u0103 c\u00e2teva teste legate de formatarea datelor care nu trec deloc la fel \u00een Bouncer \u0219i \u00een Odyssey.<\/p>\n<p><\/p>\n<p>\u00cen plus, exist\u0103 numeroase drivere care au propriile teste de verificare. Iar aceste teste le folosim pentru testarea Odyssey.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/fb01cc9a0e9724ddcd3cc3d088afb182.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>De asemenea, din cauza configura\u021biei noastre \u00een cascad\u0103, trebuie s\u0103 test\u0103m diferite combina\u021bii: Postgres + Odyssey, PgBouncer + Odyssey, Odyssey + Odyssey pentru a ne asigura c\u0103, dac\u0103 Odyssey se afl\u0103 \u00eentr-o anumit\u0103 parte a cascadei, func\u021bioneaz\u0103 \u00een continuare a\u0219a cum ne a\u0219tept\u0103m. <\/p>\n<p><\/p>\n<p><strong>Greble<\/strong><\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/25855b691180bca083025b9e8b611c09.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Folosim Odyssey \u00een produc\u021bie. \u0218i ar fi fost nedrept s\u0103 spun c\u0103 totul func\u021bioneaz\u0103 simplu. Nu, adic\u0103 da, dar nu \u00eentotdeauna. De exemplu, \u00een produc\u021bie totul a func\u021bionat perfect, apoi au venit prietenii no\u0219tri de la PostgreSQL Professional \u0219i ne-au spus c\u0103 avem o scurgere de memorie. Au avut dreptate, am rezolvat-o. Dar a fost pur \u0219i simplu. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/e638d88397b32b1ca0b74c9c1b905f81.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Apoi am descoperit c\u0103 \u00een connection pooler exist\u0103 conexiuni TLS de intrare \u0219i conexiuni TLS de ie\u0219ire. \u0218i pentru conexiuni sunt necesare certificate client \u0219i certificate server.<\/p>\n<p><\/p>\n<p>Certificatele serverului Bouncer \u0219i Odyssey \u00ee\u0219i citesc pcache-ul, dar certificatele client nu trebuie citite din pcache, deoarece Odyssey-ul nostru scalabil se love\u0219te \u00een cele din urm\u0103 de performan\u021ba de citire a acestui certificat. A fost o surpriz\u0103 pentru noi, deoarece s-a \u00eempotmolit nu de la bun \u00eenceput. La \u00eenceput s-a scalat liniar, iar dup\u0103 20.000 de conexiuni simultane de intrare, aceast\u0103 problem\u0103 s-a manifestat.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/6cff094cd59a990e448b82fa0b1572b7.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Metoda de autentificare pluggable este capacitatea de a ne autentifica prin mijloacele \u00eencorporate \u00een Linux. \u00cen PgBouncer este implementat\u0103 astfel \u00eenc\u00e2t exist\u0103 un fir de execu\u021bie separat pentru a a\u0219tepta un r\u0103spuns de la PAM \u0219i un fir principal PgBouncer care se ocup\u0103 de conexiunea curent\u0103 \u0219i poate cere acestora s\u0103 stea \u00een firul PAM. <\/p>\n<p><\/p>\n<p>Nu am implementat acest lucru dintr-un simplu motiv. Avem multe fire. De ce ne-ar mai trebui asta?<\/p>\n<p><\/p>\n<p>\u00cen cele din urm\u0103, acest lucru poate provoca probleme, deoarece, dac\u0103 ave\u021bi autentificare PAM \u0219i ne-PAM, o mare onduire de autentificare PAM poate \u00eent\u00e2rzia semnificativ autentificarea ne-PAM. Aceasta este una dintre acele probleme pe care nu le-am rezolvat. Dar dac\u0103 dori\u021bi s\u0103 rezolva\u021bi asta, pute\u021bi ocupa\u021bi-v\u0103 de aceast\u0103 problem\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/c7bd9e9980ff7298b30a59c130f03dbb.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>O alt\u0103 capcan\u0103 a fost c\u0103 avem un fir care accept\u0103 toate conexiunile de intrare. \u0218i apoi le transmite la grupul de lucr\u0103tori, unde va avea loc handshake-ul TLS.<\/p>\n<p><\/p>\n<p>\u00cen cele din urm\u0103, dac\u0103 ave\u021bi o und\u0103 coerent\u0103 de 20.000 de conexiuni de re\u021bea, toate vor fi acceptate. \u0218i de partea clientului, libpq va \u00eencepe s\u0103 numere timpii de a\u0219teptare. \u00cen mod implicit, se pare c\u0103 acolo sunt 3 secunde. <\/p>\n<p><\/p>\n<p>Dac\u0103 nu pot intra simultan \u00een baza de date, atunci nu pot intra, deoarece totul poate fi acoperit printr-un retry non-exponen\u021bial. <\/p>\n<p><\/p>\n<p>Am ajuns la concluzia c\u0103 am copiat aici schema de la PgBouncer cu ceea ce avem, respectiv throttling-ul num\u0103rului de conexiuni TCP pe care le accept\u0103m. <\/p>\n<p><\/p>\n<p>Dac\u0103 observ\u0103m c\u0103 accept\u0103m conexiuni, dar acestea nu reu\u0219esc s\u0103 finalizeze handshake-ul, le punem \u00een a\u0219teptare pentru a nu consuma resursele procesorului central. Acest lucru duce la faptul c\u0103 handshake-ul simultan poate s\u0103 nu se realizeze pentru toate conexiunile primite. Dar m\u0103car cineva va accesa baza de date, chiar dac\u0103 \u00eenc\u0103rcarea este destul de mare.<\/p>\n<p><\/p>\n<p><strong>Planul de ac\u021biune<\/strong><\/p>\n<p><\/p>\n<p>Ce ne dorim s\u0103 vedem \u00een viitorul Odyssey? Ce suntem preg\u0103ti\u021bi s\u0103 dezvolt\u0103m noi \u0219i ce a\u0219tept\u0103m de la comunitate?<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/4719437d80b33efc3da6572af9a4dedb.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p><strong>\u00cen august 2019.<\/strong><\/p>\n<p><\/p>\n<p>A\u0219a ar\u0103ta planul de ac\u021biune Odyssey \u00een august:<\/p>\n<p><\/p>\n<ul>\n<li>Am dori autentificarea SCRAM \u0219i PAM.<\/li>\n<li>Am dori s\u0103 redirec\u021bion\u0103m cererile de citire c\u0103tre standby. <\/li>\n<li>Ne-ar pl\u0103cea un restart online. <\/li>\n<li>\u0218i posibilitatea de a face o pauz\u0103 pe server. <\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/455c92f92df826257c2c57dc2b034e22.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Jum\u0103tate din acest plan de ac\u021biune a fost realizat, \u0219i nu de noi. \u0218i asta e bine. A\u0219a c\u0103 haide\u021bi s\u0103 discut\u0103m despre ce a mai r\u0103mas \u0219i s\u0103 ad\u0103ug\u0103m \u0219i altele. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/4a1daafbfe8d65ba0dd3e1ebdae8dd94.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex\/odyssey\/issues\/12\">\u00cen ceea ce prive\u0219te redirec\u021bionarea interog\u0103rilor de citire \u00een mod exclusiv c\u0103tre standby<\/a><\/noindex>? \u0423 \u043d\u0430\u0441 \u0435\u0441\u0442\u044c \u0440\u0435\u043f\u043b\u0438\u043a\u0438, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0431\u0435\u0437 \u0432\u044b\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u044f \u0437\u0430\u043f\u0440\u043e\u0441\u043e\u0432 \u0431\u0443\u0434\u0443\u0442 \u043f\u0440\u043e\u0441\u0442\u043e \u0433\u0440\u0435\u0442\u044c \u0432\u043e\u0437\u0434\u0443\u0445. \u041e\u043d\u0438 \u043d\u0430\u043c \u043d\u0435\u043e\u0431\u0445\u043e\u0434\u0438\u043c\u044b \u0434\u043b\u044f \u043e\u0431\u0435\u0441\u043f\u0435\u0447\u0435\u043d\u0438\u044f failover \u0438 switchover. \u0412 \u0441\u043b\u0443\u0447\u0430\u0435 \u043f\u0440\u043e\u0431\u043b\u0435\u043c \u0432 \u043e\u0434\u043d\u043e\u043c \u0438\u0437 \u0434\u0430\u0442\u0430-\u0446\u0435\u043d\u0442\u0440\u0435 \u0445\u043e\u0442\u0435\u043b\u043e\u0441\u044c \u0431\u044b \u0438\u0445 \u0437\u0430\u043d\u044f\u0442\u044c \u043a\u0430\u043a\u043e\u0439-\u0442\u043e \u043f\u043e\u043b\u0435\u0437\u043d\u043e\u0439 \u0440\u0430\u0431\u043e\u0442\u043e\u0439. \u041f\u043e\u0442\u043e\u043c\u0443 \u0447\u0442\u043e \u0442\u0435 \u0436\u0435 \u0441\u0430\u043c\u044b\u0435 \u0446\u0435\u043d\u0442\u0440\u0430\u043b\u044c\u043d\u044b\u0435 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u044b, \u0442\u0443 \u0436\u0435 \u0441\u0430\u043c\u0443\u044e \u043f\u0430\u043c\u044f\u0442\u044c \u043c\u044b \u043d\u0435 \u043c\u043e\u0436\u0435\u043c \u0441\u043a\u043e\u043d\u0444\u0438\u0433\u0443\u0440\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043f\u043e-\u0434\u0440\u0443\u0433\u043e\u043c\u0443, \u043f\u043e\u0442\u043e\u043c\u0443 \u0447\u0442\u043e \u0438\u043d\u0430\u0447\u0435 \u043d\u0435 \u0431\u0443\u0434\u0435\u0442 \u0440\u0430\u0431\u043e\u0442\u0430\u0442\u044c \u0440\u0435\u043f\u043b\u0438\u043a\u0430\u0446\u0438\u044f.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/0873cffd4f1ae15d9ffa6d080671051b.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u00cen principiu, \u00een Postgres, \u00eencep\u00e2nd cu versiunea 10, exist\u0103 posibilitatea de a specifica session_attrs la conectare. Pute\u021bi enumera toate gazdele bazei de date \u00een conexiune \u0219i pute\u021bi spune de ce merge\u021bi la baza de date: pentru a scrie sau doar pentru a citi. Iar driverul va alege singur prima gazd\u0103 din list\u0103 care \u00eei place mai mult \u0219i care \u00eendepline\u0219te cerin\u021bele session_attrs. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/0693d1327fe5826502a01c21d05d5b4c.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Dar problema acestei abord\u0103ri este c\u0103 nu controleaz\u0103 \u00eent\u00e2rzierea replic\u0103rii. Poate exista o replic\u0103 care este \u00eent\u00e2rziat\u0103 cu un timp inacceptabil pentru serviciul dumneavoastr\u0103. Pentru a realiza o execu\u021bie complet\u0103 a cererilor de citire pe replic\u0103, \u00een esen\u021b\u0103, trebuie s\u0103 sprijinim \u00een Odyssey capacitatea de a nu lucra atunci c\u00e2nd citirea nu este permis\u0103. <\/p>\n<p><\/p>\n<p>Odyssey ar trebui s\u0103 solicite din c\u00e2nd \u00een c\u00e2nd baza de date \u0219i s\u0103 \u00eentrebe despre distan\u021ba de replicare de la primar. \u0218i dac\u0103 aceasta a atins un anumit prag, s\u0103 blocheze noile cereri c\u0103tre baz\u0103, s\u0103 informeze clientul c\u0103 trebuie s\u0103 reini\u021bializeze conexiunile \u0219i, posibil, s\u0103 aleag\u0103 o alt\u0103 gazd\u0103 pentru a executa cererile. Aceasta va permite bazei s\u0103 recupereze mai repede \u00eent\u00e2rzierea replic\u0103rii \u0219i s\u0103 revin\u0103 pentru a r\u0103spunde cererilor. <\/p>\n<p><\/p>\n<p>Este dificil de stabilit termenele de realizare, deoarece este open source. Dar, sper c\u0103 nu va dura 2,5 ani ca la colegii no\u0219tri de la PgBouncer. Aceast\u0103 func\u021bie ne-ar pl\u0103cea s\u0103 o vedem \u00een Odyssey. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/4ec3cd2ff7b2298927cccfe8e0ba890a.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p><noindex><a rel=\"nofollow\" href=\"https:\/\/github.com\/yandex\/odyssey\/issues\/16\">\u00cen comunitate, oamenii au \u00eentrebat despre suportul pentru prepared statement<\/a><\/noindex>. Acum pute\u021bi crea un prepared statement \u00een dou\u0103 moduri. \u00cen primul r\u00e2nd, pute\u021bi executa comanda SQL, \u0219i anume \u201eprepared\u201d. Pentru a \u00een\u021belege aceast\u0103 comand\u0103 SQL, trebuie s\u0103 \u00eenv\u0103\u021b\u0103m s\u0103 \u00een\u021belegem SQL din partea Bouncer. Ar fi o exagerare, deoarece avem nevoie de un parser complet. Nu putem analiza fiecare comand\u0103 SQL. <\/p>\n<p><\/p>\n<p>Dar exist\u0103 un prepared statement la nivelul protocolului de mesaje pe proto3. \u0218i acesta este momentul \u00een care informa\u021bia c\u0103 se creeaz\u0103 un prepared statement ajunge \u00eentr-o form\u0103 structurat\u0103. Am putea sus\u021bine \u00een\u021belegerea c\u0103 pe o anumit\u0103 conexiune de server, clientul a solicitat s\u0103 se creeze prepared statements. Chiar dac\u0103 tranzac\u021bia s-a \u00eencheiat, trebuie s\u0103 men\u021binem \u00een continuare coeren\u021ba \u00eentre server \u0219i client. <\/p>\n<p><\/p>\n<p>Dar aici apare o discrepan\u021b\u0103 \u00een dialog, deoarece cineva spune c\u0103 trebuie s\u0103 \u00een\u021belegem ce anume prepared statements a creat clientul \u0219i s\u0103 separ\u0103m conexiunea serverului \u00eentre to\u021bi clien\u021bii care au creat aceast\u0103 conexiune de server, adic\u0103 care au creat un astfel de prepared statement. <\/p>\n<p><\/p>\n<p>Andres Freund a spus c\u0103, dac\u0103 a venit un client la voi care a creat deja un astfel de prepared statement \u00een alt\u0103 conexiune de server, atunci s\u0103-l crea\u021bi pentru el. Dar, se pare, c\u0103 este pu\u021bin gre\u0219it s\u0103 efectuezi interog\u0103ri \u00een baza de date \u00een locul clientului, dar din perspectiva dezvoltatorului care scrie protocolul de interac\u021biune cu baza, ar fi convenabil s\u0103 i se ofere pur \u0219i simplu o conexiune de re\u021bea \u00een care exist\u0103 o astfel de interogare preg\u0103tit\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/cd56b3ad015195d86a64e52be365b16a.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>\u0218i \u00eenc\u0103 o caracteristic\u0103 pe care trebuie s\u0103 o implement\u0103m. Acum avem monitorizare compatibil\u0103 cu PgBouncer. Putem returna timpul mediu de execu\u021bie a interog\u0103rii. Dar timpul mediu este ca o temperatur\u0103 medie \u00een spital: cineva este rece, cineva cald \u2013 \u00een medie, to\u021bi sunt s\u0103n\u0103to\u0219i. Aceasta nu este adev\u0103rul. <\/p>\n<p><\/p>\n<p>Trebuie s\u0103 implement\u0103m suport pentru percentiles care ar indica existen\u021ba unor interog\u0103ri lente ce consum\u0103 resurse \u0219i ar face monitorizarea mai acceptabil\u0103. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/6f285f3eb0d79e7f25d42f0e792b85b1.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Cel mai important \u2013 se dore\u0219te versiunea 1.0 (Versiunea 1.1 a fost deja lansat\u0103). Problema este c\u0103 Odyssey se afl\u0103 acum \u00een versiunea 1.0rc, adic\u0103 release candidate. \u0218i toate problemele pe care le-am enumerat au fost remediate exact cu acea versiune, cu excep\u021bia scurgerii de memorie. <\/p>\n<p><\/p>\n<p>Ce va \u00eensemna pentru noi versiunea 1.0? Lans\u0103m Odyssey pe baza noastr\u0103. Deja func\u021bioneaz\u0103 pe serverele noastre, dar c\u00e2nd va atinge 1.000.000 de cereri pe secund\u0103, putem spune c\u0103 aceasta este versiunea de lansare, versiunea pe care o putem numi 1.0. <\/p>\n<p><\/p>\n<p>\u00cen comunitate, c\u00e2\u021biva oameni au cerut ca versiunea 1.0 s\u0103 aib\u0103 \u0219i o suspendare \u0219i SCRAM. Dar acest lucru va \u00eensemna c\u0103 va trebui s\u0103 lans\u0103m deja o versiune urm\u0103toare \u00een produc\u021bie, deoarece nici SCRAM, nici suspendarea nu sunt \u00eenc\u0103 integrate. Totu\u0219i, este probabil ca aceast\u0103 problem\u0103 s\u0103 fie rezolvat\u0103 destul de repede. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Odyssey roadmap: ce altceva ne dorim de la agregatorul de conexiuni. Andrei Borodin (2019)\" src=\"\/wp-content\/uploads\/2020\/10\/2f50de9f51f653a7396965239bd545e1.jpg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>A\u0219tept cererile dvs. de pull. De asemenea, a\u0219 dori s\u0103 aud despre problemele pe care le ave\u021bi cu Bouncer. Hai s\u0103 le discut\u0103m. Poate c\u0103 putem implementa unele func\u021bii de care ave\u021bi nevoie. <\/p>\n<p><\/p>\n<p>Aici se \u00eencheie partea mea, a\u0219 dori s\u0103 v\u0103 ascult. Mul\u021bumesc!<\/p>\n<p><\/p>\n<p>\u00centreb\u0103ri<\/p>\n<p><\/p>\n<p><em>Dac\u0103 \u00eemi setez application_name, va fi gestionat corect, inclusiv \u00een transaction pooling \u00een Odyssey?<\/em><\/p>\n<p><\/p>\n<p>\u00cen Odyssey sau \u00een Bouncer?<\/p>\n<p><\/p>\n<p><em>\u00cen Odyssey. \u00cen Bouncer este gestionat.<\/em> <\/p>\n<p><\/p>\n<p>Vom face configura\u021bia. <\/p>\n<p><\/p>\n<p><em>\u0218i dac\u0103 conexiunea mea real\u0103 va s\u0103ri \u00eentre alte conexiuni, va fi transmis\u0103?<\/em><\/p>\n<p><\/p>\n<p>Vom face setul tuturor parametrilor enumera\u021bi \u00een list\u0103. Nu pot spune dac\u0103 application_name se afl\u0103 \u00een acea list\u0103. Mi se pare c\u0103 l-am v\u0103zut acolo. Vom seta to\u021bi aceia\u0219i parametri. Cu o cerere acolo, setarea va face tot ceea ce a fost configurat de client la startup. <\/p>\n<p><\/p>\n<p><em>Mul\u021bumesc, Andrei, pentru prezentare! O prezentare bun\u0103! M\u0103 bucur c\u0103 Odyssey evolueaz\u0103 din ce \u00een ce mai repede cu fiecare minut. V\u0103 doresc s\u0103 continua\u021bi la fel. Ne-am adresat deja cu rug\u0103mintea de a avea o conexiune multi data-source, astfel \u00eenc\u00e2t Odyssey s\u0103 se poat\u0103 conecta simultan la diferite baze de date, adic\u0103 master-slave, iar apoi, dup\u0103 failover, s\u0103 se conecteze automat la noul master.<\/em> <\/p>\n<p><\/p>\n<p>Da, parc\u0103 \u00eemi amintesc aceast\u0103 discu\u021bie. Acum exist\u0103 c\u00e2teva stoc\u0103ri. Dar nu exist\u0103 comut\u0103ri \u00eentre ele. Trebuie s\u0103 verific\u0103m serverul de partea noastr\u0103, s\u0103 ne asigur\u0103m c\u0103 este \u00eenc\u0103 activ \u0219i s\u0103 \u00een\u021belegem c\u0103 a avut loc un failover, cine va apela pg_recovery. Am o metod\u0103 standard de a \u00een\u021belege c\u0103 nu am ajuns pe master. \u0218i trebuie s\u0103 \u00een\u021belegem cumva din erori sau cum? Adic\u0103, ideea este interesant\u0103, se discut\u0103. Scrie\u021bi mai multe comentarii. Dac\u0103 ave\u021bi oameni care \u0219tiu C, atunci este minunat. <\/p>\n<p><\/p>\n<p>\u00centrebarea privind scalarea replicilor ne intereseaz\u0103 de asemenea, deoarece dorim s\u0103 facem adoptarea clusterelor replicate c\u00e2t mai simpl\u0103 pentru dezvoltatorii de aplica\u021bii. Totu\u0219i, ne-ar pl\u0103cea s\u0103 avem mai multe comentarii, adic\u0103 cum anume s\u0103 proced\u0103m, cum s\u0103 facem corect. <\/p>\n<p><\/p>\n<p><em>\u00centrebarea se refer\u0103 \u0219i la replici. Se pare c\u0103 ave\u021bi un master \u0219i mai multe replici. Este clar c\u0103 la replici se fac mai pu\u021bine conexiuni dec\u00e2t la master, deoarece acestea pot avea diferen\u021be. A\u021bi men\u021bionat c\u0103 diferen\u021ba \u00een date poate fi at\u00e2t de mare \u00eenc\u00e2t s\u0103 nu corespund\u0103 afacerii dvs. \u0219i nu ve\u021bi accesa acea replic\u0103 p\u00e2n\u0103 c\u00e2nd nu va fi complet replicat\u0103. Totu\u0219i, dac\u0103 nu a\u021bi accesat-o de mult timp \u0219i apoi \u00eencepe\u021bi s\u0103 o face\u021bi, datele de care ave\u021bi nevoie nu vor fi imediat disponibile. Adic\u0103, dac\u0103 accedem constant la master, acolo cache-ul este \u00eenc\u0103rcat, \u00een timp ce \u00een replic\u0103 cache-ul \u00eent\u00e2rzie pu\u021bin.<\/em> <\/p>\n<p><\/p>\n<p>Da, este adev\u0103rat. \u00cen pcache nu vor fi blocurile de date pe care le dori\u021bi, \u00een real cache nu va exista informa\u021bia despre tabelele pe care le dori\u021bi, iar \u00een planuri nu vor fi interog\u0103rile parsate, nu va exista nimic. <\/p>\n<p><\/p>\n<p><em>\u0218i c\u00e2nd ave\u021bi un cluster \u0219i ad\u0103uga\u021bi o nou\u0103 replic\u0103, atunci \u00een timp ce aceasta se porne\u0219te, \u00een ea totul este prost, adic\u0103 acumuleaz\u0103 cache-ul.<\/em> <\/p>\n<p><\/p>\n<p>Am \u00een\u021beles ideea. Abordarea corect\u0103 ar fi s\u0103 rula\u021bi un mic procent din interog\u0103ri ini\u021bial pe replic\u0103, care s\u0103 \u00eenc\u0103lzeasc\u0103 cache-ul. Practic, avem condi\u021bia c\u0103 nu trebuie s\u0103 r\u0103m\u00e2nem \u00een urm\u0103 cu mai mult de 10 secunde fa\u021b\u0103 de master. \u0218i aceast\u0103 condi\u021bie s\u0103 fie activat\u0103 nu dintr-o dat\u0103, ci treptat pentru c\u00e2\u021biva clien\u021bi. <\/p>\n<p><\/p>\n<p><em>Da, s\u0103 cre\u0219tem greutatea.<\/em><\/p>\n<p><\/p>\n<p>Este o idee bun\u0103. Dar mai \u00eent\u00e2i trebuie s\u0103 implement\u0103m aceast\u0103 dezactivare. Trebuie s\u0103 ne oprim mai \u00eent\u00e2i \u0219i abia apoi s\u0103 ne g\u00e2ndim cum s\u0103 ne activ\u0103m. Este o func\u021bionalitate excelent\u0103 pentru a ne activa treptat.<\/p>\n<p><\/p>\n<p><em>\u00cen nginx exist\u0103 aceast\u0103 op\u021biune. <code>slowly start<\/code> \u00een clusterul pentru server. \u0218i el cre\u0219te treptat sarcina.<\/em> <\/p>\n<p><\/p>\n<p>Da, idee excelent\u0103, vom \u00eencerca c\u00e2nd vom ajunge la asta.<\/p>\n<p>Sursa: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/post\/522734\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>\u0412 \u0434\u043e\u043a\u043b\u0430\u0434\u0435 \u0410\u043d\u0434\u0440\u0435\u0439 \u0411\u043e\u0440\u043e\u0434\u0438\u043d \u0440\u0430\u0441\u0441\u043a\u0430\u0436\u0435\u0442, \u043a\u0430\u043a \u043e\u043d\u0438 \u0443\u0447\u043b\u0438 \u043e\u043f\u044b\u0442 \u043c\u0430\u0441\u0448\u0442\u0430\u0431\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u044f PgBouncer \u043f\u0440\u0438 \u043f\u0440\u043e\u0435\u043a\u0442\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0438 \u043f\u0443\u043b\u0435\u0440\u0430 \u0441\u043e\u0435\u0434\u0438\u043d\u0435\u043d\u0438\u0439 Odyssey, \u043a\u0430\u043a \u0432\u044b\u043a\u0430\u0442\u044b\u0432\u0430\u043b\u0438 \u0435\u0433\u043e \u0432 production. \u041a\u0440\u043e\u043c\u0435 \u0442\u043e\u0433\u043e, \u043e\u0431\u0441\u0443\u0434\u0438\u043c \u043a\u0430\u043a\u0438\u0435 \u0444\u0443\u043d\u043a\u0446\u0438\u0438 \u043f\u0443\u043b\u0435\u0440\u0430 \u0445\u043e\u0442\u0435\u043b\u043e\u0441\u044c \u0431\u044b \u0432\u0438\u0434\u0435\u0442\u044c \u0432 \u043d\u043e\u0432\u044b\u0445 \u0432\u0435\u0440\u0441\u0438\u044f\u0445: \u043d\u0430\u043c \u0432\u0430\u0436\u043d\u043e \u043d\u0435 \u0442\u043e\u043b\u044c\u043a\u043e \u0437\u0430\u043a\u0440\u044b\u0432\u0430\u0442\u044c \u0441\u0432\u043e\u0438 \u043f\u043e\u0442\u0440\u0435\u0431\u043d\u043e\u0441\u0442\u0438, \u043d\u043e \u0440\u0430\u0437\u0432\u0438\u0432\u0430\u0442\u044c \u0441\u043e\u043e\u0431\u0449\u0435\u0441\u0442\u0432\u043e \u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u0435\u0439 \u041e\u0434\u0438\u0441\u0441\u0435\u044f. \u0412\u0438\u0434\u0435\u043e: \u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442! \u041c\u0435\u043d\u044f \u0437\u043e\u0432\u0443\u0442 \u0410\u043d\u0434\u0440\u0435\u0439. \u0412 \u042f\u043d\u0434\u0435\u043a\u0441\u0435 \u044f \u0437\u0430\u043d\u0438\u043c\u0430\u044e\u0441\u044c [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":96516,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-96515","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.2.1 - aioseo.com -->\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/odyssey-roadmap-chto-eshhyo-my-hotim-ot-pulera-soedinenij-andrej-borodin-2019\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.2.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"ro_RO\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47Odyssey roadmap: \u0447\u0442\u043e \u0435\u0449\u0451 \u043c\u044b \u0445\u043e\u0442\u0438\u043c \u043e\u0442 \u043f\u0443\u043b\u0435\u0440\u0430 \u0441\u043e\u0435\u0434\u0438\u043d\u0435\u043d\u0438\u0439. \u0410\u043d\u0434\u0440\u0435\u0439 \u0411\u043e\u0440\u043e\u0434\u0438\u043d (2019) | ProHoster\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/odyssey-roadmap-chto-eshhyo-my-hotim-ot-pulera-soedinenij-andrej-borodin-2019\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-10-12T17:43:02+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-10-12T17:43:02+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Odyssey roadmap: ce mai dorim de la pool-ul de conexiuni. Andrei Borodin (2019) | ProHoster","description":"","canonical_url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/odyssey-roadmap-chto-eshhyo-my-hotim-ot-pulera-soedinenij-andrej-borodin-2019","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"ro_RO","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47Odyssey roadmap: \u0447\u0442\u043e \u0435\u0449\u0451 \u043c\u044b \u0445\u043e\u0442\u0438\u043c \u043e\u0442 \u043f\u0443\u043b\u0435\u0440\u0430 \u0441\u043e\u0435\u0434\u0438\u043d\u0435\u043d\u0438\u0439. \u0410\u043d\u0434\u0440\u0435\u0439 \u0411\u043e\u0440\u043e\u0434\u0438\u043d (2019) | ProHoster","og:url":"https:\/\/prohoster.info\/ro\/blog\/administrirovanie\/odyssey-roadmap-chto-eshhyo-my-hotim-ot-pulera-soedinenij-andrej-borodin-2019","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-10-12T17:43:02+00:00","article:modified_time":"2020-10-12T17:43:02+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"96515","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 10:41:30","updated":"2022-10-01 00:02:14","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/96515","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/comments?post=96515"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/posts\/96515\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media\/96516"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/media?parent=96515"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/categories?post=96515"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/ro\/wp-json\/wp\/v2\/tags?post=96515"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}