Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

A cikin jawabinsa, Andrey Borodin zai gaya muku yadda suka yi la'akari da kwarewar PgBouncer lokacin zayyana mahaɗin haɗin gwiwa. Odyssey, yadda suka fitar da shi a samarwa. Bugu da ƙari, za mu tattauna abin da ayyuka na pooler za mu so mu gani a cikin sababbin sigogi: yana da mahimmanci a gare mu ba kawai don rufe bukatunmu ba, amma don bunkasa al'ummar masu amfani. Odyssey.

Video:

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Sannu duka! Sunana Andrew.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

A Yandex, Ina haɓaka buɗaɗɗen bayanan bayanai. Kuma a yau muna da batu game da haɗin haɗin haɗin gwiwa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Idan kun san yadda ake kiran haɗin haɗin gwiwa cikin Rashanci, to ku gaya mani. Ina so in sami kyakkyawan lokaci na fasaha wanda ya kamata a kafa a cikin wallafe-wallafen fasaha.

Maudu'in yana da rikitarwa sosai, saboda a cikin ɗakunan bayanai da yawa an gina haɗin haɗin gwiwa kuma ba kwa buƙatar saninsa. Wasu saitunan, ba shakka, suna ko'ina, amma a cikin Postgres wannan baya aiki. Kuma a cikin layi daya (a HighLoad ++ 2019) akwai rahoton Nikolai Samokhvalov akan kafa tambayoyi a cikin Postgres. Kuma na fahimci cewa mutane sun zo nan waɗanda suka riga sun tsara buƙatun daidai, kuma waɗannan mutane ne waɗanda ke fuskantar matsalolin tsarin da ba su da yawa da suka shafi hanyar sadarwa, amfani da albarkatu. Kuma a wasu wurare yana iya zama da wahala a ma'anar cewa matsalolin ba a bayyane suke ba.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Yandex yana da Postgres. Yawancin ayyukan Yandex suna rayuwa a cikin Yandex.Cloud. Kuma muna da adadin petabytes na bayanai waɗanda ke samar da aƙalla buƙatun miliyan ɗaya a cikin Postgres.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Kuma muna samar da gungu na yau da kullun ga duk sabis - wannan shine babban kumburi na kumburi, kwafi biyu na yau da kullun (synchronous da asynchronous), madadin, buƙatun karatu akan kwafin.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Kowane kullin gungu shine Postgres, wanda, ban da Postgres da tsarin sa ido, ana kuma shigar da mahaɗar haɗin gwiwa. Ana amfani da pooler Connection don shinge kuma don babban manufarsa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Menene babban manufar mahaɗar haɗin gwiwa?

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Postgres yana ɗaukar ƙirar tsari don aiki tare da bayanan bayanai. Wannan yana nufin cewa haɗi ɗaya tsari ɗaya ne, ɗayan Postgres baya. Kuma akwai caches daban-daban da yawa a cikin wannan bangon baya, waɗanda suke da tsada sosai don yin daban don haɗin gwiwa daban-daban.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Hakanan, akwai tsararru a cikin lambar Postgres da ake kira procArray. Ya ƙunshi mahimman bayanai game da haɗin yanar gizo. Kuma kusan dukkanin algorithms na sarrafa procArray suna da sarƙaƙƙiya na layi, suna gudana ta hanyar haɗin yanar gizo gaba ɗaya. Yana da kyakkyawan zagayowar sauri, amma tare da ƙarin haɗin haɗin yanar gizo mai shigowa, abubuwa suna samun ɗan tsada. Kuma lokacin da abubuwa suka ɗan yi tsada, za ku biya farashi mai yawa don yawan adadin hanyoyin sadarwa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Akwai hanyoyi guda uku masu yiwuwa:

  • A gefen aikace-aikacen.
  • A gefen bayanan bayanai.
  • Kuma tsakanin, wato, duk haɗuwa mai yiwuwa.

Abin takaici, ginannen ma'aunin ruwa a halin yanzu yana kan haɓakawa. Abokai a PostgreSQL Professional suna yin wannan galibi. Lokacin da zai bayyana yana da wuyar tsinkaya. Kuma a gaskiya ma, muna da mafita guda biyu don zaɓin gine-gine. Waɗannan wuraren tafki ne-gefen aikace-aikace da wurin wakillan wakili.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Pool-gefen aikace-aikace ita ce hanya mafi sauƙi. Kuma kusan duk direbobin abokin ciniki suna ba ku hanya: don wakiltar miliyoyin haɗin haɗin ku a cikin lambar azaman wasu dozin na haɗin yanar gizo zuwa bayanan.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Akwai matsala tare da gaskiyar cewa a wani lokaci da kake son ƙaddamar da ƙarshen baya, kana so ka tura shi zuwa na'urori masu mahimmanci da yawa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Sannan har yanzu kuna gane cewa kuna da ƙarin wuraren samuwa da yawa, cibiyoyin bayanai da yawa. Kuma tsarin haɗin gwiwar abokin ciniki yana kaiwa ga manyan lambobi. Manyan su kusan haɗin gwiwa 10 ne. Wannan gefe ne wanda zai iya aiki da kyau.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Idan muka yi magana game da masu ba da izini, to akwai masu ruwa da tsaki guda biyu waɗanda za su iya yin abubuwa da yawa. Ba kawai masu tada ruwa ba ne. Su ne masu wasan ruwa + ƙarin ayyuka masu kyau. Wannan pgpool и Crunchy Proxy.

Amma, abin takaici, ba kowa ba ne ke buƙatar wannan ƙarin aikin. Kuma yana haifar da gaskiyar cewa masu taruwa suna goyan bayan haɗawar zaman kawai, watau abokin ciniki ɗaya mai shigowa, abokin ciniki ɗaya mai fita zuwa bayanan bayanai.

Wannan bai dace sosai ga ayyukanmu ba, don haka muna amfani da PgBouncer, wanda ke aiwatar da hada-hadar ma'amala, watau haɗin yanar gizo ana tsara taswirar haɗin gwiwar abokin ciniki kawai don tsawon lokacin ciniki.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Kuma a kan nauyinmu - gaskiya ne. Amma akwai matsaloli da yawa.Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Matsaloli suna farawa lokacin da kake son tantance zaman, saboda duk haɗin da ke shigowa gida ne. Kowa ya zo da loopback kuma ko ta yaya ya zama da wahala a gano zaman.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Tabbas zaku iya amfani da application_name_add_host. Wannan ita ce hanyar gefen Bouncer don ƙara adireshin IP zuwa sunan aikace-aikacen. Amma application_name an saita ta da ƙarin haɗin gwiwa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

A kan wannan ginshiƙi, inda layin rawaya shine ainihin buƙatun, kuma inda layin shuɗi shine buƙatun da ke tashi zuwa cikin bayanan. Kuma wannan bambancin shine daidai saitin application_name, wanda kawai ake buƙata don ganowa, amma ba shi da kyauta.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Bugu da ƙari, Bouncer ba zai iya iyakance tafkin guda ɗaya ba, watau adadin haɗin bayanai na kowane mai amfani, kowane ɗayan bayanai.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Menene wannan ke haifarwa? Kuna da sabis ɗin da aka ɗora a rubuce a cikin C ++ kuma wani wuri kusa da ƙaramin sabis akan kumburin da ba ya yin wani laifi tare da tushe, amma direbansa ya haukace. Yana buɗe haɗin haɗin 20 kuma komai zai jira. Hatta lambar ku daidai ne.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Tabbas, mun rubuta ƙaramin faci don Bouncer wanda ya ƙara wannan saitin, watau iyakance abokan ciniki zuwa tafkin.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Zai yiwu a yi haka a gefen Postgres, watau iyakance ayyuka a cikin bayanan zuwa adadin haɗin.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Amma sai ka rasa ikon fahimtar dalilin da yasa ba ku da haɗin kai zuwa uwar garken. PgBouncer baya jefa kuskuren haɗi, koyaushe yana dawo da bayanai iri ɗaya. Kuma ba za ku iya fahimta ba: watakila kalmar sirri ta canza, watakila ma'aunin bayanai ya ragu, watakila wani abu ba daidai ba ne. Amma babu ganewar asali. Idan ba za a iya kafa zaman ba, ba za ku san dalilin da ya sa ba za a iya yin shi ba.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

A wani lokaci, kuna duba hotuna na aikace-aikacen kuma ku ga cewa aikace-aikacen ba ya aiki.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Dubi saman ku ga cewa Bouncer mai zaren guda ɗaya ne. Wannan shine sauyi a rayuwar sabis. Kun fahimci cewa kuna shirye-shiryen auna ma'auni a cikin shekara guda da rabi, kuma kuna buƙatar sikelin ma'auni.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Mun kai ga ƙarshe cewa muna buƙatar ƙarin PgBouncers.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

https://lwn.net/Articles/542629/

Bouncer an ɗan yi masa faci.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Kuma sun sanya shi don a iya tayar da Bouncers da yawa tare da sake amfani da tashar TCP. Kuma riga tsarin aiki ta atomatik yana canja wurin haɗin TCP mai shigowa tsakanin su ta hanyar zagaye-robin'om.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Wannan a bayyane yake ga abokan ciniki, watau yana kama da kuna da Bouncer guda ɗaya, amma kuna da rarrabuwar haɗin kai mara amfani tsakanin Gudun Bouncers.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Kuma a wani lokaci, zaku iya lura cewa waɗannan 3 Bouncers kowannensu yana cin ainihin su ta 100%. Kuna buƙatar 'yan Bouncers kaɗan. Me yasa?

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Domin kuna da TLS. Kuna da rufaffen haɗi. Kuma idan kun nuna alamar Postgres tare da kuma ba tare da TLS ba, zaku ga cewa adadin haɗin haɗin da aka kafa ya ragu da kusan umarni biyu na girma tare da kunna ɓoyayye, saboda musayen hannu na TLS yana cinye albarkatun CPU.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Kuma a saman, zaku iya ganin ƴan ayyukan sirrin sirri waɗanda ake aiwatar da su yayin guguwar haɗin kai mai shigowa. Tun da farko na mu na iya canzawa tsakanin yankunan samuwa, guguwar haɗin haɗin gwiwa wani yanayi ne na yau da kullun. Wato, saboda wasu dalilai, tsohon firamare bai samu ba, an aika duka kayan zuwa wata cibiyar bayanai. Duk za su zo su gai da TLS a lokaci guda.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Kuma babban adadin musafaha na TLS na iya zama ba za su gaishe Bouncer ba, amma ya matse makogwaronsa. Guguwar haɗin haɗin da ke shigowa na iya zama mara lalacewa saboda ƙarewar lokaci. Idan kuna da sake gwadawa zuwa tushe ba tare da juzu'i mai ma'ana ba, ba za su sake dawowa akai-akai a cikin madaidaicin igiyar ruwa ba.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Anan akwai misalin 16 PgBouncers waɗanda ke ɗaukar nauyin 16 a 100%.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Mun isa PgBouncer cascading. Wannan shine mafi kyawun tsari da zamu iya cimma akan nauyin Bouncer mu. Bouncers ɗin mu na waje suna aiki don musafaha na TCP, kuma Bouncers na ciki suna aiki don haɗawa ta gaske, don kar su ɓata haɗin waje sosai.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

A cikin wannan saitin, sake kunnawa mai laushi yana yiwuwa. Kuna iya sake kunna duk waɗannan Bouncers 18 ɗaya bayan ɗaya. Amma kiyaye irin wannan tsari yana da wuyar gaske. Masu gudanar da tsarin, DevOps, da mutanen da ke da alhakin wannan uwar garken ba za su yi farin ciki da wannan makirci ba.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Da alama ana iya haɓaka duk abubuwan haɓakarmu a buɗe tushen, amma Bouncer baya tallafawa sosai. Misali, ikon gudanar da PgBouncers da yawa akan tashar jiragen ruwa guda an yi shi wata guda da ta gabata. Buƙatar ja tare da wannan fasalin shine ƴan shekarun da suka gabata.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

https://www.postgresql.org/docs/current/libpq-cancel.html

https://github.com/pgbouncer/pgbouncer/pull/79

Ko kuma wani misali guda ɗaya. A cikin Postgres, zaku iya soke buƙatun da ke gudana ta hanyar aika sirrin zuwa wata hanyar haɗi ba tare da ƙarin tabbaci ba. Amma wasu abokan ciniki kawai suna aika TCP-reset, watau sun karya haɗin yanar gizon. Menene Bouncer zai yi da wannan? Ba zai yi komai ba. Za ta ci gaba da aiwatar da bukatar. Idan kun sami adadi mai yawa na haɗin gwiwa waɗanda suka kafa tushe tare da ƙananan buƙatun, to kawai cire haɗin haɗin daga Bouncer ba zai isa ba, kuna buƙatar kammala waɗannan buƙatun da ke gudana a cikin bayanan.

An daidaita wannan kuma har yanzu batun ba a haɗa shi cikin babban rafi na Bouncer ba.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Sabili da haka mun zo ga ƙarshe cewa muna buƙatar haɗin haɗin haɗin gwiwarmu, wanda za a haɓaka, patched, wanda zai yiwu a hanzarta gyara matsalolin kuma wanda, ba shakka, dole ne ya kasance mai ɗaure-tsari.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Mun saita multithreading a matsayin babban aiki. Muna buƙatar samun damar ɗaukar igiyoyin haɗin TLS masu shigowa da kyau.

Don yin wannan, dole ne mu haɓaka wani ɗakin karatu daban mai suna Machinarium, wanda aka ƙera shi don bayyana yanayin injin hanyar haɗin yanar gizo azaman serial code. Idan ka duba lambar tushe na libpq, za ka ga kyawawan kiraye-kirayen da za su iya mayar maka da sakamako su ce, “Kira ni kaɗan kaɗan. A yanzu ina da IO a yanzu, amma lokacin da IO ya wuce, Ina da kaya akan processor. Kuma wannan tsari ne mai matakai da yawa. Na'urar jiha yawanci ana siffanta hulɗar hanyar sadarwa. Dokoki da yawa kamar "Idan a baya na karɓi fakiti na girman N, to yanzu ina jiran N bytes", "Idan na aika fakitin SYNC, to yanzu ina jiran fakiti tare da metadata sakamako." Ya zama lambar ƙima mai wuyar gaske, kamar dai an canza maze ɗin zuwa duban layi. Mun sanya shi ta yadda a maimakon na'ura na jihar, mai tsara shirye-shirye ya bayyana babban hanyar mu'amala a cikin nau'i na lambobi na yau da kullun. Kawai a cikin wannan lambar mahimmanci, kuna buƙatar shigar da wuraren da jerin aiwatarwa ke buƙatar katsewa ta hanyar jiran bayanai daga hanyar sadarwar, wuce yanayin aiwatarwa zuwa wani coroutine (zaren kore). Wannan hanya ta yi kama da gaskiyar cewa muna rubuta hanyar da aka fi tsammanin a cikin maze a jere, sa'an nan kuma ƙara rassa zuwa gare ta.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

A sakamakon haka, muna da zaren guda ɗaya wanda ke sa TCP ta karɓi kuma zagaye-robin ya wuce haɗin TPC zuwa yawancin ma'aikata.

A wannan yanayin, kowane haɗin abokin ciniki koyaushe yana gudana akan processor ɗaya. Kuma wannan yana ba ku damar sanya shi cikin amintaccen cache.

Bayan haka, mun ɗan inganta tarin ƙananan fakiti cikin babban fakiti guda ɗaya domin sauke tsarin TCP.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Bugu da kari, mun inganta hada-hadar ma'amala ta ma'anar cewa Odyssey, lokacin da aka daidaita shi, zai iya aika CANCEL da ROLLBACK idan akwai gazawar haɗin yanar gizo, watau idan babu wanda ke jiran buƙatar, Odyssey zai gaya wa ma'ajiyar bayanai kada suyi ƙoƙarin cikawa. buƙatar da za ta iya ɓata albarkatu masu daraja.

Kuma duk lokacin da zai yiwu, muna ci gaba da haɗi zuwa abokin ciniki ɗaya. Wannan yana guje wa sake shigar da application_name_add_host. Idan zai yiwu, to, ba mu da ƙarin sake saiti na sigogi waɗanda ake buƙata don ganowa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Muna aiki a cikin bukatun Yandex.Cloud. Kuma idan kuna amfani da PostgreSQL mai sarrafawa kuma kuna da haɗin haɗin haɗin gwiwa, zaku iya ƙirƙirar kwafi mai ma'ana a waje, watau bar mu idan kuna so, ta amfani da kwafi na ma'ana. Bouncer a waje da kwararar kwafin hankali ba zai bayar ba.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Wannan misali ne na kafa kwafin hankali.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Bugu da kari, muna da goyon baya ga kwafi na jiki a waje. A cikin Cloud, ba shakka, ba zai yiwu ba, saboda to, gungu zai ba ku bayanai da yawa game da kansa. Amma a cikin shigarwar ku, idan kuna buƙatar kwafin jiki ta hanyar haɗin haɗin gwiwa a cikin Odyssey, yana yiwuwa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Odyssey yana da cikakkiyar kulawa tare da PgBouncer. Muna da na'ura mai kwakwalwa iri ɗaya wanda ke aiwatar da kusan duk umarni iri ɗaya. Idan wani abu ya ɓace, aika buƙatar ja, ko aƙalla matsala akan GitHub, za mu cika umarnin da suka dace. Amma mun riga mun sami babban aikin PgBouncer console.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Kuma tabbas muna da kuskuren turawa. Za mu dawo da kuskuren da tushe ya ruwaito. Za ku sami bayanin dalilin da yasa ba ku cikin tushe ba kawai cewa ba ku cikinsa ba.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

An kashe wannan fasalin idan kuna buƙatar dacewa 100% tare da PgBouncer. Za mu iya zama kamar Bouncer, kawai idan akwai.

Ƙaddamarwa

Kalmomi kaɗan game da lambar tushe Odyssey.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

https://github.com/yandex/odyssey/pull/66

Misali, akwai umarnin "Dakata / Ci gaba". Yawancin lokaci ana amfani da su don sabunta bayanan bayanai. Idan kuna buƙatar haɓaka Postgres, zaku iya dakatar da shi a cikin haɗin haɗin gwiwa, yi pg_upgrade, sannan ku ci gaba. Kuma daga bangaren abokin ciniki, zai yi kama da ma'aunin bayanai yana raguwa. Mutane daga al'umma ne suka kawo mana wannan aikin. Har yanzu ba ta mutu ba, amma nan da nan komai zai kasance. (ya riga ya mutu)

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

https://github.com/yandex/odyssey/pull/73 - ya riga ya mutu

Bugu da ƙari, ɗaya daga cikin sababbin abubuwan da ke cikin PgBouncer shine goyon bayan Tabbatar da SCRAM, wanda kuma mutumin da ba ya aiki a Yandex.Cloud ya kawo mana. Dukansu ayyuka ne masu rikitarwa da mahimmanci.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Don haka, Ina so in gaya muku abin da Odyssey aka yi dashi, idan har kuna son rubuta wasu lambobi yanzu.

Kuna da tushen asalin Odyssey, wanda ya dogara da manyan ɗakunan karatu guda biyu. Laburaren Kiwi aiwatar da ka'idar saƙon Postgres ne. Wato, Postgres' na asali proto 3 shine daidaitattun saƙonni waɗanda gaba da baya zasu iya musanyawa. Ana aiwatar da su a cikin ɗakin karatu na Kiwi.

Laburaren Machinarium ɗakin karatu ne na aiwatar da zaren. An rubuta ƙaramin guntun wannan Machinarium a cikin mahaɗa. Amma kar ku damu, akwai layi 15 kawai.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Odyssey architecture. Akwai babban injin da ke aiki da coroutines. Wannan injin yana aiwatar da karɓar haɗin TCP masu shigowa da rarrabawa tsakanin ma'aikata.

A cikin ma'aikaci ɗaya, mai kula da abokan ciniki da yawa na iya aiki. Hakanan a cikin babban zaren, na'ura mai kwakwalwa da sarrafa ayyukan crone don cire haɗin da ba a buƙata a cikin tafkin suna juyawa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Ana gwada Odyssey ta amfani da daidaitaccen ɗakin gwaji na Postgres. Muna gudanar da shigar-duba ta hanyar Bouncer kuma ta hanyar Odyssey, muna samun div mara kyau. Akwai gwaje-gwaje da yawa masu alaƙa da tsara kwanan wata waɗanda suka gaza daidai iri ɗaya a cikin Bouncer da Odyssey.

Bugu da kari, akwai direbobi da yawa da suke da nasu gwajin. Kuma muna amfani da gwaje-gwajen su don gwada Odyssey.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Har ila yau, saboda tsarin tsarin mu na cascading, dole ne mu gwada nau'o'i daban-daban: Postgres + Odyssey, PgBouncer + Odyssey, Odyssey + Odyssey don tabbatar da cewa idan Odyssey yana cikin kowane sassa a cikin cascade, har yanzu yana aiki kamar yadda ake tsammani. .

Rake

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Muna amfani da Odyssey wajen samarwa. Kuma ba zai yi kyau ba idan na ce komai yana aiki kawai. A'a, i.e., amma ba koyaushe ba. Misali, a cikin samarwa duk abin da kawai ya yi aiki, to abokanmu daga PostgreSQL Professional sun zo sun ce muna da ƙwaƙwalwar ƙwaƙwalwa. Da gaske sun kasance, mun gyara su. Amma ya kasance mai sauƙi.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Sannan mun gano cewa mai haɗin haɗin gwiwa yana da haɗin TLS masu shigowa da haɗin TLS masu fita. Kuma haɗin kai yana buƙatar takaddun shaida na abokin ciniki da takaddun shaida na uwar garken.

Takaddun shaida na uwar garken Bouncer da Odyssey ana sake karantawa ta hanyar pcache, amma takaddun shaida na abokin ciniki baya buƙatar sake karantawa daga pcache, saboda Odyssey ɗinmu mai ƙima daga ƙarshe ya dogara akan tsarin aikin karanta wannan takaddun shaida. Wannan ya zo mana da mamaki, domin bai huta nan da nan ba. Da farko, ya daidaita a kan layi, kuma bayan 20 masu shigowa lokaci guda, wannan matsala ta bayyana kanta.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Hanyar Tabbatar da Pluggable ita ce ikon tantancewa tare da ginanniyar kayan aikin lunux. A cikin PgBouncer, an aiwatar da shi ta hanyar da ke akwai keɓaɓɓen zaren da ke jiran amsa daga PAM kuma akwai babban zaren PgBouncer wanda ke ba da haɗin kai na yanzu kuma yana iya tambayar su su zauna a cikin zaren PAM.

Ba mu aiwatar da wannan don dalili ɗaya mai sauƙi ba. Muna da koguna da yawa. Me yasa muke bukata?

A sakamakon haka, wannan na iya haifar da matsaloli a cikin cewa idan kuna da ingantaccen PAM da kuma ba na PAM ba, to, babban igiyoyin tabbatar da PAM na iya jinkirta ba da tabbacin PAM. Yana daga cikin abubuwan da ba mu gyara ba. Amma idan kuna son gyara shi, kuna iya yin wannan.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Wani rake shine tare da gaskiyar cewa muna da zaren guda ɗaya wanda ke karɓar duk haɗin da ke shigowa. Sannan ana tura su zuwa wurin ma'aikaci, inda za a yi musafaha na TLS.

Sakamakon haka, idan kuna da madaidaicin igiyar hanyar sadarwa guda 20, duk za a karɓi su. Kuma a gefen abokin ciniki, libpq zai fara ba da rahoton lokacin ƙarewa. Ta hanyar tsoho, yana kama da daƙiƙa 000 a wurin.

Idan ba za su iya shiga cikin tushe a lokaci ɗaya ba, to ba za su iya shiga cikin tushe ba, saboda duk waɗannan za a iya rufe su ta hanyar sake gwadawa maras ma'ana.

Mun ƙare yin kwafin makircin PgBouncer a nan domin mu sami srottling adadin haɗin TCP da muka karɓa.

Idan muka ga cewa muna karɓar haɗin kai, amma ba su da lokacin yin musafaha a ƙarshe, muna sanya su a cikin layi don kada su cinye albarkatun CPU. Wannan yana haifar da gaskiyar cewa ba za a iya yin musafaha na lokaci ɗaya don duk haɗin da ya zo ba. Amma aƙalla wani zai shiga cikin ma'ajin bayanai, koda kuwa nauyin yana da ƙarfi sosai.

jadawalin

Me kuke so ku gani nan gaba a Odyssey? Me muka shirya don bunkasa kanmu kuma me muke tsammani daga al'umma?

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Domin Agusta 2019.

Wannan shine yadda taswirar hanya ta Odyssey tayi a watan Agusta:

  • Muna son tantancewar SCRAM da PAM.
  • Muna so mu tura buƙatun karantawa zuwa jiran aiki.
  • Ina son sake farawa akan layi
  • Da ikon tsayawa akan uwar garken.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Rabin wannan taswirar hanya an yi, ba ta mu ba. Kuma wannan yana da kyau. Don haka mu tattauna abin da ya rage mu kara.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Game da tambayoyin karantawa kawai zuwa jiran aiki? Muna da kwafi waɗanda, ba tare da biyan buƙatun ba, za su yi zafi kawai. Muna buƙatar su don samar da gazawar da canji. Idan akwai matsaloli a ɗaya daga cikin cibiyoyin bayanai, Ina so in shagaltar da su da wasu ayyuka masu amfani. Saboda ba za mu iya saita na'urori masu sarrafawa iri ɗaya ba, ƙwaƙwalwar ajiya iri ɗaya ta wata hanya dabam, saboda kwafi ba zai yi aiki ba.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

A ka'ida, a cikin Postgres, farawa daga 10, yana yiwuwa a ƙayyade session_attrs lokacin haɗi. Kuna iya jera duk rundunonin bayanan da ke cikin haɗin kuma ku faɗi dalilin da yasa kuke zuwa rumbun adana bayanai: rubuta ko karanta kawai. Kuma direban da kansa zai zabi mai masaukin baki na farko a jerin da ya fi so, wanda ya cika bukatun session_atrs.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Amma matsalar wannan hanyar ita ce ba ta sarrafa larurar kwafi. Kuna iya samun wani nau'in kwafi wanda ke baya ta lokacin da ba za a yarda da shi ba don sabis ɗin ku. Don yin cikakken aikin aiwatar da buƙatun karantawa akan kwafi, a zahiri, muna buƙatar tallafawa a cikin Odyssey ikon rashin aiki lokacin da ba zai yiwu a karanta ba.

Odyssey dole ne ya je rumbun adana bayanai lokaci zuwa lokaci kuma ya nemi nisa daga na farko. Kuma idan ya kai iyaka, kar a bar sabbin buƙatu a cikin ma'ajin bayanai, gaya wa abokin ciniki cewa kuna buƙatar sake fara haɗin gwiwa kuma, wataƙila, zaɓi wani runduna don aiwatar da buƙatun. Wannan zai ba da damar ma'ajin bayanai da sauri dawo da koma baya don amsa tambaya.

Yana da wahala a bayyana kwanakin aiwatarwa, saboda buɗaɗɗen tushe ne. Amma, ina fata, ba shekaru 2,5 ba kamar abokan aiki daga PgBouncer. Wannan shine fasalin da nake so in gani a Odyssey.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

A cikin al'umma, mutane sun yi tambaya game da tallafin sanarwa da aka shirya. Yanzu zaku iya ƙirƙirar sanarwa da aka shirya ta hanyoyi biyu. Da farko, zaku iya aiwatar da umarnin SQL, wato "shirya". Domin fahimtar wannan umarnin SQL, muna buƙatar koyon yadda ake fahimtar SQL a gefen Bouncer. Wannan zai zama wuce gona da iri saboda ya yi yawa tunda muna buƙatar gabaɗayan parser. Ba za mu iya rarraba kowane umarnin SQL ba.

Amma akwai shirye-shiryen sanarwa a matakin ka'idar saƙo akan proto3. Kuma wannan shi ne wurin da bayanin da aka shirya bayanin da aka shirya ya zo a cikin tsari mai tsari. Kuma zamu iya tallafawa fahimtar cewa akan wasu haɗin uwar garken abokin ciniki ya nemi ƙirƙirar maganganun da aka shirya. Kuma ko da an rufe ma'amala, har yanzu muna buƙatar ci gaba da haɗa uwar garken da abokin ciniki.

Amma a nan an sami sabani a cikin tattaunawar, saboda wani ya ce kuna buƙatar fahimtar waɗanne shirye-shiryen maganganun da abokin ciniki ya ƙirƙira da raba haɗin uwar garken tsakanin duk abokan cinikin da suka ƙirƙiri wannan haɗin uwar garken, watau, waɗanda suka ƙirƙiri irin wannan bayanan da aka shirya.

Andres Freund ya ce idan abokin ciniki ya zo gare ku wanda ya riga ya ƙirƙira irin wannan bayanin da aka shirya a cikin wani haɗin yanar gizo, to, ƙirƙira masa. Amma da alama yana da ɗan kuskure don aiwatar da tambayoyi a cikin ma'ajin bayanai maimakon abokin ciniki, amma daga ra'ayi na mai haɓakawa wanda ya rubuta yarjejeniya don hulɗa tare da bayanan, zai dace idan an ba shi haɗin yanar gizo kawai. wanda ke da irin wannan buƙatar da aka shirya.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Da kuma ƙarin fasalin da muke buƙatar aiwatarwa. Yanzu muna da saka idanu masu dacewa da PgBouncer. Za mu iya mayar da matsakaicin lokacin aiwatar da tambaya. Amma matsakaicin lokaci shine matsakaicin zafin jiki a asibiti: wani yana sanyi, wani yana dumi - a matsakaici kowa yana da lafiya. Ba gaskiya bane.

Muna buƙatar aiwatar da tallafi don ƙima, wanda zai nuna cewa akwai jinkirin buƙatun da ke cinye albarkatu kuma zai sa sa ido ya zama karɓuwa.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Abu mafi mahimmanci shine cewa ina son sigar 1.0 (An riga an fitar da sigar 1.1). Gaskiyar ita ce, yanzu Odyssey yana cikin sigar 1.0rc, watau ɗan takarar saki. Kuma duk rake da na lissafo an gyara su daidai da sigar guda ɗaya, sai dai ƙwaƙwalwar ƙwaƙwalwa.

Menene ma'anar sigar 1.0 a gare mu? Muna fitar da Odyssey zuwa sansanonin mu. Ya riga ya fara aiki a ma'ajin mu, amma idan ya kai ga buƙatun 1 a cikin daƙiƙa guda, to muna iya cewa wannan sigar saki ce kuma wannan sigar ce da za a iya kiran ta 000.

Mutane da yawa a cikin al'umma sun nemi ƙarin dakatarwa da SCRAM a cikin sigar 1.0. Amma wannan yana nufin cewa za mu buƙaci fitar da sigar gaba don samarwa, saboda ba a haɗa SCRAM ko dakatarwa ba tukuna. Amma, mai yiwuwa, za a warware wannan batu cikin sauri.

Taswirar hanya ta Odyssey: menene kuma muke so daga mahaɗar haɗin gwiwa. Andrey Borodin (2019)

Ina jiran bukatar ja ku. Kuma ina so in ji irin matsalolin da kuke da su tare da Bouncer. Mu tattauna su. Wataƙila za mu iya aiwatar da wasu ayyuka da kuke buƙata.

Wannan ya kawo karshen bangarena, ina son ji daga gare ku. Na gode!

Tambayoyi

Idan na sanya sunan application_na, za a jefa shi daidai, gami da hada-hadar hada-hadar kasuwanci a Odyssey?

Odyssey ko Bouncer?

In Odyssey. An jefa Bouncer.

Za mu yi saiti.

Kuma idan haɗina na ainihi ya yi tsalle a kan wasu hanyoyin sadarwa, za a yada shi?

Za mu yi saitin duk sigogin da aka jera. Ba zan iya sanin ko application_name yana cikin wannan jeri ba. Da alama ya ganshi a wajen. Za mu saita duk sigogi iri ɗaya. Tare da buƙatu ɗaya, saitin zai yi duk abin da abokin ciniki ya shigar yayin farawa.

Na gode Andrey da rahoton! Kyakkyawan rahoto! Na yi farin ciki cewa Odyssey yana haɓaka sauri da sauri kowane minti daya. Ina so in ci gaba da hakan. Mun riga mun nemi ku sami haɗin tushen bayanai da yawa ta yadda Odyssey zai iya haɗawa zuwa ma'ajin bayanai daban-daban a lokaci guda, watau master master, sannan ta atomatik haɗi zuwa sabon master bayan gazawar.

Eh, da alama na tuna waccan tattaunawar. Yanzu akwai ɗakunan ajiya da yawa. Amma babu sauyi a tsakaninsu. A gefenmu, dole ne mu tambayi uwar garken cewa har yanzu yana raye kuma mu fahimci cewa gazawar ta faru, wanda zai kira pg_recovery. Ina da daidaitaccen hanyar fahimtar cewa ba mu zo wurin maigida ba. Kuma dole ne mu fahimci ko ta yaya daga kuskure ko ta yaya? Wato ra'ayin yana da ban sha'awa, ana tattaunawa. Rubuta ƙarin sharhi. Idan kuna da hannayen aiki waɗanda suka san C, to wannan yana da ban mamaki gabaɗaya.

Batun sikeli a cikin kwafi shima yana da sha'awar mu, saboda muna so mu sanya riƙon gungu da aka kwafi a sauƙaƙe ga masu haɓaka aikace-aikacen. Amma a nan ina son ƙarin sharhi, wato, yadda za a yi, yadda za a yi shi da kyau.

Tambayar kuma game da kwafi. Sai dai itace cewa kana da master da yawa kwafi. Kuma a bayyane yake cewa suna zuwa kwafin sau da yawa fiye da maigidan don haɗi, saboda suna iya samun bambanci. Kun ce bambancin bayanai na iya zama wanda kasuwancin ku ba zai gamsar da ku ba kuma ba za ku je wurin ba har sai an maimaita shi. A lokaci guda, idan ba ku je wurin ba na dogon lokaci, sannan ku fara zuwa, to bayanan da kuke buƙata ba za su kasance nan take ba. Wato, idan muka ci gaba da zuwa wurin maigidan, to, cache yana dumama a can, kuma cache yana ɗan baya a cikin kwafin.

E gaskiya ne. Ba za a sami bayanan toshe bayanai a cikin pcache da kuke so ba, a cikin ainihin cache ba za a sami wani bayani game da tebur ɗin da kuke so ba, ba za a sami tambayoyin da aka warware ba a cikin tsare-tsaren, ba komai.

Kuma idan kana da wani nau'i na cluster, kuma ka ƙara sabon kwafi a can, to, yayin da ya fara, komai yana da kyau a ciki, watau yana girma cache.

Na sami ra'ayin. Hanyar da ta dace ita ce fara gudanar da ƙananan kaso na tambayoyin akan kwafin farko, wanda zai dumama cache. Kusan magana, muna da yanayin cewa dole ne mu kasance ba fiye da daƙiƙa 10 a bayan maigidan ba. Kuma wannan yanayin bai kamata a haɗa shi a cikin raƙuman ruwa ɗaya ba, amma a hankali ga wasu abokan ciniki.

Ee, ƙara nauyi.

Wannan kyakkyawan ra'ayi ne. Amma da farko kuna buƙatar aiwatar da wannan kashewa. Da farko muna bukatar mu kashe, sa'an nan kuma za mu yi tunanin yadda za a kunna. Wannan babban siffa ce don kunna su lafiya.

nginx yana da wannan zaɓi slowly start a cikin gungu don uwar garken. Kuma a hankali yakan gina kaya.

Ee, babban ra'ayi, za mu gwada lokacin da muka isa hakan.

source: www.habr.com

Add a comment