🥇Isticmaalka mcrouter si loo cabbiro memcached horizontal

Horumarinta mashaariicda culus ee luuqad kasta waxay u baahan tahay hab gaar ah iyo isticmaalka qalab gaar ah, laakiin marka ay timaado codsiyada PHP, xaaladdu waxay noqon kartaa mid aad u sii xumaanaysa oo ay tahay inaad horumariso, tusaale ahaan, server codsi u gaar ah. Qoraalkan waxaan kaga hadli doonaa xanuunka la yaqaan ee kaydinta fadhiga la qaybiyey iyo kaydinta xogta ee memcached iyo sida aan u xalinay dhibaatooyinkan hal mashruuc "waadh".

Halyeyga munaasabadda waa codsi PHP ah oo ku saleysan qaab dhismeedka symfony 2.3, kaas oo aan haba yaraatee ku jirin qorshooyinka ganacsiga ee cusboonaysiinta. Marka lagu daro kaydinta fadhiga caadiga ah, mashruucani wuxuu si buuxda u isticmaalay siyaasadda "wax walba kaydinta". in memcached: jawaabaha codsiyada kaydka xogta iyo adeegayaasha API, calammo kala duwan, qufulyada isku xidhka fulinta koodka iyo wax ka badan. Xaaladdan oo kale, burburka memcached wuxuu noqdaa mid dilaa ah hawlgalka codsiga. Intaa waxaa dheer, luminta kaydka waxay keenaysaa cawaaqib xumo: DBMS waxay bilaabataa inay ka dillaacdo seeraha, adeegyada API waxay bilaabaan inay mamnuucaan codsiyada, iwm. Dejinta xaaladdu waxay qaadan kartaa tobanaan daqiiqo, inta lagu jiro wakhtigan adeeggu wuxuu noqon doonaa mid aad u gaabis ah ama gebi ahaanba lama heli karo.

Waxaan u baahanahay inaan bixino kartida loo leeyahay in si siman loo cabbiro codsiga dadaal yar, i.e. iyada oo isbeddellada ugu yar ee koodhka isha iyo shaqeyn buuxda la ilaaliyo. Ka dhig khasnad kaliya maaha mid u adkaysata guuldarrooyinka, laakiin sidoo kale isku day inaad yareyso khasaaraha xogta.

Maxaa ka qaldan memcached laftiisa?

Guud ahaan, kordhinta memcached ee PHP waxay taageertaa xogta la qaybiyey iyo kaydinta kalfadhi ee sanduuqa. Habka hashing furaha joogtada ah wuxuu kuu ogolaanayaa inaad si siman u dhigto xogta servero badan, si gaar ah u wajaheysa fure kasta oo server gaar ah oo ka tirsan kooxda, iyo qalabyada ku dhex jira ee fashilinta waxay xaqiijinayaan helitaanka sare ee adeegga kaydinta (laakiin, nasiib daro, xog la'aan).

Arrimuhu wax yar bay ku roon yihiin kaydinta fadhiga: waad habayn kartaa memcached.sess_number_of_replicas, taas oo keentay in xogta lagu kaydiyo dhowr server hal mar, iyo haddii ay dhacdo in hal tusaale oo memcached ah guuldareysto, xogta waxaa laga wareejin doonaa kuwa kale. Si kastaba ha noqotee, haddii server-ku uu ku soo laabto khadka internetka iyada oo aan la helin xog (sida caadiga ah waxay dhacdaa dib u bilaabashada), qaar ka mid ah furayaasha ayaa dib loo qaybin doonaa iyada. Dhab ahaantii tani waxay macnaheedu noqon doontaa luminta xogta fadhiga, mar haddii aysan jirin si loo "tago" nuqul kale haddii ay dhacdo in la waayo.

Qalabka maktabadda caadiga ah waxaa inta badan loogu talagalay siman xoqin: waxay kuu oggolaanayaan inaad kordhiso khasnadda ilaa cabbirro aad u weyn oo aad ka hesho koodka lagu hayo server-yada kala duwan. Si kastaba ha ahaatee, xaaladeena, mugga xogta kaydsan kama badna dhowr gigabytes, iyo waxqabadka hal ama laba qanjidhada waa ku filan. Sidaa darteed, qalabka caadiga ah ee kaliya ee waxtarka leh ayaa noqon kara in la hubiyo in la heli karo memcached iyadoo la ilaalinayo ugu yaraan hal tusaale oo kayd ah xaalad shaqo. Si kastaba ha ahaatee, suurtagal ma ahayn in laga faa'iidaysto xitaa fursadan ... Halkan waxaa habboon in dib loo xasuusto qaab-dhismeedka hore ee loo isticmaalo mashruuca, taas oo ah sababta aysan suurtagal ahayn in la helo codsiga si uu ula shaqeeyo barkad server ah. Aynaan sidoo kale iloobin luminta xogta fadhiga: isha macmiilka ayaa ka soo baxday calaamadda weyn ee isticmaaleyaasha.

Sida habboon waa loo baahnaa Ku-noqoshada diiwaannada ku jira nuqullada memcaved iyo la hareer maray haddii ay dhacdo khalad ama khalad. Waxay naga caawisay hirgelinta istaraatiijiyadan mcrouter.

mcrouter

Kani waa router memcached ah oo ay samaysay Facebook si uu u xalliyo dhibaatooyinkiisa. Waxay taageertaa hab-maamuuska qoraalka memcached, kaas oo ogolaanaya rakibaadaha miisaanka memcached ilaa xad waalan. Faahfaahin faahfaahsan oo ku saabsan mcrouter ayaa laga heli karaa ogeysiiskan. Waxaa ka mid ah shaqeyn ballaaran waxay sameyn kartaa waxaan u baahanahay:

diiwaanka soo noqnoqda;
dib ugu noqo server-yada kale ee kooxda haddii uu khalad dhaco.

Shaqada tag!

qaabeynta mcrouter

Waxaan toos u aadi doonaa qaabeynta:

{
 "pools": {
   "pool00": {
     "servers": [
       "mc-0.mc:11211",
       "mc-1.mc:11211",
       "mc-2.mc:11211"
   },
   "pool01": {
     "servers": [
       "mc-1.mc:11211",
       "mc-2.mc:11211",
       "mc-0.mc:11211"
   },
   "pool02": {
     "servers": [
       "mc-2.mc:11211",
       "mc-0.mc:11211",
       "mc-1.mc:11211"
 },
 "route": {
   "type": "OperationSelectorRoute",
   "default_policy": "AllMajorityRoute|Pool|pool00",
   "operation_policies": {
     "get": {
       "type": "RandomRoute",
       "children": [
         "MissFailoverRoute|Pool|pool02",
         "MissFailoverRoute|Pool|pool00",
         "MissFailoverRoute|Pool|pool01"
       ]
     }
   }
 }
}

Waa maxay sababta saddex barkadood? Waa maxay sababta server-yada loogu celceliyo? Aynu ogaano sida ay u shaqeyso.

Qaabeyntan, mcrouter ayaa dooranaya dariiqa codsiga loo diri doono iyadoo lagu saleynayo amarka codsiga. Ninkii ayaa u sheegay tan OperationSelectorRoute.
Codsiyada HEL waxay aadayaan maamulaha RandomRouteKaas oo si bakhtiyaa nasiib ah u doorta barkad ama waddo ka mid ah walxaha diyaarsan children. Cunsur kasta oo ka mid ah shaxdan ayaa markeeda ah maamule MissFailoverRoute, kaas oo dhex mari doona server kasta oo barkada ku jira ilaa uu ka helayo jawaab celin xog ah, kaas oo lagu soo celin doono macmiilka.
Haddii aan si gaar ah u isticmaalnay MissFailoverRoute oo leh barkad ka kooban saddex server, ka dibna dhammaan codsiyada ayaa marka hore iman doona tusaalaha ugu horreeya ee memcached, inta soo hartayna waxay heli doonaan codsiyo ku saleysan hadhaaga marka aysan jirin xog. Habka noocan oo kale ah ayaa horseedi doona culeys xad dhaaf ah oo ku saabsan server-ka ugu horreeya ee liiska, sidaas darteed waxaa la go'aamiyay in la sameeyo saddex barkadood oo leh cinwaanno kala duwan oo kala duwan oo loo doorto si aan kala sooc lahayn.
Dhammaan codsiyada kale (oo kani waa diiwaan) waxaa lagu farsameeyaa iyadoo la isticmaalayo AllMajorityRoute. Xakameeyahani wuxuu codsiyo u diraa dhammaan adeegayaasha barkadda wuxuuna sugayaa jawaabaha ugu yaraan N/2 + 1 oo iyaga ka mid ah. Isticmaalka AllSyncRoute Hawlgallada qorista waxay ahayd in laga tago, maadaama habkani uu u baahan yahay jawaab celin wanaagsan всех server-yada kooxda - haddii kale way soo noqon doontaa SERVER_ERROR. In kasta oo mcrouter uu xogta ku dari doono kaydka la heli karo, wacitaanka PHP soo celin doonaa qalad waxayna dhalin doontaa ogeysiis. AllMajorityRoute ma aha mid aad u adag oo ogolaanaya ilaa kala badh unugyada in laga saaro adeega iyada oo aan dhibaatooyinka kor lagu sharaxay.

Khasaaraha ugu weyn Nidaamkan ayaa ah in haddii runtii aysan jirin wax xog ah oo ku jira kaydka, markaa codsi kasta oo ka yimaada macmiilka N codsiyada ku saabsan memcached dhab ahaantii waa la fulin doonaa - si dhammaan dadka adeegayaasha barkada dhexdeeda. Waxaan hoos u dhigi karnaa tirada server-yada barkadaha, tusaale ahaan, laba: hurinta kalsoonida kaydinta, waxaan helnaaоxawaaraha sare iyo culeys yar oo ka yimid codsiyada ilaa furayaasha maqan.

NB: Waxa kale oo aad ka heli kartaa xiriiriyayaal waxtar u leh barashada mcrouter dukumeenti ku saabsan wiki и arimaha mashruuca (oo ay ku jiraan kuwa xiran), oo matalaya bakhaar dhan oo qaabab kala duwan leh.

Dhisidda iyo socodsiinta mcrouter

Codsigeena (oo laftiisa ayaa la duubay) wuxuu ku shaqeeyaa Kubernetes - sidaas darteed, mcrouter ayaa sidoo kale ku yaal halkaas. Waayo ururinta weelka waxaan isticmaalnaa werf, qaabaynta kaas oo u ekaan doona sidan:

NBLiisaska lagu sheegay maqaalka waxaa lagu daabacay kaydka flant/mcrouter.

configVersion: 1
project: mcrouter
deploy:
 namespace: '[[ env ]]'
 helmRelease: '[[ project ]]-[[ env ]]'
---
image: mcrouter
from: ubuntu:16.04
mount:
- from: tmp_dir
 to: /var/lib/apt/lists
- from: build_dir
 to: /var/cache/apt
ansible:
 beforeInstall:
 - name: Install prerequisites
   apt:
     name: [ 'apt-transport-https', 'tzdata', 'locales' ]
     update_cache: yes
 - name: Add mcrouter APT key
   apt_key:
     url: https://facebook.github.io/mcrouter/debrepo/xenial/PUBLIC.KEY
 - name: Add mcrouter Repo
   apt_repository:
     repo: deb https://facebook.github.io/mcrouter/debrepo/xenial xenial contrib
     filename: mcrouter
     update_cache: yes
 - name: Set timezone
   timezone:
     name: "Europe/Moscow"
 - name: Ensure a locale exists
   locale_gen:
     name: en_US.UTF-8
     state: present
 install:
 - name: Install mcrouter
   apt:
     name: [ 'mcrouter' ]

(werf.yaml)

... oo sawir shaxda Helm. Waxa xiisaha lihi waa in uu jiro oo kaliya koronto-dhaliye ku salaysan tirada nuqullada (haddii qof uu haysto ikhtiyaar laconic iyo xarrago badan, la wadaag faallooyinka):

{{- $count := (pluck .Values.global.env .Values.memcached.replicas | first | default .Values.memcached.replicas._default | int) -}}
{{- $pools := dict -}}
{{- $servers := list -}}
{{- /* Заполняем  массив двумя копиями серверов: "0 1 2 0 1 2" */ -}}
{{- range until 2 -}}
 {{- range $i, $_ := until $count -}}
   {{- $servers = append $servers (printf "mc-%d.mc:11211" $i) -}}
 {{- end -}}
{{- end -}}
{{- /* Смещаясь по массиву, получаем N срезов: "[0 1 2] [1 2 0] [2 0 1]" */ -}}
{{- range $i, $_ := until $count -}}
 {{- $pool := dict "servers" (slice $servers $i (add $i $count)) -}}
 {{- $_ := set $pools (printf "MissFailoverRoute|Pool|pool%02d" $i) $pool -}}
{{- end -}}
---
apiVersion: v1
kind: ConfigMap
metadata:
 name: mcrouter
data:
 config.json: |
   {
     "pools": {{- $pools | toJson | replace "MissFailoverRoute|Pool|" "" -}},
     "route": {
       "type": "OperationSelectorRoute",
       "default_policy": "AllMajorityRoute|Pool|pool00",
       "operation_policies": {
         "get": {
           "type": "RandomRoute",
           "children": {{- keys $pools | toJson }}
         }
       }
     }
   }

(10-mcrouter.yaml)

Waxaan u gudbinaa deegaanka tijaabada oo aan hubinay:

# php -a
Interactive mode enabled

php > # Проверяем запись и чтение
php > $m = new Memcached();
php > $m->addServer('mcrouter', 11211);
php > var_dump($m->set('test', 'value'));
bool(true)
php > var_dump($m->get('test'));
string(5) "value"
php > # Работает! Тестируем работу сессий:
php > ini_set('session.save_handler', 'memcached');
php > ini_set('session.save_path', 'mcrouter:11211');
php > var_dump(session_start());
PHP Warning:  Uncaught Error: Failed to create session ID: memcached (path: mcrouter:11211) in php shell code:1
Stack trace:
#0 php shell code(1): session_start()
#1 {main}
  thrown in php shell code on line 1
php > # Не заводится… Попробуем задать session_id:
php > session_id("zzz");
php > var_dump(session_start());
PHP Warning:  session_start(): Cannot send session cookie - headers already sent by (output started at php shell code:1) in php shell code on line 1
PHP Warning:  session_start(): Failed to write session lock: UNKNOWN READ FAILURE in php shell code on line 1
PHP Warning:  session_start(): Failed to write session lock: UNKNOWN READ FAILURE in php shell code on line 1
PHP Warning:  session_start(): Failed to write session lock: UNKNOWN READ FAILURE in php shell code on line 1
PHP Warning:  session_start(): Failed to write session lock: UNKNOWN READ FAILURE in php shell code on line 1
PHP Warning:  session_start(): Failed to write session lock: UNKNOWN READ FAILURE in php shell code on line 1
PHP Warning:  session_start(): Failed to write session lock: UNKNOWN READ FAILURE in php shell code on line 1
PHP Warning:  session_start(): Unable to clear session lock record in php shell code on line 1
PHP Warning:  session_start(): Failed to read session data: memcached (path: mcrouter:11211) in php shell code on line 1
bool(false)
php >

Raadinta qoraalka khaladku wax natiijo ah kama soo bixin, laakiin waydiinta "microuter php"Kaalmada hore waxay ahayd dhibaatada ugu da'da weyn ee aan la xallin ee mashruuca - taageero la'aan nidaamka binary memcached.

NB: Hab-maamuuska ASCII ee ku jira memcached wuu ka gaabiyaa kan binary-ga, iyo habka caadiga ah ee hashing furaha joogtada ah waxay la shaqeeyaan borotokoolka binary-ga. Laakiin tani dhibaato uma abuurto kiis gaar ah.

Khiyaamadu waxay ku jirtaa boorsada: waxa kaliya ee ay tahay inaad sameyso waa u beddela borotokoolka ASCII wax walbana way shaqeyn doonaan.... Si kastaba ha noqotee, kiiskan, caado u ah raadinta jawaabaha gudaha dukumentiyada ku jira php.net ciyaaray kaftan naxariis daran. Jawaabta saxda ah halkaas kama heli doontid... ilaa, dabcan, aad u rogto dhamaadka, meesha qaybta "Xusuusyada isticmaalaha ayaa wax ku darsaday" noqon doonaa aamin iyo jawaab aan cadaalad ahayn.

Haa, magaca doorashada saxda ah waa memcached.sess_binary_protocol. Waa in la naafo, markaas ka dib fadhiyadu waxay bilaabi doonaan inay shaqeeyaan. Waxa hadhay oo dhan waa in weelka lagu rido mcrouter oo la geliyo boodh leh PHP!

gunaanad

Haddaba, isbeddelo kaabayaal dhaqaale oo keliya ayaannu awoodnay inaan xallino dhibaatada: arrinta u dulqaadashada cilladaha qarsoon waa la xalliyey, waxaana la kordhiyey isku halaynta kaydinta kaydinta. Marka lagu daro faa'iidooyinka muuqda ee codsiga, tani waxay siisay qol loogu talagalay maneuver markaad ka shaqeyneyso goobta: marka dhammaan qaybaha ay leeyihiin kayd, nolosha maamulaha ayaa si weyn loo fududeeyay. Haa, habkani waxa kale oo uu leeyahay dib-u-dhaciisa, wuxuu u ekaan karaa sida "curyaan", laakiin haddii ay lacag kaydsato, dhibka aaso oo aan keenin kuwo cusub - maxaa diidaya?