Ifu elilodwa - izinga lesikhungo sedatha OS ku-Odnoklassniki

Aloha, bantu! Igama lami ngingu-Oleg Anastasyev, ngisebenza e-Odnoklassniki eqenjini lePlatform. Futhi ngaphandle kwami, kunehardware eningi esebenza ku-Odnoklassniki. Sinezikhungo ezine zedatha ezinamarakhi angaba ngu-500 anamaseva angaphezu kwezinkulungwane eziyi-8. Esikhathini esithile, sabona ukuthi ukwethulwa kohlelo olusha lokuphatha kuzosivumela ukuthi silayishe izinto zokusebenza ngendlela ephumelela kakhudlwana, senze kube lula ukuphathwa kokufinyelela, senze ngokuzenzakalela ukusabalalisa (kabusha) kwezinsiza zekhompiyutha, kusheshiswe ukwethulwa kwezinsizakalo ezintsha, futhi kusheshiswe izimpendulo. ezingozini ezinkulu.

Kwavelani ngakho?

Ngaphandle kwami ​​​​kanye nenqwaba ye-hardware, kukhona nabantu abasebenza nale hardware: onjiniyela abatholakala ngokuqondile ezikhungweni zedatha; amanethiwekhi asungula isofthiwe yenethiwekhi; abalawuli, noma ama-SRE, ahlinzeka ngokuqina kwengqalasizinda; kanye namaqembu okuthuthukisa, ngalinye linesibopho sengxenye yemisebenzi yengosi. Isoftware abayakhayo isebenza kanje:

Izicelo zabasebenzisi zitholwa zombili emaphethelweni engosi enkulu www.ok.ru, nakwabanye, ngokwesibonelo kuma-API omculo. Ukucubungula i-logic yebhizinisi, babiza iseva yesicelo, okuthi, lapho icubungula isicelo, ibize ama-microservices akhethekile adingekayo - igrafu eyodwa (igrafu yokuxhumana nabantu), i-cache yomsebenzisi (i-cache yamaphrofayili womsebenzisi), njll.

Ngayinye yalezi zinsizakalo isetshenziswa emishinini eminingi, futhi ngamunye wabo unonjiniyela abanomthwalo wemfanelo wokusebenza kwamamojula, ukusebenza kwawo kanye nokuthuthukiswa kwezobuchwepheshe. Zonke lezi zinsizakalo zisebenza kumaseva wehadiwe, futhi kuze kube kamuva nje sethule umsebenzi owodwa ncamashi ngeseva ngayinye, okungukuthi ubukhethekile ngomsebenzi othile.

Kungani kunjalo? Le ndlela yayinezinzuzo ezimbalwa:

• Ukhululiwe ukuphathwa kwenqwaba. Ake sithi umsebenzi udinga imitapo yolwazi, ezinye izilungiselelo. Bese-ke iseva yabelwa iqembu elithile ncamashi, inqubomgomo ye-cfengine yaleli qembu iyachazwa (noma isivele ichaziwe), futhi lokhu kulungiselelwa kukhishwa phakathi nendawo futhi kukhishwa ngokuzenzakalelayo kuwo wonke amaseva akuleli qembu.
• Esenziwe lula zokuxilonga. Ake sithi ubheka umthwalo owengeziwe kuphrosesa emaphakathi futhi uqaphele ukuthi lo mthwalo ungakhiqizwa kuphela umsebenzi osebenza kule processor yehadiwe. Ukufuna umuntu ozomsola kuyaphela ngokushesha.
• Esenziwe lula ukuqapha. Uma kukhona okungalungile ngeseva, umqaphi uyabika, futhi wazi kahle ukuthi ubani okufanele asolwe.

Isevisi ehlanganisa ama-replicas amaningana inikezwa amaseva amaningana - eyodwa kwenye ngayinye. Khona-ke insiza yekhompiyutha yesevisi inikezwa kalula kakhulu: inombolo yamaseva isevisi enayo, inani eliphakeme lezinsiza engazisebenzisa. “Kulula” lapha akusho ukuthi kulula ukuyisebenzisa, kodwa ngomqondo wokuthi ukwabiwa kwezinsiza kwenziwa mathupha.

Le ndlela futhi yasivumela ukuba senze ukumiswa okukhethekile kwensimbi ngomsebenzi osebenza kule seva. Uma umsebenzi ugcina idatha enkulu, khona-ke sisebenzisa iseva ye-4U ene-chassis enamadiski angama-38. Uma umsebenzi ungowokubala, khona-ke singathenga iseva eshibhile ye-1U. Lokhu kusebenza kahle ngokwezibalo. Phakathi kokunye, le ndlela isivumela ukuthi sisebenzise imishini embalwa ngokuphindwe kane enomthwalo oqhathaniswa nenethiwekhi yokuxhumana nabantu enobungane.

Ukusebenza okunjalo ekusetshenzisweni kwezinsiza zekhompiyutha kufanele futhi kuqinisekise ukusebenza kahle kwezomnotho, uma sisuka esicathulweni sokuthi into ebiza kakhulu amaseva. Isikhathi eside, i-hardware yayibiza kakhulu, futhi senza umzamo omkhulu ekwehliseni intengo ye-hardware, siqhamuka ne-algorithms yokubekezelela iphutha ukuze kuncishiswe izidingo zokuthembeka kwehadiwe. Futhi namuhla sesifinyelele esigabeni lapho intengo yeseva iyekile ukunquma. Uma ungacabangi ama-exotics akamuva, khona-ke ukucushwa okuqondile kwamaseva ku-rack akusho lutho. Manje sinenye inkinga - intengo yesikhala esithathwe iseva esikhungweni sedatha, okungukuthi, isikhala ku-rack.

Ngokubona ukuthi kwakunjalo, sanquma ukubala ukuthi sasiwasebenzisa kahle kangakanani ama-rack.
Sithathe intengo yesiphakeli esinamandla kunawo wonke kwalawo athethelelekayo ngokwezomnotho, sabala ukuthi mangaki amaseva anjalo esingawabeka kuma-rack, mingaki imisebenzi esizowenza kuyo ngokusekelwe kumodeli endala “iseva eyodwa = umsebenzi owodwa” nokuthi ingakanani imisebenzi ingasebenzisa imishini. Babala zehla izinyembezi. Kuvele ukuthi ukusebenza kahle kwethu ekusebenziseni ama-rack cishe ku-11%. Isiphetho sisobala: sidinga ukwandisa ukusebenza kahle kokusebenzisa izikhungo zedatha. Kubukeka sengathi isisombululo sisobala: udinga ukwenza imisebenzi eminingana kuseva eyodwa ngesikhathi esisodwa. Kodwa yilapho ubunzima buqala khona.

Ukucushwa kwenqwaba kuba nzima kakhulu - manje akwenzeki ukwabela iqembu elilodwa kuseva. Phela, manje imisebenzi eminingana yemiyalo ehlukene ingaqaliswa kuseva eyodwa. Ukwengeza, ukulungiselelwa kungase kungqubuzane ezinhlelweni zokusebenza ezahlukene. Ukuxilongwa nakho kuba nzima kakhulu: uma ubona ukwanda kwe-CPU noma ukusetshenziswa kwediski kuseva, awazi ukuthi yimuphi umsebenzi obangela inkinga.

Kodwa into eyinhloko ukuthi akukho ukuhlukaniswa phakathi kwemisebenzi esebenza emshinini ofanayo. Lapha, isibonelo, igrafu yesilinganiso sesikhathi sokuphendula somsebenzi weseva ngaphambi nangemva kolunye uhlelo lokusebenza lokubala luqaliswe kuseva efanayo, ngendlela engahambisani neyokuqala - isikhathi sokuphendula somsebenzi oyinhloko sikhuphuke kakhulu.

Ngokusobala, udinga ukwenza imisebenzi ngeziqukathi noma emishinini ebonakalayo. Njengoba cishe yonke imisebenzi yethu isebenza ngaphansi kwe-OS (Linux) eyodwa noma iguqulelwe yona, asikho isidingo sokusekela amasistimu wokusebenza amaningi ahlukene. Ngakho-ke, i-virtualization ayidingeki; ngenxa ye-overhead eyengeziwe, izosebenza kancane kunokuqukatha.

Njengokusetshenziswa kweziqukathi zokuqhuba imisebenzi ngqo kumaseva, i-Docker iyikhandidethi elihle: izithombe zesistimu yefayela zixazulula izinkinga ngokucushwa okungqubuzanayo kahle. Iqiniso lokuthi izithombe zingakhiwa izendlalelo ezimbalwa lisivumela ukuthi sinciphise kakhulu inani ledatha edingekayo ukuze siwasebenzise engqalasizinda, sihlukanise izingxenye ezivamile zibe izendlalelo eziyisisekelo ezihlukene. Khona-ke izendlalelo eziyisisekelo (kanye neziningi kakhulu) zizogcinwa kunqolobane ngokushesha kuyo yonke ingqalasizinda, futhi ukuze kulethwe izinhlobo eziningi zezinhlelo zokusebenza nezinguqulo, izingqimba ezincane kuphela ezizodinga ukudluliselwa.

Futhi, ukubhaliswa okwenziwe ngomumo kanye nokumaka kwesithombe ku-Docker kusinikeza ama-primitives esenziwe ngomumo wokwenza inguqulo nokuletha ikhodi ekukhiqizweni.

I-Docker, njenganoma ibuphi obunye ubuchwepheshe obufanayo, isinika izinga elithile lokuhlukaniswa kweziqukathi ngaphandle kwebhokisi. Isibonelo, ukuhlukaniswa kwememori - isitsha ngasinye sinikezwa umkhawulo wokusetshenziswa kwememori yomshini, okungaphezu kwalokho okungeke kudle. Ungakwazi futhi ukuhlukanisa iziqukathi ngokusekelwe ekusetshenzisweni kwe-CPU. Kithina, nokho, ukwahlukanisa okujwayelekile kwakunganele. Kodwa okuningi kulokho ngezansi.

Ukusebenzisa ngokuqondile iziqukathi kumaseva kuyingxenye yenkinga. Enye ingxenye ihlobene neziqukathi zokusingatha kumaseva. Udinga ukuqonda ukuthi yisiphi isiqukathi esingabekwa kuso iseva. Lona akuwona umsebenzi olula, ngoba iziqukathi zidinga ukubekwa kumaseva aminyene ngangokunokwenzeka ngaphandle kokunciphisa isivinini sazo. Ukubekwa okunjalo kungase futhi kube nzima ngokombono wokubekezelela amaphutha. Ngokuvamile sifuna ukubeka izifanekiso zesevisi efanayo kuma-rack ahlukene noma ngisho emakamelweni ahlukene esikhungo sedatha, ukuze uma i-rack noma igumbi lihluleka, singalahlekelwa ngokushesha zonke izifanekiso zesevisi.

Ukusabalalisa iziqukathi mathupha akuyona inketho uma unamaseva ayizinkulungwane eziyi-8 neziqukathi eziyizinkulungwane eziyi-8-16.

Ngaphezu kwalokho, besifuna ukunikeza onjiniyela ukuzimela okwengeziwe ekwabiweni kwezinsiza ukuze bakwazi ukusingatha izinsiza zabo ekukhiqizeni ngokwabo, ngaphandle kosizo lomlawuli. Ngesikhathi esifanayo, besifuna ukugcina ukulawula ukuze isevisi ethile encane ingadli zonke izinsiza zezikhungo zethu zedatha.

Ngokusobala, sidinga isendlalelo sokulawula esingenza lokhu ngokuzenzakalelayo.

Ngakho-ke sifike esithombeni esilula nesiqondakalayo bonke abaklami bezakhiwo abasithandayo: izikwele ezintathu.

i-one-cloud masters iyiqoqo le-failover elibhekele ukucula kwamafu. Unjiniyela uthumela i-manifest kumphathi, equkethe lonke ulwazi oludingekayo ukuze usongethe isevisi. Ngokusekelwe kuyo, inkosi inika imiyalo kumaminion akhethiwe (imishini yakhelwe ukusebenzisa iziqukathi). Ama-minions ane-ejenti yethu, ethola umyalo, ikhipha imiyalo yayo ku-Docker, futhi i-Docker ilungisa i-linux kernel ukuze yethule isitsha esihambisanayo. Ngaphezu kokwenza imiyalo, i-ejenti ibika ngokuqhubekayo kumphathi mayelana nezinguquko esimweni sakho kokubili umshini omncane neziqukathi ezisebenza kuwo.

Ukwabiwa kwezinsiza

Manje ake sibheke inkinga yokwabiwa kwezinsiza eziyinkimbinkimbi kubalandeli abaningi.

Insiza yokwenza ikhompuyutha ku-one-cloud ithi:

• Inani lamandla okucubungula asetshenziswa umsebenzi othile.
• Inani lememori elitholakalayo kumsebenzi.
• Ithrafikhi yenethiwekhi. I-minion ngayinye ine-interface yenethiwekhi ethize enomkhawulokudonsa olinganiselwe, ngakho-ke akunakwenzeka ukusabalalisa imisebenzi ngaphandle kokucabangela inani ledatha abayidlulisela kunethiwekhi.
• Amadiski. Ngaphezu kwalokho, ngokusobala, esikhaleni sale misebenzi, siphinde sabe uhlobo lwediski: i-HDD noma i-SSD. Amadiski anganikeza inani elilinganiselwe lezicelo ngomzuzwana - IOPS. Ngakho-ke, emisebenzini ekhiqiza i-IOPS eningi kunediski eyodwa engakwazi ukuphatha, siphinde sabe "ama-spindles" - okungukuthi, amadivaysi ediski okufanele abekelwe umsebenzi kuphela.

Bese-ke ngesevisi ethile, ngokwesibonelo kunqolobane yomsebenzisi, singarekhoda izinsiza ezisetshenzisiwe ngale ndlela: 400 processor cores, 2,5 TB of memory, 50 Gbit/s traffic kuzo zombili izinkomba, 6 TB yesikhala se-HDD esitholakala kuma-spindle angu-100 . Noma ngendlela ejwayelekile kanjena:

alloc:
    cpu: 400
    mem: 2500
    lan_in: 50g
    lan_out: 50g
    hdd:100x6T

Izinsiza zesevisi yenqolobane yabasebenzisi zisebenzisa kuphela ingxenye yazo zonke izinsiza ezitholakalayo kungqalasizinda yokukhiqiza. Ngakho-ke, ngifuna ukwenza isiqiniseko sokuthi ngokuzumayo, ngenxa yephutha lomsebenzisi noma cha, i-cache yomsebenzisi ayidli izinsiza eziningi kunalokho ezabelwe yona. Okusho ukuthi, kufanele silinganisele izinsiza. Kodwa singabophelani isabelo?

Ake sibuyele kumdwebo wethu owenziwe lula kakhulu wokusebenzisana kwezingxenye futhi siwudwebe kabusha ngemininingwane eyengeziwe - kanje:

Yini ebamba iso lakho:

• I-frontend yewebhu nomculo kusebenzisa amaqoqo angawodwa eseva yohlelo lokusebenza efanayo.
• Singakwazi ukuhlukanisa izendlalelo ezinengqondo la maqoqo ayingxenye yazo: imingcele, inqolobane, ukugcinwa kwedatha nesendlalelo sokuphatha.
• Indawo engaphambili ine-heterogeneous; iqukethe amasistimu angaphansi ahlukene asebenzayo.
• Izinqolobane zingasatshalaliswa kusistimu engaphansi abagcina idatha yazo.

Ake sidwebe kabusha isithombe futhi:

Bah! Yebo, sibona isigaba sabaphathi! Lokhu kusho ukuthi ungasabalalisa izinsiza ngeziqephu ezinkudlwana: nikeza unjiniyela onomthwalo wemfanelo endaweni yalesi sigaba sokulandelana esihambisana nesistimu engaphansi yokusebenza (“njengomculo” esithombeni), futhi unamathisele isabelo ezingeni elifanayo lesigaba. Lokhu kuhlelwa kwezigaba futhi kusivumela ukuthi sihlele izinsizakalo ngendlela evumelana nezimo ukuze kube lula ukuphatha. Isibonelo, sihlukanisa yonke iwebhu, njengoba lokhu kuyiqoqo elikhulu kakhulu lamaseva, abe ngamaqembu amaningana amancane, aboniswe esithombeni njengeqembu1, iqembu2.

Ngokususa imigqa eyengeziwe, singabhala inodi ngayinye yesithombe sethu ngendlela eyisicaba: group1.web.front, api.music.front, i-user-cache.cache.

Yile ndlela esifika ngayo emcabangweni "womugqa we-hierarchical". Inegama elifana ne-"group1.web.front". Isabelo sezinsiza namalungelo abasebenzisi sabelwe sona. Sizonikeza umuntu ovela ku-DevOps amalungelo okuthumela isevisi kulayini, futhi isisebenzi esinjalo singaqalisa okuthile kulayini, futhi umuntu ovela ku-OpsDev uzoba namalungelo okuphatha, futhi manje usengakwazi ukuphatha ulayini, abele abantu lapho, nika laba bantu amalungelo, njll. Amasevisi asebenza kulo mugqa azosebenza phakathi kwesabelo somugqa. Uma i-quota yekhompuyutha yomugqa inganele ukwenza zonke izinkonzo ngesikhathi esisodwa, izobe isisetshenziswa ngokulandelana, ngaleyo ndlela yakhe ulayini ngokwawo.

Ake sibhekisise izinkonzo. Isevisi inegama elifaneleke ngokugcwele, elihlala lihlanganisa igama lomugqa. Khona-ke isevisi yewebhu yangaphambili izoba negama ok-web.group1.web.front. Futhi isevisi yeseva yohlelo lokusebenza efinyelela kuyo izobizwa ok-app.group1.web.front. Isevisi ngayinye ine-manifest, ecacisa lonke ulwazi oludingekayo lokubekwa emishinini ethile: zingaki izinsiza ezisetshenziswa lo msebenzi, yikuphi ukulungiselelwa okudingekayo kuwo, zingaki izifaniso okufanele zibe khona, izakhiwo zokusingatha ukwehluleka kwale sevisi. Futhi ngemva kokuba isevisi ibekwe ngokuqondile emishinini, izimo zayo ziyavela. Aphinde aqanjwe ngokusobala - njengenombolo yesibonelo kanye negama lesevisi: 1.ok-web.group1.web.front, 2.ok-web.group1.web.front, …

Lokhu kulula kakhulu: ngokubheka kuphela igama lesitsha esigijimayo, singathola ngokushesha okuningi.

Manje ake sibhekisise ukuthi lezi zimo empeleni zenzani: imisebenzi.

Amakilasi Wokuzihlukanisa Umsebenzi

Yonke imisebenzi kokuthi KULUNGILE (futhi, mhlawumbe, yonke indawo) ingahlukaniswa ngamaqembu:

• Imisebenzi yokubambezeleka emifushane - i-prod. Kuleyo misebenzi namasevisi, ukubambezeleka kwempendulo (ukubambezeleka) kubaluleke kakhulu, ukuthi isicelo ngasinye sizocutshungulwa ngokushesha kangakanani uhlelo. Izibonelo zemisebenzi: imingcele yewebhu, inqolobane, iziphakeli zohlelo, isitoreji se-OLTP, njll.
• Izinkinga zokubala - iqoqo. Lapha, isivinini sokucubungula isicelo ngasinye asibalulekile. Kubo, kubalulekile ukuthi zingaki izibalo ezizokwenziwa yilo msebenzi esikhathini esithile (eside) sesikhathi (umphumela). Lena kuzoba yimiphi imisebenzi ye-MapReduce, Hadoop, ukufunda ngomshini, izibalo.
• Imisebenzi yangemuva - ayisebenzi. Emisebenzini enjalo, akukho ukubambezeleka noma ukuphuma okubaluleke kakhulu. Lokhu kufaka phakathi ukuhlola okuhlukahlukene, ukufuduka, ukubala kabusha, nokuguqulwa kwedatha isuka kufomethi eyodwa iye kwenye. Ngakolunye uhlangothi, zifana nezibaliwe, ngakolunye uhlangothi, akusho lutho kithi ukuthi ziqedwa ngokushesha kangakanani.

Ake sibone ukuthi imisebenzi enjalo idla kanjani izinsiza, isibonelo, iphrosesa ephakathi.

Imisebenzi yokulibaziseka emifushane. Umsebenzi onjalo uzoba nephethini yokusetshenziswa kwe-CPU efana nalokhu:

Isicelo esivela kumsebenzisi samukelwe ukuze sicutshungulwe, umsebenzi uqala ukusebenzisa wonke ama-CPU cores atholakalayo, awucubungule, ubuyisela impendulo, ulinde isicelo esilandelayo bese uyama. Isicelo esilandelayo safika - futhi sakhetha konke okwakukhona, sakubala, futhi silindele okulandelayo.

Ukuqinisekisa ukubambezeleka okuncane komsebenzi onjalo, kufanele sithathe izinsiza eziphezulu esizisebenzisayo futhi sigcine inombolo edingekayo yama-cores ku-minion (umshini ozokwenza umsebenzi). Bese ifomula yokubhukha yenkinga yethu izoba kanje:

alloc: cpu = 4 (max)

futhi uma sinomshini omncane onama-cores angu-16, khona-ke imisebenzi emine enjalo ingabekwa kuwo. Siqaphela ngokukhethekile ukuthi ukusetshenziswa kweprosesa okujwayelekile kwemisebenzi enjalo kuvame ukuphansi kakhulu - okusobala, njengoba ingxenye ebalulekile yesikhathi umsebenzi ulinda isicelo futhi ungenzi lutho.

Imisebenzi yokubala. Iphethini yabo izohluka kancane:

Isilinganiso sokusetshenziswa kwensiza ye-CPU yemisebenzi enjalo kuphezulu kakhulu. Ngokuvamile sifuna umsebenzi wokubala uqedelwe ngesikhathi esithile, ngakho-ke sidinga ukugodla inani elincane lamaphrosesa eliwadingayo ukuze sonke isibalo siqedwe ngesikhathi esamukelekile. Ifomula yakho yokubhukha izobukeka kanje:

alloc: cpu = [1,*)

"Ngicela uyibeke endaweni encane lapho okungenani kunomgogodla owodwa wamahhala, bese kuthi ngobuningi babo, izoshwabadela yonke into."

Lapha ukusebenza kahle kokusetshenziswa sekuvele kungcono kakhulu kunemisebenzi enokubambezeleka okufushane. Kodwa inzuzo izoba nkulu kakhulu uma uhlanganisa zombili izinhlobo zemisebenzi emshinini owodwa omncane futhi usabalalisa izinsiza zawo ekuhambeni. Uma umsebenzi onokubambezeleka okufushane udinga iphrosesa, iwuthola ngokushesha, futhi lapho izinsiza zingasadingeki, zidluliselwa kumsebenzi wokubala, okungukuthi into efana nale:

Kepha ukwenza kanjani lokho?

Okokuqala, ake sibheke i-prod ne-alloc yayo: cpu = 4. Sidinga ukugcina ama-cores amane. Ku-Docker run lokhu kungenziwa ngezindlela ezimbili:

• Ukusebenzisa inketho --cpuset=1-4, i.e. nikeza ama-cores amane athile emshinini emsebenzini.
• Sebenzisa --cpuquota=400_000 --cpuperiod=100_000, nika isabelo sesikhathi sokucubungula, okungukuthi bonisa ukuthi njalo ngo-100 ms wesikhathi sangempela umsebenzi awudli ngaphezu kuka-400 ms wesikhathi sokucubungula. Ama-cores amane afanayo ayatholakala.

Kodwa iyiphi yalezi zindlela ezifanele?

i-cpuset ibukeka ikhanga impela. Umsebenzi unama-cores amane azinikezele, okusho ukuthi i-caches ye-processor izosebenza ngokuphumelelayo ngangokunokwenzeka. Lokhu futhi kunobubi: kuzodingeka sithathe umsebenzi wokusabalalisa izibalo kuwo wonke ama-cores angalayishiwe omshini esikhundleni se-OS, futhi lona umsebenzi ongewona omncane, ikakhulukazi uma sizama ukubeka imisebenzi ye-batch umshini. Ukuhlola kubonise ukuthi inketho enesabelo ifaneleka kangcono lapha: ngale ndlela isistimu yokusebenza inenkululeko eyengeziwe ekukhetheni umongo wokwenza umsebenzi okwamanje kanye nesikhathi sokucutshungulwa sisatshalaliswa kahle kakhulu.

Ake sithole ukuthi ukubhuka kanjani ku-Docker ngokusekelwe enanini elincane lama-cores. I-quota yemisebenzi yeqoqo ayisasebenzi, ngoba asikho isidingo sokukhawulela ubukhulu, kwanele ukuqinisekisa ubuncane. Futhi lapha inketho ifanelana kahle docker run --cpushares.

Sivumelene ngokuthi uma inqwaba idinga isiqinisekiso okungenani somgogodla owodwa, sizobe sesikhomba --cpushares=1024, futhi uma kukhona okungenani ama-cores amabili, khona-ke sibonisa --cpushares=2048. Amasheya e-Cpu awaphazamisi nganoma iyiphi indlela ekusatshalalisweni kwesikhathi sokucubungula inqobo nje uma sanele. Ngakho-ke, uma i-prod okwamanje ingasebenzisi wonke ama-cores ayo amane, akukho lutho olukhawulela imisebenzi ye-batch, futhi bangasebenzisa isikhathi esengeziwe sokucubungula. Kodwa esimweni lapho kukhona ukushoda kwama-processor, uma i-prod idle wonke ama-cores ayo amane futhi isifinyelele isilinganiso sayo, isikhathi esisele sokucubungula sizohlukaniswa ngokulingana nama-cpushares, okungukuthi esimweni sama-cores amathathu amahhala, eyodwa izoba unikezwe umsebenzi onama-cpushare angu-1024, futhi amabili asele azonikezwa umsebenzi onama-cpushare angu-2048.

Kodwa ukusebenzisa isabelo namasheya akwanele. Sidinga ukwenza isiqiniseko sokuthi umsebenzi onokubambezeleka okufushane uthola kuqala umsebenzi wenqwaba lapho sabela isikhathi sokucubungula. Ngaphandle kokubekwa phambili okunjalo, umsebenzi wenqwaba uzothatha sonke isikhathi sokucubungula ngaleso sikhathi lapho udingeka khona yi-prod. Azikho izinketho zokubekwa phambili kweziqukathi ekugijimeni kwe-Docker, kodwa izinqubomgomo ze-Linux CPU scheduler ziyasiza. Ungafunda ngazo ngokuningiliziwe lapha, futhi ngaphakathi kohlaka lwalesi sihloko sizoxoxa ngazo kafushane:

• SCHED_OTHER
  Ngokuzenzakalelayo, zonke izinqubo zomsebenzisi ezijwayelekile emshinini we-Linux ziyatholwa.
• SCHED_BATCH
  Idizayinelwe izinqubo ezidinga izinsiza. Uma ubeka umsebenzi kumcubungula, kwethulwa lokho okubizwa ngokuthi inhlawulo yokuvula: umsebenzi onjalo mancane amathuba okuthi uthole izinsiza zokucubungula uma okwamanje usetshenziswa umsebenzi one-SCHED_OTHER.
• SCHED_IDLE
  Inqubo yangemuva enokubaluleka okuphansi kakhulu, okuphansi kakhulu kunokuhle -19. Sisebenzisa ilabhulali yethu yomthombo ovulekile eyodwa-nio, ukuze usethe inqubomgomo edingekayo lapho uqala isiqukathi ngokushaya ucingo

one.nio.os.Proc.sched_setscheduler( pid, Proc.SCHED_IDLE )

Kodwa noma ungahleli ku-Java, into efanayo ingenziwa usebenzisa umyalo we-chrt:

chrt -i 0 $pid

Ake sifingqe wonke amazinga ethu okuzihlukanisa sibe ithebula elilodwa ukuze kucace:

Isigaba se-insulation
Isibonelo se-Alloc
Izinketho zokusebenzisa i-Docker
sched_setscheduler chrt*

Prod
cpu = 4
--cpuquota=400000 --cpuperiod=100000
SCHED_OTHER

Isigaba
Cpu = [1, *)
--cpushares=1024
SCHED_BATCH

Ingahle
Cpu= [2, *)
--cpushares=2048
SCHED_IDLE

*Uma wenza i-chrt ngaphakathi kwesiqukathi, ungase udinge ikhono le-sys_nice, ngoba ngokuzenzakalela i-Docker isusa leli khono lapho iqala isiqukathi.

Kodwa imisebenzi ayidli kuphela iprosesa, kodwa futhi ithrafikhi, ethinta ukubambezeleka komsebenzi wenethiwekhi ngisho nangaphezu kokwabiwa okungalungile kwezinsiza zeprosesa. Ngakho-ke, ngokwemvelo sifuna ukuthola isithombe esifanayo ncamashi sethrafikhi. Okusho ukuthi, lapho umsebenzi we-prod uthumela amaphakethe athile kunethiwekhi, sikhawulela isivinini esikhulu (ifomula i-alloc: lan=[*,500mbps) ), iyiphi i-prod engenza ngayo lokhu. Futhi nge-batch siqinisekisa kuphela ubuncane bokuphuma, kodwa ungakhawuleli ubuningi (ifomula i-alloc: lan=[10Mbps,*) ) Kulesi simo, ithrafikhi ye-prod kufanele ithole kuqala kunemisebenzi yeqoqo.
Lapha i-Docker ayinazo izinto zokuqala esingazisebenzisa. Kodwa kuyasiza I-Linux Traffic Control. Sikwazile ukuzuza umphumela esiwufunayo ngosizo lwesiyalo I-Hierarchical Fair Service Curve. Ngosizo lwayo, sihlukanisa izigaba ezimbili zethrafikhi: umkhiqizo obaluleke kakhulu kanye nenqwaba/ukungenzi lutho. Ngenxa yalokho, ukucushwa kwethrafikhi ephumayo kunjena:

lapha 1:0 "yimpande qdisc" yesiyalo se-hsfc; 1:1 - ikilasi lezingane le-hsfc elinomkhawulo ophelele womkhawulokudonsa ongu-8 Gbit/s, okubekwa ngaphansi kwawo amakilasi ezingane azo zonke iziqukathi; 1:2 - Ikilasi lezingane le-hsfc livamile kuyo yonke inqwaba nemisebenzi engenzi lutho enomkhawulo “oguqukayo,” okuxoxwa ngawo ngezansi. Amakilasi ezingane e-hsfc asele angamakilasi anikezelwe eziqukathi ezisetshenziswayo njengamanje ezinemikhawulo ehambisana nama-manifest azo - 450 kanye no-400 Mbit/s. Ikilasi ngalinye le-hsfc linikezwa umugqa we-qdisc fq noma i-fq_codel, kuye ngenguqulo ye-Linux kernel, ukugwema ukulahleka kwephakethe ngesikhathi sokuqhuma kwethrafikhi.

Ngokuvamile, iziyalo ze-tc zisebenza ukubeka kuqala ithrafikhi ephumayo kuphela. Kodwa sifuna ukubeka phambili ithrafikhi engenayo - phela, omunye umsebenzi wenqwaba ungakhetha kalula sonke isiteshi esingenayo, ukuthola, isibonelo, inqwaba yedatha yokufaka yemephu&enciphise. Kulokhu sisebenzisa imojula ifb, edala isixhumi esibonakalayo se-ifbX ku-interface ngayinye yenethiwekhi futhi iqondise kabusha ithrafikhi engenayo ukusuka kusixhumi esibonakalayo kuye kuthrafikhi ephumayo ku-ifbX. Ngaphezu kwalokho, ku-ifbX, zonke iziyalo ezifanayo zisebenza ukulawula ithrafikhi ephumayo, lapho ukucushwa kwe-hsfc kuzofana kakhulu:

Ngesikhathi sokuhlolwa, sithole ukuthi i-hsfc ibonisa imiphumela ehamba phambili uma ikilasi elingu-1:2 lethrafikhi yenqwaba engabalulekile/engenzi lutho ikhawulelwe emishinini emincane ukuba ibe ngaphezu komzila othile wamahhala. Uma kungenjalo, ithrafikhi engabalulekile inomthelela omkhulu ekubambeni kwemisebenzi ye-prod. i-miniond inquma inani lamanje lomkhawulokudonsa wamahhala njalo ngesekhondi, ikala ukusetshenziswa kwethrafikhi okumaphakathi kwayo yonke imisebenzi yokukhiqiza yeminion ethile futhi iyisuse kumkhawulokudonsa wesixhumi esibonakalayo senethiwekhi ngomkhawulo omncane, i.e.

Amabhendi achazwa ngokuzimela kuthrafikhi engenayo nephumayo. Futhi ngokwamanani amasha, i-miniond iphinda ilungise umkhawulo wesigaba esingabalulekile 1:2.

Ngakho-ke, sisebenzise zonke izigaba ezintathu zokuzihlukanisa: i-prod, i-batch kanye ne-idle. Lezi zigaba zinethonya kakhulu izici zokusebenza zemisebenzi. Ngakho-ke, sinqume ukubeka lesi sici phezulu kwesigaba, ukuze kuthi uma sibheka igama lomugqa wezikhundla kucace ngokushesha ukuthi sibhekene nani:

Bonke abangani bethu web и umculo ama-fronts abe esebekwa ku-hierarchy ngaphansi kwe-prod. Isibonelo, ngaphansi kwenqwaba, ake sibeke isevisi ikhathalogi yomculo, ehlanganisa ngezikhathi ezithile ikhathalogi yamathrekhi kusukela kusethi yamafayela we-mp3 alayishwe ku-Odnoklassniki. Isibonelo sesevisi engaphansi kokungenzi lutho kungaba i-transformer yomculo, okwenza izinga levolumu yomculo lijwayeleke.

Njengoba imigqa eyengeziwe isusiwe futhi, singabhala amagama ethu esevisi kangcono ngokwengeza isigaba sokuhlukanisa umsebenzi ekupheleni kwegama lesevisi eligcwele: web.front.prod, ikhathalogi.music.batch, i-transformer.music.idle.

Futhi manje, sibheka igama lenkonzo, asiqondi nje kuphela ukuthi yimuphi umsebenzi owenzayo, kodwa futhi nesigaba sayo sokuzihlukanisa, okusho ukugxeka kwayo, njll.

Konke kuhle, kodwa kuneqiniso elilodwa elibabayo. Akunakwenzeka ukuhlukanisa ngokuphelele imisebenzi esebenza emshinini owodwa.

Esikwazile ukukufeza: uma iqoqo lidla kakhulu kuphela Izinsiza ze-CPU, bese isihleli se-Linux CPU esakhelwe ngaphakathi senza umsebenzi waso kahle kakhulu, futhi cishe awukho umthelela emsebenzini wokukhiqiza. Kodwa uma lo msebenzi we-batch uqala ukusebenza ngenkuthalo ngenkumbulo, khona-ke ithonya elihlangene selivele livela. Lokhu kwenzeka ngoba umsebenzi we-prod "uhlanziwe" ezinqolobaneni zememori yephrosesa - ngenxa yalokho, ukuphuthelwa kwenqolobane kuyanda, futhi iphrosesa icubungula umsebenzi wokukhiqiza kancane kakhulu. Umsebenzi we-batch onjalo ungakhuphula ukubambezeleka kwesiqukathi sethu esijwayelekile se-prod ngo-10%.

Ukuhlukanisa ithrafikhi kunzima nakakhulu ngenxa yokuthi amakhadi enethiwekhi yesimanje anomugqa wangaphakathi wamaphakethe. Uma iphakethe elivela emsebenzini we-batch lifika lapho kuqala, khona-ke kuyoba ngowokuqala ukudluliselwa phezu kwekhebula, futhi akukho lutho olungenziwa ngakho.

Ngaphezu kwalokho, kuze kube manje sikwazile kuphela ukuxazulula inkinga yokubeka kuqala ithrafikhi ye-TCP: indlela ye-hsfc ayisebenzi ku-UDP. Futhi noma esimweni sethrafikhi ye-TCP, uma umsebenzi we-batch ukhiqiza ithrafikhi eningi, lokhu kunikeza nokukhuphuka okungaba ngu-10% ekubambezelekeni komsebenzi we-prod.

ukubekezelelana kwamaphutha

Enye yezinhloso lapho kwakhiwa ifu elilodwa kwakuwukuthuthukisa ukubekezelelana kwamaphutha kwe-Odnoklassniki. Ngakho-ke, ngokulandelayo ngithanda ukucabangela ngokuningiliziwe izimo ezingase zibe khona zokuhluleka kanye nezingozi. Ake siqale ngesimo esilula - ukwehluleka kweziqukathi.

Isiqukathi ngokwaso singahluleka ngezindlela eziningana. Lokhu kungaba uhlobo oluthile lokuhlola, iphutha noma iphutha ku-manifest, ngenxa yokuthi umsebenzi wokukhiqiza uqala ukusebenzisa izinsiza eziningi kunalokho okukhonjiswe ku-manifest. Sibe necala: umthuthukisi wenze i-algorithm eyodwa eyinkimbinkimbi, wayilungisa kabusha izikhathi eziningi, wazicabangela ngokweqile futhi wadideka kangangokuthi ekugcineni inkinga yangena ku-loop engeyona into encane kakhulu. Futhi njengoba umsebenzi we-prod ubaluleke kakhulu kunawo wonke amanye kuma-minion afanayo, waqala ukusebenzisa zonke izinsiza ezitholakalayo zeprosesa. Kulesi simo, ukuhlukaniswa, noma isilinganiso sesikhathi se-CPU, silondoloze usuku. Uma umsebenzi wabelwe isabelo, umsebenzi ngeke udle okwengeziwe. Ngakho-ke, i-batch neminye imisebenzi ye-prod eyayisebenza emshinini ofanayo ayizange ibone lutho.

Inkinga yesibili engaba khona ukuwa kwesitsha. Futhi lapha izinqubomgomo zokuqalisa kabusha zisisindisa, wonke umuntu uyazazi, i-Docker ngokwayo yenza umsebenzi omuhle. Cishe yonke imisebenzi ye-prod inenqubomgomo ehlala iqala kabusha. Kwesinye isikhathi sisebenzisa i-on_failure kwimisebenzi yenqwaba noma ukulungisa amaphutha iziqukathi ze-prod.

Yini ongayenza uma i-minion yonke ingatholakali?

Ngokusobala, sebenzisa isiqukathi komunye umshini. Ingxenye ethokozisayo lapha ukuthi kwenzekani ekhelini (ama) IP elinikezwe esitsheni.

Singanikeza iziqukathi amakheli e-IP afanayo njengemishini emincane lapho lezi ziqukathi zisebenza khona. Khona-ke, lapho isitsha sethulwa komunye umshini, ikheli laso le-IP liyashintsha, futhi wonke amaklayenti kufanele aqonde ukuthi isitsha sithuthile, futhi manje adinga ukuya ekhelini elihlukile, elidinga isevisi ehlukile Yokutholwa Kwesevisi.

I-Service Discovery ilungile. Kunezixazululo eziningi emakethe zamazinga ahlukahlukene okubekezelela amaphutha ekuhleleni ukubhaliswa kwesevisi. Ngokuvamile izixazululo ezinjalo zisebenzisa i-logic balancer logic, gcina ukucushwa okwengeziwe ngendlela yesitoreji se-KV, njll.
Kodwa-ke, singathanda ukugwema isidingo sokuqalisa ukubhalisa okuhlukile, ngoba lokhu kungasho ukwethula uhlelo olubucayi olusetshenziswa yizo zonke izinsizakalo ekukhiqizeni. Lokhu kusho ukuthi leli yiphuzu elingase lehluleke, futhi udinga ukukhetha noma ukuthuthukisa isisombululo esibekezelela amaphutha, okusobala ukuthi sinzima kakhulu, sidla isikhathi futhi siyabiza.

Nokunye okushiyekayo okukhulu: ukuze ingqalasizinda yethu endala isebenze nentsha, kuzodingeka ukuthi sibhale kabusha yonke imisebenzi ukuze sisebenzise uhlobo oluthile lwesistimu Yokutholwa Kwesevisi. Kunomsebenzi omningi, futhi kwezinye izindawo cishe akunakwenzeka uma kuziwa kumadivayisi asezingeni eliphansi asebenza ezingeni le-OS kernel noma ngokuqondile ngehadiwe. Ukuqaliswa kwalokhu kusebenza kusetshenziswa amaphethini esixazululo amisiwe, njenge imoto eseceleni kuzosho kwezinye izindawo umthwalo owengeziwe, kwezinye - inkimbinkimbi yokusebenza kanye nezimo zokuhluleka ezengeziwe. Besingafuni ukwenza izinto zibe nzima, ngakho-ke sinqume ukwenza ukusebenzisa i-Service Discovery ngokuzithandela.

Efini elilodwa, i-IP ilandela isiqukathi, okungukuthi isenzakalo ngasinye somsebenzi sinekheli laso le-IP. Leli kheli “limile”: linikezwa isikhathi ngasinye lapho isevisi ithunyelwa okokuqala emafini. Uma isevisi ibe nenani elihlukile lezenzakalo ngesikhathi sokuphila kwayo, lapho-ke ekugcineni izonikezwa amakheli e-IP amaningi njengoba bekunezimo eziphezulu.

Kamuva, lawa makheli awashintshi: abelwa kanye futhi aqhubeke ekhona kuyo yonke impilo yenkonzo ekukhiqizeni. Amakheli e-IP alandela iziqukathi kunethiwekhi yonkana. Uma isiqukathi sidluliselwa kwenye i-minion, khona-ke ikheli lizoyilandela.

Ngakho, ukuhlelwa kwegama lesevisi ohlwini lwayo lwamakheli e-IP kushintsha kakhulu. Uma ubheka futhi amagama ezimo zesevisi esizishilo ekuqaleni kwesihloko (1.ok-web.group1.web.front.prod, 2.ok-web.group1.web.front.prod, …), sizoqaphela ukuthi afana nama-FQDN asetshenziswa ku-DNS. Kulungile, ukumepha amagama ezenzakalo zesevisi kumakheli abo e-IP, sisebenzisa iphrothokholi ye-DNS. Ngaphezu kwalokho, le DNS ibuyisela wonke amakheli e-IP agciniwe azo zonke iziqukathi - zombili zisebenza futhi zamiswa (ake sithi kusetshenziswa izifaniso ezintathu, futhi sinamakheli amahlanu agcinwe lapho - wonke amahlanu azobuyiswa). Amaklayenti, ngemva kokuthola lolu lwazi, azozama ukusungula uxhumano nazo zonke izifaniso ezinhlanu - futhi ngaleyo ndlela anqume ukuthi yiziphi ezisebenzayo. Le nketho yokunquma ukutholakala ithembeke kakhulu; ayibandakanyi i-DNS noma i-Service Discovery, okusho ukuthi azikho izinkinga ezinzima ukuzixazulula ekuqinisekiseni ukuhambisana kolwazi nokubekezelela amaphutha kwalezi zinhlelo. Ngaphezu kwalokho, ezinkonzweni ezibucayi lapho ukusebenza kwayo yonke ingosi kuncike khona, asikwazi nhlobo ukusebenzisa i-DNS, kodwa mane nje sifake amakheli e-IP ekucushweni.

Ukusebenzisa ukudlulisa okunjalo kwe-IP ngemuva kweziqukathi kungase kungabi into encane - futhi sizobheka ukuthi kusebenza kanjani ngesibonelo esilandelayo:

Ake sithi inkosi yefu elilodwa inika umyalo ku-minion M1 ukuthi isebenze 1.ok-web.group1.web.front.prod ngekheli 1.1.1.1. Isebenza ku-minion INYONI, ekhangisa leli kheli kumaseva akhethekile isibonisi somzila. Laba bakamuva baneseshini ye-BGP ne-hardware yenethiwekhi, lapho umzila wekheli 1.1.1.1 ku-M1 uhunyushwa khona. Amaphakethe emizila ye-M1 ngaphakathi kwesiqukathi esebenzisa i-Linux. Kunamaseva amathathu okubonisa umzila, njengoba lokhu kuyingxenye ebaluleke kakhulu yengqalasizinda yefu elilodwa - ngaphandle kwabo, inethiwekhi efwini elilodwa ngeke isebenze. Siwabeka kuma-racks ahlukene, uma kungenzeka abekwe emakamelweni ahlukene esikhungo sedatha, ukunciphisa amathuba okuba bobathathu bahluleke ngesikhathi esisodwa.

Manje ake sicabange ukuthi ukuxhumana phakathi kwe-one-cloud master kanye ne-M1 minion kulahlekile. Inkosi yefu elilodwa manje izothatha isinyathelo ngokucabanga ukuthi i-M1 yehlulekile ngokuphelele. Okusho ukuthi, izonikeza umyalo ku-M2 minion ukuthi yethule web.group1.web.front.prod ngekheli elifanayo 1.1.1.1. Manje sinemizila emibili engqubuzanayo kunethiwekhi ye-1.1.1.1: ku-M1 naku-M2. Ukuze sixazulule izingxabano ezinjalo, sisebenzisa i-Multi Exit Discriminator, eshiwo esimemezelweni se-BGP. Lena inombolo ekhombisa isisindo somzila okhangiswayo. Phakathi kwemizila engqubuzanayo, umzila onenani eliphansi le-MED uzokhethwa. Inkosi yefu elilodwa isekela i-MED njengengxenye ebalulekile yamakheli e-IP esiqukathi. Ngokokuqala ngqa, ikheli libhalwa nge-MED = 1 enkulu ngokwanele. Esimweni sokudluliselwa kwesitsha esiphuthumayo esinjalo, inkosi inciphisa i-MED, futhi i-M000 izovele ithole umyalo wokukhangisa ikheli 000 nge-MED = 2. Isigameko esigijima ku-M1.1.1.1 sizohlala sikhona kulokhu akukho ukuxhumana, futhi isiphetho sakhe asisithakazelisi kuze kube yilapho sekubuyelwa ukuxhumana nomphathi, lapho ezomiswa njengethathi elidala.

izingozi

Wonke amasistimu okuphatha isikhungo sedatha ahlala ephatha ukwehluleka okuncane ngokwamukelekayo. Ukuchichima kwesitsha kuyinto evamile cishe yonke indawo.

Ake sibheke ukuthi sisisingatha kanjani isimo esiphuthumayo, njengokuhluleka kwamandla egumbini elilodwa noma amaningi esikhungo sedatha.

Isho ukuthini ingozi ngohlelo lokuphatha isikhungo sedatha? Okokuqala, lokhu ukwehluleka okukhulu kwesikhathi esisodwa kwemishini eminingi, futhi uhlelo lokulawula ludinga ukuthutha iziqukathi eziningi ngesikhathi esisodwa. Kodwa uma inhlekelele inkulu kakhulu, khona-ke kungenzeka ukuthi yonke imisebenzi ayikwazi ukuphinda inikezwe abanye abancane, ngoba umthamo wezinsiza zesikhungo sedatha wehla ngaphansi kwe-100% yomthwalo.

Ngokuvamile izingozi zihambisana nokwehluleka kwe-control layer. Lokhu kungenzeka ngenxa yokwehluleka kwemishini yayo, kodwa kaningi ngenxa yokuthi izingozi azihlolwa, futhi ungqimba lokulawula ngokwalo luwela ngenxa yomthwalo owandayo.

Yini ongayenza ngakho konke lokhu?

Ukufuduka kwabantu abaningi kusho ukuthi kunenani elikhulu lemisebenzi, ukufuduka, kanye nokusatshalaliswa okwenzeka engqalasizinda. Ukuthutha ngakunye kungase kuthathe isikhathi esidingekayo ukuletha nokuthulula izithombe zeziqukathi kwabancane, ukuqalisa nokuqalisa iziqukathi, njll. Ngakho-ke, kuyafiseleka ukuthi imisebenzi ebaluleke kakhulu yethulwe ngaphambi kwengabalulekile kangako.

Ake sibheke futhi isigaba samasevisi esiwajwayele futhi sizame ukunquma ukuthi yimiphi imisebenzi esifuna ukuyenza kuqala.

Yebo, lezi yizinqubo ezibandakanyeka ngokuqondile ekucubunguleni izicelo zabasebenzisi, okungukuthi i-prod. Sikhombisa lokhu nge ukubeka kuqala — inombolo enganikezwa kulayini. Uma ulayini unokubaluleka okuphezulu, izinsizakalo zawo zibekwa kuqala.

Emkhiqizweni sibeka izinto eziza kuqala kakhulu, 0; ku-batch - kancane kancane, i-100; kokungenzi lutho - ngisho nangaphansi, 200. Okubalulekile kusetshenziswe ngokohlelo. Yonke imisebenzi engaphansi kwesigaba izoba nokuhamba phambili okuhambisanayo. Uma sifuna ama-cache angaphakathi kwe-prod yethulwe ngaphambi kwezindawo ezingaphambili, sizobe sesinika okubalulekile kunqolobane = 0 kanye nakuma-subqueues angaphambili = 1. Uma, ngokwesibonelo, sifuna ukuthi ingosi eyinhloko yethulwe kusukela kumaphambili kuqala, futhi ngaphambili komculo kuphela. ke, khona-ke singabeka okubalulekile okuphansi kokugcina - 10.

Inkinga elandelayo ukushoda kwezinsiza. Ngakho-ke, inani elikhulu lemishini, wonke amahholo esikhungo sedatha, ahlulekile, futhi siphinde sethula izinsiza eziningi kangangokuthi manje azikho izinsiza ezanele zawo wonke umuntu. Udinga ukunquma ukuthi yimiphi imisebenzi ongayidela ukuze ugcine izinsiza ezibalulekile zisebenza.

Ngokungafani nokubekwa kuqala, asikwazi ukudela yonke imisebenzi yeqoqo ngokungakhethi; eminye yayo ibalulekile ekusebenzeni kwephothali. Ngakho-ke, siye saqokomisa ngokwehlukana ukukhululwa kuqala imisebenzi. Uma ubekwe, umsebenzi obaluleke kakhulu ungalungiselela, okungukuthi, ukumisa, umsebenzi ongabalulekile kakhulu uma bengasekho abancane abamahhala. Kulokhu, umsebenzi onokubaluleka okuphansi cishe uzohlala ungabekiwe, okungukuthi ngeke kusaba khona i-minion efanelekile enezinsiza ezanele zamahhala.

Esigabeni sethu sokulandelana, kulula kakhulu ukucacisa okubaluleke kakhulu ngaphambi kokuthi imisebenzi ye-prod ne-batch iqale noma imise imisebenzi engenzi lutho, kodwa hhayi enye nenye, ngokucacisa okubalulekile kokungenzi lutho okulingana no-200. Njengasendabeni yokubeka kuqala, nathi singasebenzisa i-hierarchy yethu ukuze sichaze imithetho eyinkimbinkimbi. Isibonelo, asibonise ukuthi sidela umsebenzi womculo uma singenazo izinsiza ezanele zengosi yewebhu eyinhloko, sibeka okubalulekile kwamanodi ahambisanayo abe ngaphansi: 10.

Zonke izingozi ze-DC

Kungani sonke isikhungo sedatha singase singaphumeleli? Isici. Bekuyiposi elihle isiphepho sithinte umsebenzi wesikhungo sedatha. Izakhi zingabhekwa njengabantu abangenamakhaya abake bashisa ama-optics ezindaweni eziningi, futhi isikhungo sedatha salahlekelwa ngokuphelele ukuxhumana namanye amasayithi. Isizathu sokwehluleka kungase futhi kube isici somuntu: opharetha uzokhipha umyalo onjalo ukuthi yonke isikhungo sedatha sizowa. Lokhu kungenzeka ngenxa yesiphazamisi esikhulu. Ngokuvamile, ukuwa kwezikhungo zedatha akuvamile. Lokhu kwenzeka kithi kanye njalo ezinyangeni ezimbalwa.

Futhi yilokhu esikwenzayo ukuvimbela noma ubani ukuthi abhale ku-tweet #alive.

Isu lokuqala wukuzihlukanisa. Isenzakalo ngasinye sefu elilodwa sisodwa futhi singaphatha imishini esikhungweni sedatha esisodwa kuphela. Okusho ukuthi, ukulahlekelwa kwefu ngenxa yeziphazamisi noma imiyalo engalungile yomqhubi ukulahlekelwa isikhungo sedatha esisodwa kuphela. Sikulungele lokhu: sinenqubomgomo yokungaphindeki umsebenzi lapho izifaniso zohlelo lokusebenza nedatha zitholakala kuzo zonke izikhungo zedatha. Sisebenzisa imininingo egciniwe yokubekezelela amaphutha futhi sihlole ukwehluleka ngezikhathi ezithile.
Njengoba namuhla sinezikhungo zedatha ezine, lokho kusho izimo ezine ezihlukene, ezihlukene ngokuphelele zefu elilodwa.

Le ndlela ayivikeli kuphela ekuhlulekeni ngokomzimba, kodwa futhi ingavikela ephutheni lomsebenzisi.

Yini enye engenziwa ngesici somuntu? Lapho osebenzisa ifu enikeza ifu umyalo othile ongajwayelekile noma ongase ube yingozi, angase acelwe kungazelelwe ukuthi axazulule inkinga encane ukuze abone ukuthi ubecabanga kahle kangakanani. Isibonelo, uma lokhu kuwuhlobo oluthile lokuma kwenqwaba yamakhophi amaningi noma umyalo nje ongajwayelekile - ukunciphisa inani lezifaniso noma ukushintsha igama lesithombe, futhi hhayi nje inombolo yenguqulo ku-manifest entsha.

Imiphumela

Izici ezihlukile zefu elilodwa:

• Isikimu sokuqamba esinezinga eliphezulu nesibonwayo sezinsizakalo neziqukathi, okuvumela ukuthi uthole ngokushesha okukhulu ukuthi uyini umsebenzi, ukuthi uhlobene kanjani nokuthi usebenza kanjani nokuthi ubani obhekene nawo.
• Sisebenzisa yethu indlela yokuhlanganisa i-prod- ne-batch-imisebenzi kwabancane ukuthuthukisa ukusebenza kahle kokwabelana komshini. Esikhundleni se-cpuset sisebenzisa izilinganiso ze-CPU, amasheya, izinqubomgomo ze-CPU scheduler kanye ne-Linux QoS.
• Bekungeke kwenzeke ukuhlukanisa ngokuphelele iziqukathi ezisebenza emshinini ofanayo, kodwa umthelela wazo ofanayo uhlala ngaphakathi kwama-20%.
• Ukuhlela izinsiza ngohlelo kusiza ngokuthola inhlekelele okuzenzakalelayo kusetshenziswa ukubeka kanye nokulungiselela kuqala.

FAQ

Kungani singazange sithathe isixazululo esenziwe ngomumo?

• Izigaba ezihlukene zokuhlukaniswa komsebenzi zidinga ukucabanga okuhlukile uma kubekwe kuma-minion. Uma imisebenzi ye-prod ingase ibekwe ngokugcina izinsiza nje, kuzomele kwenziwe inqwaba nemisebenzi engenzi lutho, kulandelelwe ukusetshenziswa kwangempela kwezisetshenziswa emishinini emincane.
• Isidingo sokucabangela izinsiza ezisetshenziswa imisebenzi, njenge:
  • umkhawulokudonsa wenethiwekhi;
  • izinhlobo kanye "nama-spindle" amadiski.
• Isidingo sokubonisa izinto eziza kuqala ezinsizeni ngesikhathi sokuphendula izimo eziphuthumayo, amalungelo kanye nezilinganiso zemiyalo yezinsiza, ezixazululwa kusetshenziswa imigqa ye-hierarchical efwini elilodwa.
• Isidingo sokuthi kuqanjwe amakhonteyna amagama abantu ukuze kuncishiswe isikhathi sokuphendula ezingozini nasezigamekweni
• Ukungenzeki kokusetshenziswa okusabalele ngesikhathi esisodwa kwe-Service Discovery; isidingo sokuhlalisana isikhathi eside nemisebenzi esingathwa kubasingathi behadiwe - into exazululwa ngamakheli e-IP "amile" alandela iziqukathi, futhi, ngenxa yalokho, isidingo sokuhlanganiswa okuyingqayizivele nengqalasizinda yenethiwekhi enkulu.

Yonke le misebenzi ingadinga ukuguqulwa okubalulekile kwezixazululo ezikhona ukuze zisifanele, futhi, ngemva kokuhlola inani lomsebenzi, sabona ukuthi singase sithuthukise isixazululo sethu cishe ngezindleko ezifanayo zabasebenzi. Kodwa isixazululo sakho sizoba lula kakhulu ukusebenzisa nokuthuthukisa - asiqukethe izifinyezo ezingadingekile ezisekela ukusebenza esingakudingi.

Kulabo abafunde imigqa yokugcina, ngiyabonga ngesineke sakho nokunaka!

Source: www.habr.com