Tupperware: Facebook's Kubernetes muurayi?

Inoshanda uye yakavimbika manejimendi emasumbu pane chero chiyero neTupperware

Nhasi on Systems@Scale musangano isu takaunza Tupperware, yedu cluster manejimendi system inoronga midziyo mumamiriyoni emaseva anomhanya anenge ese masevhisi edu. Takatanga kutumira Tupperware muna 2011, uye kubva ipapo zvivakwa zvedu zvakakura kubva 1 data centre kuzere 15 geo-yakagoverwa data nzvimbo. Nguva yese iyi, Tupperware haina kumira uye yakagadzira nesu. Isu tinokuratidza kuti Tupperware inopa sei yekutanga-kirasi manejimendi manejimendi, kusanganisira yakaringana tsigiro yemasevhisi ane matunhu, imwe chete control panel yeese data data, uye kugona kugovera huwandu pakati pemasevhisi munguva chaiyo. Tichagoverawo zvidzidzo zvatakadzidza sezvo zvivakwa zvedu zvinoshanduka.

Tupperware inoita mabasa akasiyana. Vagadziri vekushandisa vanoishandisa kuendesa uye kubata maapplication. Iyo inorongedza iyo kodhi yekushandisa uye zvinoenderana mumufananidzo uye inoendesa kune maseva semidziyo. Containers inopa yekuzviparadzanisa pakati pezvishandiso pane imwechete server kuitira kuti vanogadzira vabate neiyo application logic uye havafanirwe kunetseka nekutsvaga maseva kana kugadzirisa zvigadziriso. Tupperware inotarisisawo kuita kwesevha, uye kana ikawana kukundikana, inotamisa midziyo kubva kune inonetsa server.

Kugona kuronga mainjiniya anoshandisa Tupperware kugovera server simba kuzvikwata zvinoenderana nebhajeti uye zvipingaidzo. Ivo zvakare vanoshandisa iyo kuvandudza sevha kushandiswa. Vashandisi venzvimbo yedata vanotendeukira kuTupperware kuti vagovane zvakanaka midziyo munzvimbo dzese dzedata uye kumisa kana kufambisa midziyo panguva yekugadzirisa. Nekuda kweizvi, kuchengetedza maseva, network uye zvishandiso zvinoda kushoma kupindira kwevanhu.

Tupperware architecture

Tupperware PRN architecture ndeimwe yematunhu edu data data. Dunhu rine akati wandei data center zvivakwa (PRN1 uye PRN2) iri padyo. Isu tinoronga kugadzira imwe control panel inozobata ese maseva mudunhu rimwe.

Vagadziri vekushandisa vanoendesa masevhisi nenzira yeTupperware mabasa. Basa rine akawanda midziyo, uye iwo ese anowanzo mhanyisa imwechete application kodhi.

Tupperware ine basa rekupa midziyo uye kutonga hupenyu hwavo. Inosanganisira zvinhu zvakati wandei:

• Iyo Tupperware yekumberi inopa maAPIs emushandisi interface, CLI, uye mamwe otomatiki maturusi aunokwanisa kudyidzana neTupperware. Vanovanza chimiro chemukati cheTupperware varidzi vebasa.
• Tupperware Scheduler inzvimbo yekudzora ine mutoro wekugadzirisa mudziyo uye hupenyu hwebasa. Inoiswa pamatunhu nepasirese mazinga, uko mugadziri wedunhu anodzora maseva mudunhu rimwe uye mugadziri wepasi rose anotonga maseva kubva kumatunhu akasiyana. Iyo scheduler yakakamurwa kuita shards, uye shard imwe neimwe inobata seti yemabasa.
• Tupperware's Scheduler Proxy inovanza iyo yemukati sharding uye inopa yakanakira imwe pane yegirazi kune vashandisi veTupperware.
• Iyo Tupperware allocator inopa midziyo kumaseva. Iyo scheduler inobata kumira, kutanga, kugadzirisa, uye kukundikana kwemidziyo. Parizvino, mugoveri mumwe anogona kubata dunhu rese pasina kupatsanura kuita shards. (Cherechedza mutsauko wemazwi. Semuyenzaniso, muronga muTupperware anoenderana nepanera rekudzora mukati Kubernetes, uye Tupperware allocator inonzi scheduler muKubernetes.)
• Iyo resource broker inochengetedza kunobva chokwadi kune server uye sevhisi zviitiko. Isu tinomhanyisa imwe sosi bhuroka kune yega data data, uye inochengeta ruzivo rwese nezvesevha mune iyo data data. Iyo resource broker uye iyo simba manejimendi sisitimu, kana zviwanikwa zvekupa sisitimu, zvine simba sarudza kuti ndeipi scheduler kuendesa inodzora iyo server. Iyo yehutano yekutarisa sevhisi inotarisisa maseva uye inochengetedza data nezvehutano hwavo mune resource broker. Kana sevha iine matambudziko kana inoda kugadziriswa, mutengesi wezviwanikwa anoudza anogovera uye anoronga kuti amise midziyo kana kuiendesa kune mamwe maseva.
• Iyo Tupperware Agent idhimoni rinomhanya pane yega yega server rinobata kupihwa uye kubviswa kwemidziyo. Zvishandiso zvinomhanya mukati memudziyo, izvo zvinovapa kumwe kuzviparadzanisa uye kuberekana. On gore rapfuura Systems @Scale musangano Isu takatotsanangura magadzirirwo ega ega Tupperware midziyo inogadzirwa uchishandisa mifananidzo, btrfs, cgroupv2 uye systemd.

Mamiriro akasiyana eTupperware

Tupperware yakafanana munzira dzakawanda kune mamwe cluster manejimendi masisitimu akadai Kubernetes uye Mesos, asi panewo kusiyana:

• Yakavakirwa-mukati tsigiro yehurumende masevhisi.
• Iyo imwe chete control panel yemaseva ari munzvimbo dzakasiyana dzedata kuti aite otomatiki kuendeswa kwemidziyo zvichienderana nechinangwa, kubviswa kwemasumbu uye kugadzirisa.
• Yakajeka kupatsanurwa kwepaneru yekudzora ye zooming.
• Elastic computing inobvumidza iwe kugovera simba pakati pesevhisi munguva chaiyo.

Isu takagadzira aya anotonhorera maficha kuti titsigire akasiyana-siyana asina nyika uye ane hunyanzvi maapplication mukati mehombe yepasirese yakagovaniswa server ngarava.

Yakavakirwa-mukati tsigiro yehurumende masevhisi.

Tupperware inoshandisa akasiyana masevhisi akaomesesa anochengetedza anoramba achigadzirwa data yeFacebook, Instagram, Messenger uye WhatsApp. Izvi zvinogona kunge zviri zvitoro zvakakura zve-key-value pairs (e.g. ZippyDB) uye yekutarisa dhata repositori (semuenzaniso, ODS Gorilla и Scuba) Kuchengetedza masevhisi ehurumende hakusi nyore, nekuti sisitimu inofanirwa kuve nechokwadi chekuti kupihwa kwemidziyo kunogona kumirisana nekukanganisa kwakakura, kusanganisira kudzimwa kwetiweki kana kudzimwa kwemagetsi. Uye nepo matekiniki akajairwa, akadai sekugovera midziyo munzvimbo dzese dzakakanganisa, anoshanda nemasevhisi asina matunhu, masevhisi ane matunhu anoda imwe rutsigiro.

Semuyenzaniso, kana server yakundikana ichiita kuti imwe dhatabhesi replica ive isingawanikwe, unofanirwa kugonesa otomatiki kugadzirisa iyo inovandudza macores pamaseva makumi mashanu kubva padziva regumi? Zvinoenderana nemamiriro ezvinhu. Kana imwe yeaya makumi mashanu maseva ane imwe replica yeiyo dhatabhesi, zviri nani kumirira uye usarasikirwe 50 replicas kamwechete. Kuti tiite zvine simba kuita sarudzo pamusoro pekugadzirisa masisitimu uye kuita, isu tinoda ruzivo nezve yemukati dhata kudzokororwa uye yekuisa logic yesevhisi yega yega.

Iyo TaskControl interface inobvumira masevhisi ane hunyanzvi kukanganisa sarudzo dzinokanganisa kuwanikwa kwedata. Uchishandisa iyi interface, mugadziri anozivisa ekunze maapplication nezve mashandiro emudziyo (kutangazve, kugadzirisa, kutama, kugadzirisa). Sevhisi ine hunyanzvi inoshandisa controller inoudza Tupperware kana zvakachengeteka kuita oparesheni yega yega, uye maoparesheni aya anogona kuchinjika kana kunonoka kwenguva pfupi. Mumuenzaniso uri pamusoro, mutongi wedhatabhesi anogona kuudza Tupperware kuti agadzirise makumi mana nemapfumbamwe emaseva makumi mashanu, asi siya imwe sevha (X) yega ikozvino. Nekuda kweizvozvo, kana iyo kernel yekuvandudza nguva ikapfuura uye dhatabhesi ichiri kutadza kudzoreredza ine dambudziko replica, Tupperware icharamba ichivandudza iyo X server.

Mazhinji masevhisi ane hunyanzvi muTupperware anoshandisa TaskControl kwete zvakananga, asi kuburikidza neShardManager, chikuva chakajairika chekugadzira masevhisi ane hunyanzvi paFacebook. NeTupperware, vanogadzira vanogona kutsanangura chinangwa chavo chekuti midziyo inofanira kugovaniswa sei munzvimbo dzese dzedata. NeShardManager, vanogadzira vanotsanangura chinangwa chavo chekuti data shards inofanira kugovaniswa sei mumidziyo. ShardManager inoziva nezvekuiswa kwedata uye kudzokororwa kwemashandisirwo ayo uye inotaurirana neTupperware kuburikidza neTaskControl interface kuronga mashandiro emudziyo pasina kubatanidzwa kwekushandisa. Kubatanidzwa uku kunorerutsa zvikuru manejimendi emhando masevhisi, asi TaskControl inokwanisa zvimwe. Semuenzaniso, yedu yakakura yewebhu tier haina mamiriro uye inoshandisa TaskControl kugadzirisa zvine simba chiyero chezvigadziriso kumidziyo. Pakupedzisira iyo webhu tier inokwanisa kukurumidza kupedzisa akawanda kuburitswa kwesoftware pazuva pasina kukanganisa kuwanikwa.

Kugadzirisa maseva munzvimbo dze data

Tupperware payakatanga kuvhurwa muna 2011, sevha yega yega yaitungamirwa neakasiyana scheduler. Kare ikako, Facebook cluster yaive boka rema server racks akabatana kune imwe network switch, uye data center yaigara masumbu akati wandei. Iye anoronga aingokwanisa kubata maseva muboka rimwe chete, zvichireva kuti basa harina kupararira mumasumbu akawanda. Zvivakwa zvedu zvakakura, takawedzera kunyora masumbu. Sezvo Tupperware yaisakwanisa kufambisa basa kubva kuboka rakabviswa kuenda kune mamwe masumbu pasina shanduko, zvaida kushanda nesimba uye kurongeka kwakangwarira pakati pevagadziri vezvishandiso nevashandisi venzvimbo yedata. Iyi nzira yakakonzera kutambisa zviwanikwa apo maseva aive asina basa kwemwedzi yakawanda nekuda kwemaitiro ekubvisa.

Isu takagadzira resource broker kugadzirisa iyo cluster decommissioning dambudziko uye kuronga mamwe marudzi ekugadzirisa mabasa. Iyo resource broker inochengeta ruzivo rwese rwemuviri rwakabatana nesevha uye zvine simba inosarudza kuti ndeipi scheduler inodzora server yega yega. Dynamically kubatanidza maseva kune vanoronga inobvumira iyo scheduler kubata maseva munzvimbo dzakasiyana dze data. Sezvo basa reTupperware risisina kuganhurirwa kune rimwe boka, vashandisi veTupperware vanogona kutsanangura kuti midziyo inofanira kugovaniswa sei munzvimbo dzakakanganisa. Semuenzaniso, mugadziri anogona kuzivisa chinangwa chake (ichiti: "mhanya basa rangu pane 2 kukanganisa domains mudunhu rePRN") pasina kutsanangura nzvimbo dzekuwanikwa. Tupperware pachayo inowana maseva akakodzera kuita ichi chinangwa, kunyangwe iyo cluster kana sevhisi yakabviswa.

Scalable kutsigira pasi rose system

Nhoroondo, zvivakwa zvedu zvakakamurwa kuita mazana emadziva akatsaurirwa maseva ezvikwata zvega. Nekuda kwekutsemuka uye kushaikwa kwezviyero, isu taive nemitengo yakakwira yekushandisa, uye maseva asina basa aive akaoma kushandisa zvakare. Pamusangano wegore rapfuura Systems @Scale takaratidza zvivakwa sevhisi (IaaS), iyo inofanirwa kubatanidza zvivakwa zvedu kuita imwe hombe server park. Asi imwe server park ine matambudziko ayo. Inofanira kusangana nezvimwe zvinodiwa:

• Scalability. Zvivakwa zvedu zvakakura sezvo takawedzera nzvimbo dzedata mudunhu rega rega. Masevha ave madiki uye ane simba rakawanda, saka kune akawanda akawanda munzvimbo imwe neimwe. Nekuda kweizvozvo, mugadziri mumwe chete padunhu haakwanise kubata huwandu hwemidziyo inogona kumhanyisa pamazana ezviuru zvemaseva mudunhu rega rega.
• Kuvimbika Kunyangwe kana mugadziri achigona kukwidziridzwa zvakanyanya, chiyero chikuru chemugadziri zvinoreva kuti kune njodzi huru yekukanganisa uye dunhu rese remidziyo rinogona kusatarisika.
• Kushivirira mhosva. Muchiitiko chekutadza kukuru kwezvivakwa (somuenzaniso, maseva anomhanyisa scheduler anotadza nekuda kwekutadza kwetiweki kana kudzimwa kwemagetsi), mhedzisiro yakaipa inofanirwa kukanganisa chikamu chemaseva munharaunda.
• Kusununguka kwekushandisa. Zvinogona kuita sekuti iwe unofanirwa kumhanyisa akati wandei akazvimirira ekuronga dunhu rimwe. Asi kubva pakuona kuri nyore, imwe poindi yekupinda mudziva redunhu rakagovaniswa inoita kuti zvive nyore kubata masimba nemabasa.

Isu takakamura scheduler kuita shards kugadzirisa matambudziko ekuchengetedza dziva rakakura rakagovaniswa. Imwe neimwe scheduler shard inobata yayo yega seti yemabasa mudunhu, uye izvi zvinoderedza njodzi ine chekuita neanoronga. Sezvo dziva rakagovaniswa rinokura, tinogona kuwedzera mamwe mashedhi ekuronga. Kune vashandisi veTupperware, shards uye scheduler proxies inoita senge imwe control panel. Ivo havafanirwe kushanda neboka remashadhi anoronga mabasa. Scheduler shards zvakasiyana zvakanyanya kubva kumasumbu ekuronga atakamboshandisa, apo iyo control panel yakagovaniswa pasina kupatsanura zvakagovaniswa dziva remaseva zvinoenderana netiweki topology.

Kuvandudza Kushanda Kwekushandisa neElastic Computing

Iyo yakakura zvivakwa zvedu, zvinonyanya kukosha kushandisa maseva edu nemazvo kukwiridzira mitengo yezvivakwa uye kuderedza mutoro. Pane nzira mbiri dzekuwedzera kushanda kwesevha kushandiswa:

• Elastic computing - kudzikisira masevhisi epamhepo panguva yerunyararo uye shandisa maseva akasunungurwa ekuremerwa kwemabasa asiri pamhepo, sekudzidza muchina uye MepuDrusa mabasa.
• Kuwedzeredza - Isa masevhisi epamhepo uye basa rebatch pamasevha mamwe chete kuitira kuti batch basa rifambe zvakadzika.

Iyo bhodhoro munzvimbo dzedu dze data ndeye kushandiswa kwesimba. Naizvozvo, isu tinosarudza madiki, ane simba-anoshanda maseva ayo pamwe chete anopa akawanda ekugadzirisa simba. Nehurombo, pamaseva madiki ane diki CPU uye ndangariro, kuwandisa hakushande. Ehe, isu tinogona kuisa akati wandei masevhisi madiki pane imwe diki-inoshanda sevha inodya zvishoma processor zviwanikwa uye ndangariro, asi makuru masevhisi achave nekuita kwakaderera mumamiriro ezvinhu aya. Naizvozvo, isu tinopa zano vanogadzira masevhisi edu makuru kuti vaagadzirise kuitira kuti vashandise maseva ese.


Chaizvoizvo, isu tinovandudza mashandisiro ekushandisa tichishandisa elastic computing. Mazhinji emasevhisi edu makuru, akadai seNhau Feed, Messaging chimiro, uye yekumberi-yekupedzisira webhu tier, inosiyana zvichienderana nenguva yezuva. Isu tinodzikisira nemaune masevhisi epamhepo panguva dzakanyarara uye tinoshandisa maseva akasunungurwa ekuremerwa kwebasa pasina Indaneti, sekudzidza kwemichina uye MapReduce mabasa.

Isu tinoziva kubva pane zvakaitika kuti zvakanakisa kupa maseva ese semayuniti e elasticity nekuti masevhisi makuru ese ari maviri makuru ekupa uye vatengi vakuru ve elasticity, uye akagadziridzwa kuti ashandise ese maseva. Kana sevha yaburitswa kubva pa online sevhisi panguva dzakanyarara, iyo sosi yebhuroka inorenda sevha kune inoronga kuti iite basa risingaite pairi. Kana iyo yepamhepo sevhisi ikasangana nepamusoro-soro, mutengesi wezvishandiso anokurumidza kuyeuka sevha yakakweretwa uye, pamwe chete neanoronga, anoidzosera kune iyo online sevhisi.

Zvidzidzo uye zvirongwa zvenguva yemberi

Kwemakore masere apfuura, tanga tichigadzira Tupperware kuti tienderane nekukura nekukurumidza kweFacebook. Isu tinogovera zvatakadzidza uye tinovimba zvichabatsira vamwe kubata nekukurumidza zvivakwa:

• Gadzirisa chinongedzo chinochinjika pakati pepaneru yekudzora uye maseva ayo inobata. Uku kuchinjika kunobvumira iyo control panel kuti itore maseva munzvimbo dzakasiyana dze data, inobatsira otomatiki kumisa uye kugadzirisa masumbu, uye inogonesa simba rekugovera simba uchishandisa elastic komputa.
• Iine imwechete control panel mudunhu, zvinova nyore kushanda nemabasa uye zviri nyore kubata hombe yakagovaniswa server ngarava. Ziva kuti iyo control panel inochengetedza imwe poindi yekupinda, kunyangwe chimiro chayo chemukati chakaparadzaniswa nekuda kwezvikonzero zvekuyera kana kukanganisa kushivirira.
• Uchishandisa plugin modhi, iyo control panel inogona kuzivisa ekunze mashandisirwo ezvinouya zvemidziyo mashandiro. Uyezve, masevhisi ane hunyanzvi anogona kushandisa iyo plugin interface kugadzirisa maneja manejimendi. Iine iyi plugin modhi, iyo control panel inopa nyore apo ichishanda nemasevhisi akawanda akasiyana ehurumende.
• Isu tinotenda kuti elastic komputa, kwatinobvisa masevha ese kubva kune vanopa masevhisi ebasa rebatch, kudzidza muchina, uye mamwe masevhisi asiri-echimbichimbi, ndiyo nzira yakakwana yekuvandudza kushanda kwemaseva madiki, anoshanda nesimba.

Tiri kutanga kuita single global shared server fleet. Parizvino inenge 20% yemaseva edu ari mudziva rakagovaniswa. Kuti uwane 100%, nyaya dzakawanda dzinoda kugadziriswa, zvinosanganisira kuchengetedza dziva rekuchengetera rakagovaniswa, kugadzirisa otomatiki, kugadzirisa zvinodiwa nevanoroja, kuvandudza mashandisirwo evhavha, nekuvandudza rutsigiro rwemichina yekudzidzira basa. Hatingamirire kuti titore matambudziko aya uye tigovane kubudirira kwedu.

Source: www.habr.com