I-Tupperware: Umbulali ka-Kubernetes ka-Facebook?

Ukuphatha okusebenzayo nokuthembekile kwamaqoqo kunoma yisiphi isikali nge-Tupperware

I-Tupperware: Umbulali ka-Kubernetes ka-Facebook?

Namuhla kuqhubeke Systems@Scale conference sethule i-Tupperware, isistimu yethu yokuphatha iqoqo ehlela iziqukathi ezigidini zamaseva asebenzisa cishe zonke izinkonzo zethu. Saqala ukusebenzisa iTupperware ngo-2011, futhi kusukela lapho ingqalasizinda yethu ikhule isuka 1 isikhungo sedatha kuze kuphelele Izikhungo zedatha ezingu-15 ezisatshalaliswe nge-geo. Sonke lesi sikhathi, iTupperware ayizange ime futhi ithuthuke nathi. Sizokukhombisa ukuthi i-Tupperware ikunikeza kanjani ukuphathwa kweqoqo kwesigaba sokuqala, okuhlanganisa usekelo olulula lwezinsizakalo ezisezingeni eliphezulu, iphaneli yokulawula eyodwa yazo zonke izikhungo zedatha, kanye nekhono lokusabalalisa umthamo phakathi kwamasevisi ngesikhathi sangempela. Sizophinde sabelane ngezifundo esizifundile njengoba ingqalasizinda yethu ithuthuka.

I-Tupperware yenza imisebenzi ehlukene. Abathuthukisi bohlelo lokusebenza balusebenzisela ukuletha nokuphatha izinhlelo zokusebenza. Ipakisha ikhodi yohlelo lokusebenza kanye nokuncika esithombeni bese iyiletha kumaseva njengeziqukathi. Iziqukathi zihlinzeka ngokuhlukaniswa phakathi kwezinhlelo zokusebenza kuseva efanayo ukuze onjiniyela babhekane nengqondo yohlelo lokusebenza futhi bangakhathazeki ngokuthola amaseva noma ukuphatha izibuyekezo. I-Tupperware iphinde iqaphe ukusebenza kweseva, futhi uma ithola ukwehluleka, idlulisela iziqukathi zisuka kuseva eyinkinga.

Onjiniyela bokuhlela amandla basebenzisa i-Tupperware ukwaba umthamo weseva emaqenjini ngokusekelwe kubhajethi kanye nezingqinamba. Futhi bayisebenzisela ukuthuthukisa ukusetshenziswa kweseva. Abasebenzi besikhungo sedatha baphendukela ku-Tupperware ukuze basabalalise kahle iziqukathi kuzo zonke izikhungo zedatha futhi bamise noma bahambise iziqukathi ngesikhathi sokulungiswa. Ngenxa yalokhu, ukugcina amaseva, amanethiwekhi kanye nemishini kudinga ukungenelela kwabantu okuncane.

Tupperware architecture

I-Tupperware: Umbulali ka-Kubernetes ka-Facebook?

I-Tupperware PRN architecture ingenye yezifunda zezikhungo zethu zedatha. Lesi sifunda sinezakhiwo eziningi zesikhungo sedatha (PRN1 kanye ne-PRN2) eziseduze. Sihlela ukwenza iphaneli yokulawula eyodwa ezophatha wonke amaseva endaweni eyodwa.

Abathuthukisi bezicelo baletha izinsiza ngendlela yemisebenzi ye-Tupperware. Umsebenzi uqukethe iziqukathi eziningi, futhi zonke zivame ukusebenzisa ikhodi yesicelo efanayo.

I-Tupperware inesibopho sokuhlinzeka ngeziqukathi nokuphatha umjikelezo wabo wempilo. Iqukethe izingxenye eziningana:

  • I-Tupperware frontend ihlinzeka ngama-API okusetshenziswa kubonwa, i-CLI, namanye amathuluzi ezishintshayo lapho ungasebenzisa khona ne-Tupperware. Bafihla sonke isakhiwo sangaphakathi kubanikazi bemisebenzi yeTupperware.
  • I-Tupperware Scheduler iyiphaneli yokulawula enesibopho sokuphatha isiqukathi nomjikelezo wempilo yomsebenzi. Isetshenziswa emazingeni esifunda nawomhlaba, lapho umhleli wesifunda ephatha amaseva endaweni eyodwa futhi umhleli womhlaba wonke elawula amaseva avela ezifundeni ezihlukene. I-scheduler ihlukaniswe ngama-shards, futhi shard ngayinye ilawula isethi yemisebenzi.
  • Ummeleli Weshejuli we-Tupperware ufihla ukushadilwa kwangaphakathi futhi unikeze ifasitelana elilodwa lengilazi elilungele abasebenzisi be-Tupperware.
  • I-Tupperware allocator yabela iziqukathi kumaseva. Umhleli uphatha ukumiswa, ukuqala, ukuvuselela, kanye nokuhluleka kweziqukathi. Okwamanje, isabelo esisodwa singaphatha sonke isifunda ngaphandle kokuhlukana sibe izingcezu. (Qaphela umehluko kumatemu. Isibonelo, isihleli ku-Tupperware sihambisana nephaneli yokulawula ku Kubernetes, futhi i-Tupperware allocator ibizwa ngokuthi i-scheduler ku-Kubernetes.)
  • Umthengisi wezisetshenziswa ugcina umthombo weqiniso weseva nemicimbi yesevisi. Sisebenzisa umthengisi wensiza eyodwa kusikhungo sedatha ngasinye, futhi sigcina lonke ulwazi mayelana namaseva kuleso sikhungo sedatha. Umthengisi wensiza kanye nesistimu yokuphatha amandla, noma isistimu yokuhlinzeka ngezisetshenziswa, inquma ngamandla ukuthi iyiphi ishejuli yokulethwa elawula ukuthi iyiphi iseva. Isevisi yokuhlola impilo iqapha amaseva futhi igcine idatha emayelana nempilo yawo kumdayisi wezinsiza. Uma iseva inezinkinga noma idinga ukulungiswa, umthengisi wensiza utshela owabayo kanye nomhleli ukuthi bamise iziqukathi noma baziyise kwamanye amaseva.
  • I-Tupperware Agent iyi-daemon esebenza kuseva ngayinye ephethe ukunikezwa nokususwa kweziqukathi. Izinhlelo zokusebenza zisebenza ngaphakathi kwesiqukathi, okuzinikeza ukuhlukaniswa okwengeziwe nokukhiqizwa kabusha. Vuliwe ngonyaka odlule ingqungquthela yeSystems @Scale Sesivele sichazile ukuthi iziqukathi ze-Tupperware ngazinye zidalwa kanjani kusetshenziswa izithombe, i-btrfs, i-cgroupv2 ne-systemd.

Izici ezihlukile ze-Tupperware

I-Tupperware ifana ngezindlela eziningi kwezinye izinhlelo zokuphatha iqoqo njengeKubernetes kanye Mesos, kodwa futhi kukhona umehluko:

  • Usekelo olwakhelwe ngaphakathi lwezinsizakalo ezisezingeni eliphezulu.
  • Iphaneli yokulawula eyodwa yamaseva ezikhungweni zedatha ezihlukene ukuze zenze ukulethwa kweziqukathi ngokuzenzakalelayo ngokusekelwe kunhloso, ukuhoxiswa kwamaqoqo nokunakekelwa.
  • Sula ukwahlukaniswa kwephaneli yokulawula ukuze usondeze.
  • I-Elastic Computing ikuvumela ukuthi usabalalise amandla phakathi kwezinsizakalo ngesikhathi sangempela.

Sithuthukise lezi zici ezinhle ukuze sisekele izinhlelo zokusebenza ezihlukahlukene ezingenaso isimo nezinhle kuwo wonke amaseva abiwe emhlabeni wonke.

Usekelo olwakhelwe ngaphakathi lwezinsizakalo ezisezingeni eliphezulu.

I-Tupperware isebenzisa izinhlobonhlobo zezinsizakalo ezibucayi ezigcina idatha yomkhiqizo eqhubekayo ye-Facebook, Instagram, Messenger kanye ne-WhatsApp. Lezi kungaba izitolo ezinkulu zamapheya enani elingukhiye (isb. ZippyDB) kanye nokuqapha izinqolobane zedatha (isibonelo, I-ODS Gorilla ΠΈ Scuba). Ukugcina izinsiza ezisezingeni eliphezulu akulula, ngoba uhlelo kufanele luqinisekise ukuthi ukuhlinzekwa kweziqukathi kungakwazi ukumelana nokuphazamiseka okukhulu, okuhlanganisa ukucisha kwenethiwekhi noma ukucisha kukagesi. Futhi nakuba amasu avamile, njengokusabalalisa iziqukathi kuzo zonke izizinda ezinephutha, asebenza kahle ezinsizeni ezingenaso isimo, izinsizakalo eziphakeme zidinga ukwesekwa okwengeziwe.

Isibonelo, uma ukwehluleka kweseva kwenza isifaniso esisodwa sesizindalwazi singatholakali, ingabe kufanele unike amandla ukugcinwa okuzenzakalelayo okuzobuyekeza ama-cores kumaseva angu-50 kusukela kuqoqo lezi-10? Kuya ngesimo. Uma enye yalezi ziphakeli ezingama-50 inesinye isifaniso sesizindalwazi esifanayo, kungcono ukulinda futhi ungalahlekelwa izifaniso ezi-2 ngesikhathi esisodwa. Ukuze senze izinqumo eziguquguqukayo mayelana nokugcinwa kwesistimu nokusebenza, sidinga ulwazi mayelana nokuphindaphinda kwedatha yangaphakathi kanye nendlela yokubeka yesevisi esezingeni ngalinye.

I-interface ye-TaskControl ivumela izinsizakalo ezisezingeni eliphezulu ukuthi zibe nomthelela ezinqumweni ezithinta ukutholakala kwedatha. Isebenzisa lesi sikhombimsebenzisi, umhleli wazisa izinhlelo zokusebenza zangaphandle mayelana nokusebenza kweziqukathi (ukuqalisa kabusha, ukuvuselela, ukufuduka, ukulungiswa). Isevisi esezingeni eliphezulu isebenzisa isilawuli esitshela i-Tupperware uma kuphephile ukwenza umsebenzi ngamunye, futhi le misebenzi ingashintshwa noma ibambezeleke okwesikhashana. Esibonelweni esingenhla, isilawuli sesizindalwazi singatshela iTupperware ukuthi ibuyekeze amaseva angama-49 kwangu-50, kodwa ishiye iseva ethile (X) yodwa okwamanje. Njengomphumela, uma isikhathi sokubuyekeza i-kernel sidlula futhi isizindalwazi sisengakwazi ukubuyisela isifaniso esiyinkinga, i-Tupperware isazobuyekeza iseva ye-X.

I-Tupperware: Umbulali ka-Kubernetes ka-Facebook?

Izinsizakalo eziningi ezisezingeni eliphezulu ku-Tupperware azisebenzisi i-TaskControl ngokuqondile, kodwa nge-ShardManager, inkundla evamile yokudala izinsiza ezisezingeni eliphezulu ku-Facebook. Nge-Tupperware, onjiniyela bangacacisa inhloso yabo yokuthi iziqukathi kufanele zisatshalaliswe kanjani kuzo zonke izikhungo zedatha. Nge-ShardManager, onjiniyela bacacisa inhloso yabo yokuthi izingxenye zedatha kufanele zisatshalaliswe kanjani kuzo zonke iziqukathi. I-ShardManager iyakwazi ukubekwa kwedatha nokuphindaphindwa kwezinhlelo zayo zokusebenza futhi ixhumana ne-Tupperware ngokusebenzisa isixhumi esibonakalayo se-TaskControl ukuze ihlele ukusebenza kwesiqukathi ngaphandle kokubandakanyeka okuqondile kwezinhlelo zokusebenza. Lokhu kuhlanganiswa kwenza ukuphathwa kwezinsizakalo ezisezingeni eliphakeme kube lula, kodwa i-TaskControl ikwazi ukwenza okwengeziwe. Isibonelo, isigaba sethu sewebhu esibanzi asinaso isimo futhi sisebenzisa i-TaskControl ukulungisa ngokuguqukayo izinga lezibuyekezo ezitsheni. Ekugcineni isigaba sewebhu siyakwazi ukuqeda ngokushesha ukukhishwa kwesoftware eminingi ngosuku ngaphandle kokuyekethisa ukutholakala.

Ukuphatha amaseva ezikhungweni zedatha

Ngenkathi i-Tupperware yethulwa okokuqala ngo-2011, iqoqo ngalinye leseva laliphethwe umhleli ohlukile. Emuva ngaleso sikhathi, iqoqo le-Facebook laliyiqembu lama-racks weseva axhunywe ekushintsheni kwenethiwekhi eyodwa, futhi isikhungo sedatha sasinamaqoqo amaningana. Isihleli singakwazi ukuphatha amaseva kuphela kwiqoqo elilodwa, okusho ukuthi umsebenzi awukwazanga ukusabalala kumaqoqo amaningi. Ingqalasizinda yethu yakhula, saqhubeka sacisha amaqoqo. Njengoba i-Tupperware ingakwazi ukususa umsebenzi kusukela kuqoqo elinqanyuliwe ukuya kwamanye amaqoqo ngaphandle kwezinguquko, idinga umzamo omkhulu nokuxhumana ngokucophelela phakathi konjiniyela bohlelo lokusebenza kanye nabaqhubi besikhungo sedatha. Le nqubo iholele ekumoshweni kwezinsiza lapho amaseva engasebenzi izinyanga ngenxa yezinqubo zokuyekisa ukusebenza.

Sakhe umthengisi wezisetshenziswa ukuze axazulule inkinga yokuyeka ukusebenza kweqoqo futhi axhumanise ezinye izinhlobo zemisebenzi yokulungisa. Umthengisi wensiza ulandela umkhondo walo lonke ulwazi oluphathekayo oluhlotshaniswa neseva futhi anqume ngamandla ukuthi imuphi umhleli olawula iseva ngayinye. Ukuxhumanisa amaseva ngokunamandla kubahleli kuvumela isihleli ukuthi siphathe amaseva ezikhungweni zedatha ezihlukene. Njengoba umsebenzi we-Tupperware ungasakhawulelwe kuqoqo elilodwa, abasebenzisi be-Tupperware bangacacisa ukuthi iziqukathi kufanele zisatshalaliswe kanjani ezizindeni ezinephutha. Isibonelo, unjiniyela angasho inhloso yakhe (athi: "sebenzisa umsebenzi wami ezizindeni ezinephutha ezingu-2 endaweni ye-PRN") ngaphandle kokucacisa izindawo ezithile zokutholakala. I-Tupperware ngokwayo izothola amaseva afanelekile ukuze iqalise le nhloso, noma ngabe iqoqo noma isevisi iyekisiwe.

I-Scalable ukusekela yonke isistimu yomhlaba wonke

Ngokomlando, ingqalasizinda yethu ihlukaniswe yaba amakhulukhulu amaseva azinikele eqembu ngalinye. Ngenxa yokwehlukana nokuntuleka kwamazinga, sibe nezindleko eziphezulu zokusebenza, futhi amaseva angasebenzi kwakunzima kakhulu ukuwasebenzisa futhi. Engqungqutheleni yangonyaka odlule Amasistimu @Scale sethule ingqalasizinda njengesevisi (IaaS), okufanele ihlanganise ingqalasizinda yethu ibe ipaki enkulu yeseva eyodwa. Kodwa i-server park eyodwa inobunzima bayo. Kufanele ihlangabezane nezidingo ezithile:

  • I-Scalability. Ingqalasizinda yethu yakhula njengoba sengeza izikhungo zedatha esifundeni ngasinye. Amaseva abe mancane futhi asebenza ngendlela eyongayo, ngakho-ke kukhona amaningi kuwo esifundeni ngasinye. Njengomphumela, isihleli esisodwa isifunda ngasinye asikwazi ukuphatha inani leziqukathi ezingase zisetshenziswe kumakhulu ezinkulungwane zamaseva esifundeni ngasinye.
  • Ukuthembeka Ngisho noma umhleli angakhuliswa kakhulu kangako, ububanzi obukhulu bomhleli busho ukuthi kunengozi enkulu yamaphutha futhi sonke isifunda seziqukathi singase singalawuleki.
  • Ukubekezelela amaphutha. Esimeni sokwehluleka okukhulu kwengqalasizinda (isibonelo, amaseva asebenzisa isihleli ahluleka ngenxa yokuhluleka kwenethiwekhi noma ukuphela kukagesi), imiphumela engemihle kufanele ithinte kuphela ingxenye yamaseva esifundeni.
  • Ukusebenziseka kalula. Kungase kubonakale sengathi udinga ukusebenzisa abahleli abambalwa abazimele besifunda esisodwa. Kodwa ngokubuka okulula, indawo eyodwa yokungena endaweni ehlanganyelwe yesifunda yenza kube lula ukuphatha umthamo nemisebenzi.

Sihlukanise isihleli sibe ngamashadi ukuze sixazulule izinkinga zokugcina ichibi elikhulu okwabelwana ngalo. Ishejuli ngayinye yeshejuli ilawula isethi yayo yemisebenzi esifundeni, futhi lokhu kunciphisa ubungozi obuhlobene nesihleli. Njengoba i-pool eyabiwe ikhula, singakwazi ukwengeza izingcezu zeshejuli ezengeziwe. Kubasebenzisi be-Tupperware, ama-shards nama-schedule proxies abukeka njengephaneli yokulawula eyodwa. Akudingeki basebenze nenqwaba yamashadi ahlela imisebenzi. Amashadi esishejuli ahluke ngokuyisisekelo kubahleli beqoqo ebesiwasebenzisa ngaphambili, lapho iphaneli yokulawula ihlukaniswa ngaphandle kokuhlukanisa iqoqo elihlanganyelwe lamaseva ngokuya nge-topology yenethiwekhi.

Thuthukisa Ukusebenzisa Kahle Nge-Elastic Computing

Uma ingqalasizinda yethu inkulu, kubaluleke kakhulu ukuthi sisebenzise amaseva ethu ngendlela efanele ukuze kwandiswe izindleko zengqalasizinda kanye nokunciphisa umthwalo. Kunezindlela ezimbili zokwandisa ukusebenza kahle kokusetshenziswa kweseva:

  • Ikhompuyutha enwebekayo - yehlisa izinsizakalo ze-inthanethi phakathi namahora okuthula futhi usebenzise amaseva akhululiwe ukuze uthole imisebenzi engaxhunyiwe ku-inthanethi, njengokufunda ngomshini kanye nemisebenzi ye-MapReduce.
  • Ukulayisha ngokweqile - Beka izinsiza eziku-inthanethi kanye nenqwaba yomsebenzi kumaseva afanayo ukuze inqwaba yomsebenzi isebenze ngokuza kuqala.

Umgoqo ezikhungweni zethu zedatha uwukuthi Ukusetshenziswa kwamandla. Ngakho-ke, sikhetha amaseva amancane, awonga amandla ahlinzeka ngamandla amaningi okucubungula. Ngeshwa, kumaseva amancane ane-CPU encane nenkumbulo, ukulayisha ngokweqile akusebenzanga kangako. Yiqiniso, singabeka iziqukathi ezimbalwa zezinsizakalo ezincane kuseva eyodwa encane eyonga amandla edla izinsiza ezincane zeprosesa nenkumbulo, kodwa izinsizakalo ezinkulu zizoba nokusebenza okuphansi kulesi simo. Ngakho-ke, seluleka abathuthukisi bezinsizakalo zethu ezinkulu ukuthi bazithuthukise ukuze basebenzise wonke amaseva.


Ngokuyisisekelo, sithuthukisa ukusebenza kahle kokusetshenziswa sisebenzisa i-elastic computing. Iningi lezinsizakalo zethu ezinkulu, ezifana ne-News Feed, isici sokuthumela imiyalezo, kanye nesigaba sewebhu esisekupheleni, ziyahlukahluka kuye ngesikhathi sosuku. Sehlisa ngamabomu izinsiza eziku-inthanethi phakathi namahora okuthula futhi sisebenzisa amaseva akhululiwe emithwalweni yokusebenza engaxhunyiwe ku-inthanethi, njengokufunda ngomshini kanye nemisebenzi ye-MapReduce.

I-Tupperware: Umbulali ka-Kubernetes ka-Facebook?

Sazi ngokuhlangenwe nakho kwethu ukuthi kungcono ukunikeza amaseva wonke njengamayunithi amandla anwebekayo ngoba izinsiza ezinkulu zingabanikeli abakhulu nabathengi abakhulu bomthamo onwebekayo, futhi athuthukiswa ukuze asebenzise amaseva wonke. Uma iseva ikhululwa enkonzweni ye-inthanethi phakathi namahora okuthula, umthengisi wensiza uqashisa iseva kusihleli ukuze aqhube imisebenzi engaxhunyiwe ku-inthanethi kuso. Uma isevisi eku-inthanethi ihlangabezana nomthwalo omkhulu, umthengisi wezisetshenziswa ukhumbula ngokushesha iseva ebolekiwe futhi, kanye nomhleli, uyibuyisela kusevisi ye-inthanethi.

Izifundo ezifundiwe kanye nezinhlelo zekusasa

Kule minyaka engu-8 edlule, besithuthukisa iTupperware ukuze sihambisane nokukhula okusheshayo kwe-Facebook. Sabelana ngesikufundile futhi sithemba ukuthi kuzosiza abanye baphathe ingqalasizinda ekhula ngokushesha:

  • Setha ukuxhumana okuvumelana nezimo phakathi kwephaneli yokulawula namaseva ewaphethe. Lokhu kuvumelana nezimo kuvumela iphaneli yokulawula ukuthi ilawule amaseva ezikhungweni zedatha ezihlukene, isiza ukwenza ngokuzenzakalelayo ukuhoxiswa nokugcinwa kwamaqoqo, futhi inike amandla ukwabiwa kwamandla aguquguqukayo kusetshenziswa ikhompyutha enwebekayo.
  • Ngephaneli yokulawula eyodwa esifundeni, kuba lula kakhulu ukusebenza ngemisebenzi futhi kulula ukuphatha umkhumbi omkhulu weseva eyabiwe. Qaphela ukuthi iphaneli yokulawula igcina iphuzu elilodwa lokungena, ngisho noma isakhiwo salo sangaphakathi sihlukaniswa ngezizathu zesilinganiso noma ukubekezelela amaphutha.
  • Isebenzisa imodeli ye-plugin, iphaneli yokulawula ingazisa izinhlelo zokusebenza zangaphandle ngokusebenza kwesiqukathi okuzayo. Ngaphezu kwalokho, izinsizakalo ezisezingeni eliphakeme zingasebenzisa isixhumi esibonakalayo se-plugin ukwenza ngendlela oyifisayo ukuphathwa kweziqukathi. Ngale modeli ye-plugin, iphaneli yokulawula inikeza ubulula ngenkathi isebenzela ngokuphumelelayo izinsiza eziningi ezihlukile.
  • Sikholelwa ukuthi i-elastic computing, lapho sisusa khona wonke amaseva kumasevisi abanikeli bemisebenzi yenqwaba, ukufunda ngomshini, nezinye izinsizakalo ezingaphuthumi, kuyindlela elungile yokuthuthukisa ukusebenza kahle kwamaseva amancane, awonga amandla.

Siyaqala ukwenza imikhumbi yeseva eyodwa yomhlaba eyabiwe. Okwamanje cishe amaphesenti angu-20 amaseva ethu asendaweni yokuhlanganyela. Ukuze kuzuzwe u-100%, izinkinga eziningi zidinga ukubhekwana nazo, okuhlanganisa ukugcinwa kwedamu lesitoreji esabelwe, ukulungiswa okuzenzakalelayo, ukuphatha izidingo zabaqashi, ukuthuthukisa ukusetshenziswa kweseva, nokuthuthukisa ukusekelwa kwemithwalo yokufunda yomshini. Asikwazi ukulinda ukubhekana nalezi zinselelo futhi sabelane ngempumelelo yethu.

Source: www.habr.com

Engeza amazwana