{"id":91090,"date":"2020-08-08T13:42:11","date_gmt":"2020-08-08T11:42:11","guid":{"rendered":"https:\/\/prohoster.info\/blog\/administrirovanie\/arhitektura-s3-3-goda-evolyuczii-mail-ru-cloud-storage"},"modified":"2020-08-08T13:42:11","modified_gmt":"2020-08-08T11:42:11","slug":"arhitektura-s3-3-goda-evolyuczii-mail-ru-cloud-storage","status":"publish","type":"post","link":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/arhitektura-s3-3-goda-evolyuczii-mail-ru-cloud-storage","title":{"rendered":"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/a53775776b3fc969cc15c9d13a64039d.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em><noindex><a rel=\"nofollow\" href=\"https:\/\/www.deviantart.com\/st-pete\/art\/Storage-Corridor-408874509\">Korridori i Ruajtjes<\/a><\/noindex> nga St-Pete<\/em><\/p>\n<p><\/p>\n<p>P\u00ebrsh\u00ebndetje t\u00eb gjith\u00ebve! Un\u00eb jam Mons Anderson, arkitekt i platform\u00ebs <noindex><a rel=\"nofollow\" href=\"https:\/\/mcs.mail.ru\/\">Zgjidhjet Cloud t\u00eb Mail.ru<\/a><\/noindex>, do t\u00eb flas p\u00ebr m\u00ebnyr\u00ebn se si nd\u00ebrtuam ruajtjen ton\u00eb S3, si funksionon ajo, cilat zgjidhje ishin t\u00eb suksesshme dhe cilat do t\u00eb kishim ndryshuar n\u00ebse do t\u00eb fillonim nj\u00eb projekt t\u00eb till\u00eb nga zero tani.<\/p>\n<p><\/p>\n<p>Artikulli \u00ebsht\u00eb p\u00ebrgatitur mbi baz\u00ebn e nj\u00eb prezantimi n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/corp.mail.ru\/ru\/press\/events\/databases-2\/\">@Databases Meetup<\/a><\/noindex> nga Mail.ru Cloud Solutions &amp; Tarantool. N\u00eb artikull do t\u00eb flasim p\u00ebr:<\/p>\n<p><\/p>\n<ul>\n<li>si ishte nd\u00ebrtuar ruajtja e Mail.ru, mbi t\u00eb cil\u00ebn nd\u00ebrtuam ruajtjen S3;<\/li>\n<li>\u00e7far\u00eb shtuam p\u00ebr t\u00eb krijuar Mail.ru Cloud Storage;<\/li>\n<li>si funksionon modeli i ruajtjes objektuale dhe cilat hapa jan\u00eb ndjekur p\u00ebr t\u00eb kaluar n\u00eb prodhim;<\/li>\n<li>p\u00ebr p\u00ebrmir\u00ebsimet e sistemit operativ: d\u00ebshtimi dhe shkall\u00ebzimi;<\/li>\n<li>si e realizuam ndarjen dhe rishp\u00ebrndarjen;<\/li>\n<li>si dhe p\u00ebr pun\u00ebn me certifikatat SSL.<\/li>\n<\/ul>\n<p><\/p>\n<p>N\u00ebse nuk d\u00ebshironi t\u00eb lexoni, mund t\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/youtu.be\/NEgm1nsv-qg\">Shikoni<\/a><\/noindex>.<\/p>\n<p><noindex><a rel=\"nofollow\" name=\"habracut\"><\/a><\/noindex><\/p>\n<h2 id=\"kak-bylo-ustroeno-hranilische-mailru-poverh-kotorogo-my-stroili-s3-hranilische\">Si ishte nd\u00ebrtuar ruajtja e Mail.ru, mbi t\u00eb cil\u00ebn nd\u00ebrtuam ruajtjen S3<\/h2>\n<p><\/p>\n<p>Zhvillimi i S3-ton\u00eb filloi mbi ruajtjen e Oblaqeve Mail.ru, prandaj fillimisht duhet t\u00eb flasim p\u00ebr m\u00ebnyr\u00ebn se si \u00ebsht\u00eb nd\u00ebrtuar dhe \u00e7far\u00eb mund t\u00eb b\u00ebj\u00eb.<\/p>\n<p><\/p>\n<p>Ruajtja e oblakut Mail.ru p\u00ebrb\u00ebhet nga servera me disqe. N\u00eb mesatarje, nj\u00eb server storage modern ka 36 disqe prej 12\u201314 terabajt\u00ebsh. M\u00eb par\u00eb, disqet ishin m\u00eb t\u00eb vogla, por p\u00ebr tre vjet, kapaciteti i disqeve \u00ebsht\u00eb rritur dhe sot arrin pothuajse nj\u00eb p\u00ebr t\u00eb dh\u00ebnat e pap\u00ebrpunuara. <\/p>\n<p><\/p>\n<p>Disqet nga serverat e ndrysh\u00ebm t\u00eb ruajtjes bashkohen n\u00eb ato q\u00eb quhen \"\u00e7ifti\" (pair). Nj\u00eb \u00e7ift \u00ebsht\u00eb nj\u00ebsi e vetme p\u00ebr ruajtjen e skedar\u00ebve. N\u00eb thelb, \u00ebsht\u00eb nj\u00eb disk i montuar n\u00eb nj\u00eb seksion t\u00eb caktuar n\u00eb nj\u00eb rrug\u00eb t\u00eb caktuar, ku mund t\u00eb ndodhen skedar\u00eb t\u00eb identifikuar me hash. <\/p>\n<p><\/p>\n<p>\u00c7ifti \u00ebsht\u00eb nj\u00eb em\u00ebr historik, q\u00eb \u00ebsht\u00eb ruajtur deri sot, megjith\u00ebse tani n\u00eb nj\u00eb \u00e7ift nuk ka domosdoshm\u00ebrisht vet\u00ebm dy disqe. Mund t\u00eb ket\u00eb tre disqe dhe gjithashtu mund t\u00eb ket\u00eb ruajtje t\u00eb ndryshme hibride, p\u00ebr shembull 3\/2. <\/p>\n<p>\n<img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/544ab79846539262e2d4dff6ab95ffff.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><em>\u00c7ifti (pair) \u2014 nj\u00ebsit\u00eb e ruajtjes s\u00eb objekteve<\/em><\/p>\n<p><\/p>\n<p>T\u00eb gjitha \u00e7ifti ruajten n\u00eb PairDB \u2014 kjo \u00ebsht\u00eb nj\u00eb aplikacion i bazuar n\u00eb Tarantool. T\u00eb gjitha bazat n\u00eb ruajtjen ton\u00eb, duke filluar nga ato m\u00eb t\u00eb para, jan\u00eb Tarantool; ne nuk p\u00ebrdorim baza t\u00eb tjera.<\/p>\n<p><\/p>\n<p>PairDB ruan t\u00eb gjitha \u00e7ifti, gjendjet e tyre, hap\u00ebsirat e lira, mund\u00ebsit\u00eb e d\u00ebshtimit dhe gabimet e fundit. Ajo gjithashtu mund t\u00eb hyj\u00eb n\u00eb \u00e7ifti vet\u00eb, t\u00eb p\u00ebrdit\u00ebsoj\u00eb gjendjen e tyre, t\u00eb kontrolloj\u00eb n\u00ebse ato funksionojn\u00eb apo jo. Pra, PairDB \u00ebsht\u00eb nj\u00eb pamje e p\u00ebrgjithshme e gjendjes s\u00eb t\u00eb gjith\u00eb disqeve n\u00eb sistemin ton\u00eb.<\/p>\n<p>\n<img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/d50781b8b51ed5fe126bc84e19c934d1.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><em>Pair DB: baza e t\u00eb dh\u00ebnave me gjendjen e \u00e7ifteve<\/em><\/p>\n<p><\/p>\n<p>N\u00eb \u00e7ifti ruajten skedar\u00eb dhe, p\u00ebr t\u00eb ditur se n\u00eb cilin \u00e7ift ndodhet cila skedar, ne kemi nevoj\u00eb p\u00ebr nj\u00eb baz\u00eb tjet\u00ebr \u2014 FileDB. Ajo ruan mapimin, p\u00ebrcaktimin e lidhjes: skedari i till\u00eb ruhet n\u00eb \u00e7iftin e till\u00eb, si dhe nj\u00eb sasi t\u00eb vog\u00ebl atributesh t\u00eb nevojshme.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/b1a64737cf7f916bff446c10c1f01d4c.jpeg\" style=\"display:block;margin: 0 auto;\" \/><em>File DB: vendi ku ruhet skedari<\/em><\/p>\n<p>Nj\u00eb tjet\u00ebr element i r\u00ebnd\u00ebsish\u00ebm \u00ebsht\u00eb sh\u00ebrbimi Nylon, nj\u00eb ruter p\u00ebr pun\u00ebn me t\u00eb dh\u00ebnat. Ai \u00ebsht\u00eb pika e vetme e hyrjes, duke lejuar t\u00eb punoni p\u00ebrmes nj\u00eb nd\u00ebrfaqe t\u00eb vetme si me PairDB ashtu edhe me FileDB. Ky \u00ebsht\u00eb nj\u00eb sh\u00ebrbim pa shtet, ai b\u00ebn balancimin e k\u00ebrkesave, kupton se n\u00eb cilin shard FileDB duhet t\u00eb shkohet dhe di se cilat \u00e7ifti jan\u00eb aktiv\u00eb e cilat jo.<\/p>\n<p>\n<img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/3e2225f229c8cfa462d620d2b5fb581e.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><em>Nylon: ruter p\u00ebr pun\u00ebn me t\u00eb dh\u00ebnat<\/em><\/p>\n<p><\/p>\n<p>Gjithashtu n\u00eb ruajtje duhet t\u00eb vendosim ndonj\u00eb p\u00ebrmbajtje. P\u00ebr k\u00ebt\u00eb ka nj\u00eb sh\u00ebrbim \u2014 Streamer. Ai ofron dy metoda HTTP: metoda PUT, p\u00ebr t\u00eb ngarkuar p\u00ebrmbajtje n\u00eb ruajtje, dhe metoda GET, p\u00ebr ta marr\u00eb at\u00eb. HTTP \u00ebsht\u00eb nj\u00eb protokoll mjaft popullor dhe i p\u00ebrshtatsh\u00ebm p\u00ebr transferimin e t\u00eb dh\u00ebnave. <\/p>\n<p><\/p>\n<p>Kur kur ne i drejtohemi Streamer\u2019y, ai p\u00ebrmes Nylon i qaset PairDB, zbulojn\u00eb se n\u00eb cil\u00ebn \u00e7ift mund t\u00eb ngarkohet skedari, pastaj i transmeton t\u00eb dh\u00ebnat p\u00ebrmes WebDAV n\u00eb at\u00eb \u00e7ift. <\/p>\n<p><\/p>\n<p>N\u00eb thelb, \u00e7do server ruajt\u00ebs \u00ebsht\u00eb nginx plus disqe q\u00eb jan\u00eb t\u00eb montuar n\u00eb rrug\u00eb t\u00eb caktuara. Ne mund t\u00eb ngarkojm\u00eb nj\u00eb skedar n\u00eb ruajtje nga Streamer, ta fshijm\u00eb, ta rip\u00ebrkufizojm\u00eb ose ta kontrollojm\u00eb p\u00ebr integritet. Pra, kjo \u00ebsht\u00eb nj\u00eb nd\u00ebrfaqe e p\u00ebrshtatshme p\u00ebr nd\u00ebrveprimin e ul\u00ebt me ruajtjen. <\/p>\n<p>\n<img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/3722a89ed44cace36e9cb1f8bee4bd1e.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><em>Streamer: pika e hyrjes n\u00eb ruajtje<\/em><\/p>\n<p><\/p>\n<h2 id=\"chto-my-dobavili-chtoby-sdelat-s3-hranilische\">\u00c7far\u00eb shtuam p\u00ebr t\u00eb b\u00ebr\u00eb ruajtjen S3<\/h2>\n<p><\/p>\n<p>Pra, ne shqyrtuam nd\u00ebrtimin baz\u00eb t\u00eb ruajtjes n\u00eb momentin kur ne kishim p\u00ebr t\u00eb nisur ruajtjen S3. Me metod\u00ebn PUT, ne mund t\u00eb vendosim p\u00ebrmbajtje t\u00eb rast\u00ebsishme dhe t\u00eb marrim si identifikues t\u00eb k\u00ebtyre t\u00eb dh\u00ebnave nj\u00eb hash. Me k\u00ebt\u00eb identifikues, m\u00eb von\u00eb mund t\u00eb rikthehesh dhe t\u00eb merrni skedarin origjinal. Por kjo nuk \u00ebsht\u00eb e mjaftueshme p\u00ebr t\u00eb realizuar S3. N\u00eb protokollin S3, p\u00ebrve\u00e7 ruajtjes s\u00eb objekteve, ka:<\/p>\n<p><\/p>\n<ul>\n<li>ruajtja e meta t\u00eb dh\u00ebnave \u2014 pronave shtes\u00eb t\u00eb objekteve;<\/li>\n<li>organizimi i qasjeve n\u00eb objekte p\u00ebrmes HTTP;<\/li>\n<li>grupimi i objekteve n\u00eb koleksione \u2014 barek;<\/li>\n<li>HTTP-S3 Endpoint. S3 organizon t\u00eb dh\u00ebnat n\u00eb struktura t\u00eb caktuara \u2014 barek, secila prej t\u00eb cilave ofron nj\u00eb pik\u00eb hyrje p\u00ebr ruajtjen e skedar\u00ebve.<\/li>\n<\/ul>\n<p><\/p>\n<p>P\u00ebr t\u00eb realizuar k\u00ebt\u00eb logjik\u00eb, ishte e nevojshme nj\u00eb sh\u00ebrbim i ve\u00e7ant\u00eb. Poashtu donim t\u00eb mendonim n\u00eb fillim p\u00ebr arkitektur\u00ebn p\u00ebr rritjen e sh\u00ebrbimit me shkall\u00ebzim linear.<\/p>\n<p><\/p>\n<h2 id=\"pervye-komponenty\">Komponent\u00ebt e par\u00eb<\/h2>\n<p><\/p>\n<p>Demon, i cili realizon S3 API. Ky \u00ebsht\u00eb standardi S3 API i Amazon, i cili mb\u00ebshtet funksionimin me XML p\u00ebr metadatat dhe lejon transferimin e drejtp\u00ebrdrejt\u00eb t\u00eb p\u00ebrmbajtjes. Nuk na duhej t\u00eb shpiknim di\u00e7ka, gjith\u00e7ka \u00ebsht\u00eb e p\u00ebrshkruar dhe e dokumentuar.<\/p>\n<p><\/p>\n<p>Po ashtu, para sh\u00ebrbimit vendos\u00ebm Nginx. E p\u00ebrdor\u00ebm at\u00eb p\u00ebr terminimin e SSL, balancimin e ngarkes\u00ebs, si dhe p\u00ebr disa logjik\u00eb n\u00eb Lua (metrika, regjistrimi dhe gjurmimi).<\/p>\n<p><\/p>\n<p>P\u00ebr ruajtjen e metadatat\u00ebve S3, ne zgjodh\u00ebm gjithashtu Tarantool. N\u00eb versionin e par\u00eb, demon S3 i k\u00ebrkonte k\u00ebto t\u00eb dh\u00ebna n\u00eb k\u00ebt\u00eb baz\u00eb, nd\u00ebrsa p\u00ebrmbajtja vet\u00eb ruhesha n\u00eb nj\u00eb depo t\u00eb madhe p\u00ebrmes Streamer.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/03dddec282b5bb41eb63748ccfb6c9a4.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Nginx + S3 API + metadatat<\/em><\/p>\n<p><\/p>\n<h2 id=\"obektnaya-model-hraneniya\">Modeli objektor i ruajtjes<\/h2>\n<p><\/p>\n<p>Le t\u00eb shohim se si funksionon S3. P\u00ebrdoruesi mund t\u00eb krijoj\u00eb nj\u00eb bucket \u2014 nj\u00eb koleksion objektesh. Bucket adresohet me emrin e hostit dhe \u00ebsht\u00eb nj\u00eb subdomen i sh\u00ebrbimit. Brenda bucket-it, p\u00ebrdoruesi mund t\u00eb krijoj\u00eb objekte. Identifikuesi i objektit do t\u00eb jet\u00eb URL. P\u00ebrmbajtja e objektit \u00ebsht\u00eb nj\u00eb blob, nj\u00eb array t\u00eb dh\u00ebnash binare, t\u00eb cilat do t\u2019i ruajm\u00eb n\u00eb depo. Po ashtu, objekti ka atribute: emri \u2014 ai vet\u00eb URL, ACL (lista e kontrollit t\u00eb aksesit), atribute t\u00eb tjera shtes\u00eb ose t\u00eb rast\u00ebsishme \u2014 gjith\u00e7ka ruhen n\u00eb metadatat. <\/p>\n<p><\/p>\n<p>Schemi normale e k\u00ebtyre t\u00eb dh\u00ebnave mund t\u00eb duket k\u00ebshtu: ka projekte q\u00eb kan\u00eb bucket-e, q\u00eb kan\u00eb objekte dhe objektet mund t\u00eb jen\u00eb kompozita. Pasi nj\u00eb nga m\u00ebnyrat p\u00ebr t\u00eb ngarkuar objektet \u00ebsht\u00eb n\u00eb pjes\u00eb, p\u00ebr ngarkimin ekzistojn\u00eb dy tabela ndihm\u00ebse: uploads dhe chunks. Po ashtu, projektet kan\u00eb kredenciale p\u00ebr akses dhe faturim. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/e2bede45ba4044f658b47a9840fd5224.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Schema e t\u00eb dh\u00ebnave<\/em><\/p>\n<p><\/p>\n<p>Pasi b\u00ebm\u00eb nj\u00eb sh\u00ebrbim b2b me qasje t\u00eb paguar, n\u00eb k\u00ebt\u00eb skem\u00eb na duheshin faturimi.<br \/>\nSh\u00ebrbimin e faturimit e realizuam gjithashtu n\u00eb Tarantool. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/d5b1a63e2273e933c95deecdb054bfe6.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<h2 id=\"dorabotki-s3-hranilischa-shagi-k-prodakshenu\">P\u00ebrmir\u00ebsimet e S3 depo: hapat drejt prodhimit<\/h2>\n<p><\/p>\n<p>Kemi krijuar nj\u00eb model funksional, t\u00eb cilin mund ta p\u00ebrdorim: objektet dhe metadatat ruheshin, por p\u00ebr t\u00eb dal\u00eb n\u00eb prodhim mungonin disa aspekte.<\/p>\n<p><\/p>\n<p>S\u00eb pari, sistemi i rregullimeve t\u00eb normave. N\u00ebse e startojm\u00eb sh\u00ebrbimin pa t\u00eb, n\u00eb ngarkesa pikore mund t\u00eb mbingarkohej ndonj\u00eb pjes\u00eb e sistemit. Rregullimi i normave duhet t\u00eb punoj\u00eb k\u00ebshtu: \u00e7do k\u00ebrkes\u00eb S3 vjen n\u00eb nj\u00eb host specifik, ky host \u00ebsht\u00eb identifikuesi i bucket-it, dhe bucket-i p\u00ebrket ndonj\u00eb klienti. Na nevojitet t\u00eb p\u00ebrcaktojm\u00eb nj\u00eb funksion nga bucket-i, i cili do t\u00eb na lejoj\u00eb t\u00eb llogarisim rregullimin e normave. <\/p>\n<p><\/p>\n<p>P\u00ebrve\u00e7 k\u00ebsaj, sistemi i rregullimeve t\u00eb normave duhet t\u00eb jet\u00eb mjaft produktiv p\u00ebr t\u00eb p\u00ebrballuar ngarkes\u00ebn q\u00eb vjen n\u00eb S3.<\/p>\n<p><\/p>\n<p>P\u00ebr k\u00ebto p\u00ebrdor\u00ebm p\u00ebrs\u00ebri Tarantool. Rregullimet e normave jan\u00eb nj\u00eb klaster prej 21 instancash, instancat jan\u00eb t\u00eb shp\u00ebrndara n\u00eb grupe, t\u00eb ndara n\u00eb tri nyje fizike dhe t\u00eb bashkuara n\u00eb nj\u00eb klaster t\u00eb madh topologjik. N\u00eb t\u00eb p\u00ebrhapet automatikisht ndryshimet konfiguruese: caktuesit e rregullimeve, t\u00eb dh\u00ebnat e parazgjedhura dhe konfigurimi. \u00c7do bucket trajtohet n\u00eb m\u00ebnyr\u00eb strikte nga nj\u00eb instanc\u00eb. Kur nj\u00eb k\u00ebrkes\u00eb vjen p\u00ebr nj\u00eb bucket t\u00eb caktuar, llogaritet instanca p\u00ebrgjegj\u00ebse p\u00ebr at\u00eb bucket. Brenda k\u00ebsaj nyjeje b\u00ebhet llogaritja e norm\u00ebs aktuale t\u00eb k\u00ebrkesave sipas nj\u00eb algoritmi t\u00eb ngjash\u00ebm me Token Bucket. M\u00eb pas, sistemi i rregullimeve t\u00eb normave, n\u00eb baz\u00eb t\u00eb parametrave aktual\u00eb t\u00eb ngarkes\u00ebs dhe ve\u00e7orive t\u00eb caktuara p\u00ebr bucket-in e caktuar, tregon n\u00ebse k\u00ebrkesa mund t\u00eb ekzekutohet apo jo. Kontrolli i limitit b\u00ebhet n\u00eb faz\u00ebn m\u00eb t\u00eb hershme t\u00eb ekzekutimit t\u00eb k\u00ebrkes\u00ebs S3, duke mbrojtur t\u00eb gjitha elementet e tjera t\u00eb sistemit nga mbingarkesa. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/1b91e5d1fe067c859ba9b36d58827579.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Po ashtu, n\u00ebn ngarkes\u00eb \u00ebsht\u00eb mjaft e v\u00ebshtir\u00eb t\u00eb kalosh pa keq. N\u00eb S3 supozohet se duhen b\u00ebr\u00eb shum\u00eb k\u00ebrkesa p\u00ebr t\u00eb nj\u00ebjtat objekte, pra ky \u00ebsht\u00eb nj\u00eb depo e nxeht\u00eb. N\u00eb rastin normal, nj\u00eb k\u00ebrkes\u00eb p\u00ebr nj\u00eb skedar t\u00eb vet\u00ebm trajtohet nga e gjith\u00eb zinxhiri: Streamer, FileDB, PairDB, Storage. Por kur b\u00ebhen shum\u00eb k\u00ebrkesa p\u00ebr nj\u00eb skedar, ne optimizojm\u00eb qasjen n\u00eb k\u00ebt\u00eb p\u00ebrmbajtje n\u00ebp\u00ebrmjet nj\u00eb cache lokal.<\/p>\n<p><\/p>\n<p>Cache-i \u00ebsht\u00eb shum\u00eb-nj\u00ebsor dhe realizohet p\u00ebrmes nginx, SSD dhe RAM disk\u00ebve lokal\u00eb. K\u00ebtu nuk p\u00ebrdor\u00ebm Tarantool, sepse dor\u00ebzimi i objekteve \u00ebsht\u00eb m\u00eb i leht\u00eb nga sistemi i skedar\u00ebve, k\u00ebshtu q\u00eb mund t\u00eb b\u00ebjm\u00eb tirazh t\u00eb cache-it. P\u00ebr m\u00eb tep\u00ebr, ne kemi objekte t\u00eb m\u00ebdha me dimensionin maksimal prej 32 gigabajt, dhe n\u00eb Tarantool mund t\u00eb cache-osh vet\u00ebm objekte t\u00eb vogla.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/faa02bd52a1d70642589d775f22c5e75.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Ky ishte sistemi i par\u00eb, me t\u00eb cilin k\u00ebrkuam, q\u00eb kishte nj\u00eb kapacitet t\u00eb llogaritur, e cila mjaftoi p\u00ebr hulumtimin, p\u00ebr t\u00eb kuptuar produktin, se do t\u00eb funksiononte. <\/p>\n<p><\/p>\n<h2 id=\"dorabotki-boevoy-sistemy-feylover-i-masshtabirovanie\">P\u00ebrmir\u00ebsimet e sistemit n\u00eb pun\u00eb: failover dhe shkall\u00ebzim<\/h2>\n<p><\/p>\n<p>Sistemi ishte tashm\u00eb n\u00eb pun\u00eb, por n\u00eb fillim ne humb\u00ebm disa detaje \u2014 duhej t\u00eb shtonim failover dhe shkall\u00ebzim. <\/p>\n<p><\/p>\n<p>Demoni yn\u00eb S3 k\u00ebrkoi metadatat me protokollin Tarantool. N\u00eb vend t\u00eb baz\u00ebs origjinale, instaluam Tarantool, i cili vepronte si nj\u00eb router proxy p\u00ebr k\u00ebrkesat p\u00ebr metadatat. Nga perspektiva e aplikacionit q\u00eb implementonte API-n\u00eb, nuk ndryshoi asgj\u00eb \u2014 vazhdoi t\u00eb b\u00ebnte k\u00ebrkesat n\u00eb baz\u00eb me protokollin Tarantool, por routeri siguroi nj\u00eb aktivizim t\u00eb d\u00ebshtimit. K\u00ebshtu, mund\u00ebm t\u00eb kontrollonim disponueshm\u00ebrin\u00eb e nodit, t\u00eb mbanim nj\u00eb pauz\u00eb gjat\u00eb kalimeve dhe d\u00ebshtimeve, etj. N\u00eb t\u00eb nj\u00ebjt\u00ebn koh\u00eb, nuk e modifikuam aplikacionin vet\u00eb. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/978ff8a458f8d48b98672121cf6eca5d.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<h2 id=\"podrobnee-o-tom-kak-my-realizovali-shardirovanie\">M\u00eb shum\u00eb detaje rreth m\u00ebnyr\u00ebs si realizuam shardimin<\/h2>\n<p><\/p>\n<p>Pyetja tjet\u00ebr q\u00eb duhej t\u00eb merrej parasysh ishte shredimi. Sistemi po rritej, numri i objekteve po rritej dhe ne duhej t\u00eb siguronim mund\u00ebsi p\u00ebr rritje t\u00eb m\u00ebtejshme.<\/p>\n<p><\/p>\n<p>Le t\u00eb kthehemi te skema e t\u00eb dh\u00ebnave: ka projekte, ka bakete, kredenciale dhe faturim. K\u00ebto jan\u00eb objekte q\u00eb me shum\u00eb probabilitet n\u00eb t\u00eb ardhmen e af\u00ebrt nuk do t\u00eb rriten p\u00ebrtej nj\u00eb instance as p\u00ebr nga volumi, as p\u00ebr nga k\u00ebrkesat. K\u00ebshtu, nuk ka kuptim t'i shardojm\u00eb ato dhe ne i transferuam n\u00eb nj\u00eb instanc\u00eb t\u00eb ve\u00e7ant\u00eb, e cila do t\u00eb mbetet pa shardim. Kjo lejon menaxhim m\u00eb konsistent t\u00eb projekteve dhe baketeve, pasi ka nj\u00eb pik\u00eb t\u00eb vetme pa shardim. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/c5a1c1a875e756e3dceb5f885fbda2c8.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Po ashtu, n\u00eb skem\u00eb ka objekte q\u00eb rriten n\u00eb m\u00ebnyr\u00eb lineare \u2014 fillimisht ishin qindra mij\u00ebra, tani numri i tyre matet n\u00eb disa miliarda. K\u00ebto objekte, s\u00eb bashku me pjes\u00ebt e tyre, duhej t\u00eb migronin n\u00eb nj\u00eb grup t\u00eb sharduar. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/00bb418cc61ad2e3e8797caf5d1ab202.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>E ndam\u00eb skem\u00ebn, por objektet kan\u00eb nevoj\u00eb t\u00eb punojn\u00eb me baketet: \u00e7do objekt i p\u00ebrket nj\u00eb baketi t\u00eb caktuar, plus n\u00eb baket funksionon ACL. Prandaj, p\u00ebr \u00e7do shard me objekte mbajm\u00eb nj\u00eb kopje t\u00eb err\u00ebt t\u00eb \u00e7do baketi. P\u00ebr m\u00eb tep\u00ebr, gjat\u00eb modifikimit t\u00eb objekteve dhe kryerjes s\u00eb k\u00ebrkesave, nevojitet t\u00eb llogaritet volumi p\u00ebr t\u00eb realizuar faturimin, k\u00ebshtu q\u00eb n\u00eb \u00e7do shard ka num\u00ebrues nga faturimi.<\/p>\n<p><\/p>\n<p>Po ashtu, ne shtuam disa tabela dhe komponente t\u00eb tjera:<\/p>\n<p><\/p>\n<ul>\n<li>kosha, p\u00ebr shkat\u00ebrrimin e projekteve t\u00eb vjetra q\u00eb fshihen ose ngrijn\u00eb;<\/li>\n<li>radh\u00eb p\u00ebr detyrat n\u00eb sfond, dmth, ruajtja kryesore mund t\u00eb realizoj\u00eb detyra n\u00eb sfond q\u00eb nevojiten t\u00eb b\u00ebhen n\u00eb grup;<\/li>\n<li>mb\u00ebshtetje p\u00ebr lifecycle \u2014 nj\u00eb mekaniz\u00ebm q\u00eb lejon pun\u00ebn me objekte, duke menaxhuar ciklin e jet\u00ebs s\u00eb tyre.<\/li>\n<\/ul>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/933e98dc559eb8e6bac14003215a6836.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Duke qen\u00eb se disa t\u00eb dh\u00ebna i transferuam n\u00eb sharde, nevojitej nj\u00eb proxy shardimi. Ka qen\u00eb e mundur t\u00eb rip\u00ebrdoret routeri p\u00ebr k\u00ebt\u00eb rol, por nj\u00eb proxy e ve\u00e7ant\u00eb p\u00ebr shardim, e cila merret vet\u00ebm me shardimin e t\u00eb dh\u00ebnave, lejon q\u00eb routeri t\u00eb k\u00ebrkoj\u00eb t\u00eb dh\u00ebnat n\u00eb t\u00ebr\u00ebsi, pa menduar p\u00ebr shardimin.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/4b181ed4629180e15558f25b1e4d5718.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Do t\u00eb flas ve\u00e7mas p\u00ebr arsyen se pse nuk mor\u00ebm nj\u00eb zgjidhje t\u00eb gatshme dhe d\u00ebshiruam t\u00eb b\u00ebjm\u00eb nj\u00eb funksion personal t\u00eb shardimit. <\/p>\n<p><\/p>\n<p>Le t\u00eb shohim se si \u00ebsht\u00eb strukturuar. Ne kemi 256 sharde t\u00eb disponueshme. P\u00ebr \u00e7do baket, ne strukurojm\u00eb nj\u00eb gam\u00eb duke p\u00ebrdorur nj\u00eb funksion konsistent. \u00cbsht\u00eb e thjesht\u00eb \u2014 ashtu si\u00e7 definoni, me nj\u00eb funksion konsistent, p\u00ebrkat\u00ebsin\u00eb p\u00ebr nj\u00eb shard, p\u00ebrcaktoni shardin fillestar dhe ndajn\u00eb gam\u00ebn:<\/p>\n<p><\/p>\n<p><code>f(bucket, shards) = subset<\/code><\/p>\n<p><\/p>\n<p>K\u00ebshtu, n\u00ebse marrim baketin, mund t\u00eb themi se ai dhe t\u00eb dh\u00ebnat e tij do t\u00eb jen\u00eb gjithmon\u00eb n\u00eb nj\u00eb n\u00ebngrup t\u00eb caktuar nga t\u00eb gjith\u00eb shardet. Kjo ndihmon n\u00eb pak\u00ebsimin e ndikimit t\u00eb baketeve t\u00eb ndryshme mbi nj\u00ebra-tjetr\u00ebn dhe n\u00eb thjeshtimin e pun\u00ebs me k\u00ebrkesat map-reduce, kur nevojitet, p\u00ebr shembull, t\u00eb listojm\u00eb objektet e baketit. P\u00ebr k\u00ebt\u00eb, duhet t\u00eb anketojm\u00eb t\u00eb gjitha shardet ku k\u00ebto objekte ruhen. N\u00ebse objektet do ishin n\u00eb t\u00eb gjitha shardet, \u00e7do listim do t\u00eb godiste sistemin n\u00eb t\u00ebr\u00ebsi, nd\u00ebrsa k\u00ebtu godet vet\u00ebm nj\u00eb n\u00ebngrup t\u00eb caktuar.<\/p>\n<p><\/p>\n<p>Pastaj, \u00e7do objekt i p\u00ebrket nj\u00eb baketi t\u00eb caktuar, k\u00ebshtu q\u00eb kur k\u00ebrkohemi p\u00ebr nj\u00eb objekt, ne k\u00ebrkojm\u00eb p\u00ebr objektin me emrin n\u00eb baketin specifik. Pra, mund t\u00eb p\u00ebrcaktojm\u00eb nj\u00eb funksion p\u00ebr objektin jo nga e gjith\u00eb gama e shardeve, por vet\u00ebm nga n\u00ebngrupi i baketit t\u00eb tij:<\/p>\n<p><\/p>\n<p><code>f(object, subset) = shard<\/code><\/p>\n<p><\/p>\n<p>Ne marrim objektin specifik, si argumente t\u00eb funksionit ne futim jo t\u00eb gjith\u00eb shardet, por n\u00ebngrupin e baketit t\u00eb tij \u2014 dhe marrim shardin e caktuar. <\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/10480ad2430c228c511ff3cdc5071596.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Tani, shredimi \u00ebsht\u00eb realizuar, ka nj\u00eb proxy shardimi. M\u00eb pas, mbetet vet\u00ebm q\u00eb nga routeri dhe baza e t\u00eb dh\u00ebnave me metadata t\u00eb lidhemi me proxy-in e shardimit. P\u00ebr shembull, p\u00ebr krijimin e objekteve t\u00eb kopjeve t\u00eb errta \u2014 kur krijojm\u00eb nj\u00eb baket, ruajtja kryesore duhet t\u00eb krijoj\u00eb nj\u00eb p\u00ebrfaq\u00ebsues t\u00eb k\u00ebtij baketi n\u00eb t\u00eb gjith\u00eb shardet ku ai duhet t\u00eb jet\u00eb i pranish\u00ebm.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/bb34d1f747903a3c2f08f582e4424787.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<h2 id=\"kak-my-realizovali-resharding\">Si e realizuam rikthimin e shardimit<\/h2>\n<p><\/p>\n<p>Problemi m\u00eb i madh me shardimin \u00ebsht\u00eb rikthimi i shardimit. Ishte e r\u00ebnd\u00ebsishme p\u00ebr ne ta b\u00ebnim k\u00ebt\u00eb pa koh\u00eb t\u00eb ndaluar, pasi sistemi tashm\u00eb ishte n\u00eb prodhim. Do t\u00eb tregoj se si e zgjidh\u00ebm problemin n\u00eb shembullin e nj\u00eb detyre t\u00eb ngjashme me mig\u00ebrimin e gjall\u00eb t\u00eb t\u00eb dh\u00ebnave nga nj\u00eb projekt n\u00eb nj\u00eb tjet\u00ebr.<\/p>\n<p><\/p>\n<p>M\u00eb posht\u00eb \u00ebsht\u00eb skema e klasterit ton\u00eb, e cila u krijua pas implementimit t\u00eb sharding. Ne kemi nginx, S3 API, nj\u00eb router, baz\u00ebn kryesore me projektet, nj\u00eb proxy sharding dhe vet\u00eb shardet.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/0e6bd16c7fc47437b3b7d96b6d9809a4.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>M\u00eb sip\u00ebr kam l\u00ebn\u00eb jasht\u00eb momentin se n\u00eb nj\u00eb faz\u00eb t\u00eb caktuar t\u00eb projektit kishte nj\u00eb detyr\u00eb produkti: \"T\u00eb niste nj\u00eb depo tjet\u00ebr, Icebox, - si Hotbox, vet\u00ebm p\u00ebr t\u00eb dh\u00ebnat e ftohta\". N\u00eb thelb, nj\u00eb depo e ngjashme, por me URL t\u00eb tjera dhe pa cache.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/9ce9085e1d03e9a99b6b8ccc0717c653.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Icebox u p\u00ebrdor m\u00eb pak se Hotbox, prandaj ai kaloi nj\u00eb koh\u00eb t\u00eb gjat\u00eb pa asnj\u00eb sharding. N\u00eb fund, ne vendos\u00ebm ta heqim at\u00eb dhe t\u00eb bashkojm\u00eb Hotbox dhe Icebox n\u00eb nj\u00eb sh\u00ebrbim, thjesht duke ndar\u00eb klasat e ruajtjes. <\/p>\n<p><\/p>\n<p>Bucket n\u00eb depo nuk ishin t\u00eb nd\u00ebrthurura, ato mund t\u00eb merreshin leht\u00ebsisht dhe t\u00eb shkrinin bashk\u00eb, por klient\u00ebt p\u00ebrdornin si nj\u00ebri ashtu edhe tjetrin depo, prandaj duhej t\u00eb zgjidhim problemin e munges\u00ebs s\u00eb downtime. Nuk mund t\u00eb thjesht fiknim dhe t\u00eb kopjonim. Ne b\u00ebm\u00eb migrimin n\u00eb disa etapa.<\/p>\n<p><\/p>\n<p>P\u00ebr fillim, ne synchimizuam depo kryesore. Ne kishim Tarantool dhe mund t\u00eb b\u00ebnim n\u00eb krijimin e nj\u00eb objekti: <\/p>\n<p><\/p>\n<ul>\n<li>baza merr nj\u00eb k\u00ebrkes\u00eb p\u00ebr t\u00eb krijuar nj\u00eb bucket, p\u00ebr shembull n\u00eb Hotbox;<\/li>\n<li>Tarantool kontrollet n\u00eb nj\u00eb baz\u00eb tjet\u00ebr (n\u00eb k\u00ebt\u00eb rast - n\u00eb Icebox), q\u00eb nuk ekziston asnj\u00eb bucket i till\u00eb;<\/li>\n<li>n\u00ebse bucket ekziston, baza thot\u00eb se nuk mund t\u00eb krijohet dhe ai \u00ebsht\u00eb sinkronizuar si ekzistues.<br \/>\n<img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/6694c9804f09942416c1717b455b9fd3.jpeg\" style=\"display:block;margin: 0 auto;\" \/><br \/>\n<em>Sinkronizimi i buckets<\/em><\/li>\n<\/ul>\n<p><\/p>\n<p>N\u00eb at\u00eb depo, e cila duhet t\u00eb merrte t\u00eb gjitha t\u00eb dh\u00ebnat, u fut nj\u00eb atribut p\u00ebr projektet dhe buckets q\u00eb thoshte se ku ruhej ky objekt. Ai mund t\u00eb ruhej lokal, pra n\u00eb Hotbox, n\u00eb Icebox - at\u00ebher\u00eb n\u00eb depo t\u00eb re nuk ka asnj\u00eb t\u00eb dh\u00ebn\u00eb, ose mund t\u00eb jet\u00eb n\u00eb gjendjen e migrimit.<\/p>\n<p><\/p>\n<p>N\u00ebse nj\u00eb projekti ose bucket kishte atributin Migrating, at\u00ebher\u00eb gjat\u00eb migrimit k\u00ebrkesa kryhej fillimisht n\u00eb depo t\u00eb re, ku duhet t\u00eb ishin t\u00eb dh\u00ebnat, dhe n\u00ebse ato nuk ishin atje, k\u00ebrkesat p\u00ebrcilleshin n\u00eb depo alternative.<\/p>\n<p><\/p>\n<p>M\u00eb pas ne ndryshuam trafikun. Meq\u00eb API mund t\u00eb sh\u00ebrbej\u00eb dhe k\u00ebrkesat p\u00ebr Icebox, dhe k\u00ebrkesat p\u00ebr Hotbox, ne arrit\u00ebm t\u00eb ndryshojm\u00eb trafikun pa downtime, thjesht duke transferuar hostet dhe duke shtuar regjistrimet p\u00ebrkat\u00ebse n\u00eb Nginx. <\/p>\n<p><\/p>\n<p>Pas transferimit t\u00eb trafikut, Nginx dhe API nga Icebox mund t\u00eb hiqeshin.<br \/>\nM\u00eb pas heq\u00ebm Icebox nginx dhe S3 API - dhe gjith\u00e7ka filloi t\u00eb funksionoj\u00eb:<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/62d9f960fc1f11b875a8f8a5aa8e8152.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>M\u00eb pas filluam nj\u00eb proces t\u00eb prapsh\u00ebm migrimi, i cili punonte brenda baz\u00ebs - ndjek \u00e7do projekt dhe buckets e tij, vendos atributin Migrating p\u00ebr ta, transferon t\u00eb dh\u00ebnat dhe pas p\u00ebrfundimit t\u00eb transferimit vendos atributin Local.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/1a3129136f7ccacfdcff260f10cb0605.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Pas transferimit t\u00eb t\u00eb dh\u00ebnave nuk na nevojitet m\u00eb depo e vjet\u00ebr, dhe ne heqim pjes\u00ebt e mbetura t\u00eb sistemit t\u00eb vjet\u00ebr, si dhe heqim nga kodi mb\u00ebshtetje p\u00ebr statusin e migrimit.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/13b6f11add40c21a3b9afc05dd0dfea9.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<p>Sipas t\u00eb nj\u00ebjtave principe u realizua dhe resh\u00ebrdimi nga depoja e vjet\u00ebr n\u00eb t\u00eb sharding:<\/p>\n<p><\/p>\n<ul>\n<li>Kemi sh\u00ebnuar t\u00eb gjitha buckets si <code>Non-sharded<\/code>. T\u00eb gjitha k\u00ebrkesat p\u00ebr to shkonin n\u00eb depo origjinale, q\u00eb nuk ishte sharded.<\/li>\n<li>Buckets e reja krijoheshin menj\u00ebher\u00eb n\u00eb statusin <code>Sharded<\/code>.<\/li>\n<li>Marim nj\u00eb n\u00eb nj\u00eb buckets, vendosim statusin <code>Migrating<\/code> dhe transferojm\u00eb t\u00eb dh\u00ebnat.<\/li>\n<\/ul>\n<p><\/p>\n<p>K\u00ebrkesat n\u00eb k\u00ebt\u00eb rast sh\u00ebrbehen sipas parimit:<\/p>\n<p><\/p>\n<ul>\n<li>Lexojm\u00eb n\u00eb t\u00eb re, pastaj n\u00eb t\u00eb vjetra.<\/li>\n<li>Krijojm\u00eb vet\u00ebm n\u00eb t\u00eb re.<\/li>\n<li>P\u00ebrdit\u00ebsojm\u00eb n\u00eb dy faza: n\u00ebse nuk ka n\u00eb t\u00eb re, transferojm\u00eb nga e vjetra n\u00eb t\u00eb re, pastaj p\u00ebrdit\u00ebsojm\u00eb.<\/li>\n<\/ul>\n<p><\/p>\n<h2 id=\"rabota-s-ssl-sertifikatami\">Puna me certifikatat SSL<\/h2>\n<p><\/p>\n<blockquote><p>N\u00eb frontend p\u00ebrdorim Nginx. N\u00eb rastin ton\u00eb, ky nuk \u00ebsht\u00eb Nginx i zakonsh\u00ebm, por OpenResty, Nginx me mb\u00ebshtetje p\u00ebr LuaJIT.<\/p><\/blockquote>\n<p>Nj\u00eb pjes\u00eb tjet\u00ebr e sistemit \u00ebsht\u00eb puna me certifikatat SSL. N\u00eb depo S3 mund t\u00eb vendosni nj\u00eb domen tuaj p\u00ebr qasje n\u00eb nj\u00eb bucket t\u00eb ve\u00e7ant\u00eb, thjesht p\u00ebrmes <code>CNAME<\/code>. Por pa HTTPS sot nuk \u00ebsht\u00eb e mundur: nj\u00eb domen tuaj sugjeron nj\u00eb certifikat\u00eb SSL tuaj. <\/p>\n<p><\/p>\n<p>Si\u00e7 e kam th\u00ebn\u00eb m\u00eb par\u00eb, p\u00ebr balancimin dhe p\u00ebrfundimin e SSL p\u00ebrgjigjet Nginx. N\u00eb rastin ton\u00eb, ky nuk \u00ebsht\u00eb Nginx i zakonsh\u00ebm, por OpenResty, Nginx me mb\u00ebshtetje p\u00ebr LuaJIT.<\/p>\n<p><\/p>\n<p>Kjo na lejon t\u00eb m\u00ebsojm\u00eb mjaft leht\u00eb Nginx ton\u00eb p\u00ebr t\u00eb ofruar certifikata t\u00eb ndryshme. Dhe na nevojitej t'i ofronim certifikatat dinamikisht (pa nevoj\u00ebn p\u00ebr t'i shkruar n\u00eb dosjen e konfigurimit). Ne p\u00ebrdor\u00ebm zgjerimin <code>ssl_certificate_by_lua<\/code>, i cili lejon t\u00eb lexoni certifikat\u00ebn nga nj\u00eb burim t\u00eb rast\u00ebsish\u00ebm gjat\u00eb TLS-handshake. Si depo e certifikatave, ne gjithashtu mor\u00ebm Tarantool: kjo na lejon t\u00eb menaxhojm\u00eb certifikatat nga jasht\u00eb dhe siguron nj\u00eb p\u00ebrgjigje shum\u00eb t\u00eb shpejt\u00eb.<\/p>\n<p><\/p>\n<p>Gjithashtu \u00ebsht\u00eb realizuar nj\u00eb demon i ve\u00e7ant\u00eb, me detyr\u00ebn p\u00ebr t\u00eb p\u00ebrdit\u00ebsuar rregullisht certifikatat q\u00eb jan\u00eb l\u00ebshuar me ndihm\u00ebn e Let\u2019s Encrypt.<\/p>\n<p><\/p>\n<p><img decoding=\"async\" alt=\"Arkitektura S3: 3 vjet zhvillim i Mail.ru Cloud Storage\" src=\"\/wp-content\/uploads\/2020\/08\/8d61c1cb3084d9eb4eb8faa745e212db.jpeg\" style=\"display:block;margin: 0 auto;\" \/><\/p>\n<p><\/p>\n<h2 id=\"chto-by-ya-sohranil-a-chto-sdelal-po-drugomu-esli-razrabatyvat-hranilische-zanovo\">\u00c7far\u00eb do t\u00eb mbaja, dhe \u00e7far\u00eb do t\u00eb b\u00ebja ndryshe n\u00ebse do t\u00eb zhvilloja depo nga e para<\/h2>\n<p><\/p>\n<h3 id=\"chto-nuzhno-bylo-ispolzovat-s-samogo-nachala\">\u00c7far\u00eb duhej t\u00eb ishte p\u00ebrdorur q\u00eb n\u00eb fillim<\/h3>\n<p><\/p>\n<p><strong>Sharding menj\u00ebher\u00eb<\/strong>. Ka pasur shum\u00eb probleme me shardingun. \u00cbsht\u00eb e leht\u00eb t\u00eb b\u00ebhet, por megjithat\u00eb, n\u00ebse filloni projekte q\u00eb nevojiten p\u00ebr t\u2019u shkall\u00ebzuar, \u00ebsht\u00eb m\u00eb mir\u00eb t\u00eb merrni menj\u00ebher\u00eb nj\u00eb klaster t\u00eb sharduar, ndon\u00ebse me numrin minimal t\u00eb nyjave. Zbatimi i shardingut n\u00eb fillim \u00ebsht\u00eb pothuajse falas krahasuar me integrimin e shardingut n\u00eb nj\u00eb sistem n\u00eb pun\u00eb.<\/p>\n<p><\/p>\n<p><strong>Puna me Tarantool p\u00ebrmes balancuesve<\/strong>. Aktualisht, ne lidhemi me t\u00eb gjitha bazat e reja direkt n\u00ebp\u00ebrmjet balancuesve. Kjo lejon zgjerimin e funksionalitetit dhe arritjen e nj\u00eb q\u00ebndrueshm\u00ebrie m\u00eb t\u00eb madhe. <\/p>\n<p><\/p>\n<p><strong>Auto-failover<\/strong>. Do t\u00eb instalonim t\u00eb gjitha mjetet q\u00eb k\u00ebrkohen p\u00ebr auto-failover, pasi d\u00ebshtimet e para pas lan\u00e7imit ishin t\u00eb lidhura me munges\u00ebn e tij. Pas p\u00ebrvoj\u00ebs me S3, t\u00eb gjith\u00eb produktet e ardhshme u lan\u00e7uan duke e mbajtur k\u00ebt\u00eb n\u00eb mend.<\/p>\n<p><\/p>\n<p><strong>Karakteristika S3 \"Versionimi\"<\/strong>. Fillimisht dukej se kjo funksionalitet nuk ishte shum\u00eb i k\u00ebrkuar. Integraimi i k\u00ebsaj mund\u00ebsie n\u00eb arkitektur\u00ebn e nj\u00eb sistemi n\u00eb pun\u00eb \u00ebsht\u00eb jasht\u00ebzakonisht i v\u00ebshtir\u00eb.<\/p>\n<p><\/p>\n<p><strong>Billing i ve\u00e7ant\u00eb<\/strong>. Menyxhimi i faturimit n\u00eb sistemin ton\u00eb tregoi rezultate t\u00eb mira n\u00eb fillim, por m\u00eb von\u00eb u b\u00eb penges\u00eb, do t\u00eb ishte m\u00eb mir\u00eb ta kishim si nj\u00eb sh\u00ebrbim plot\u00ebsisht t\u00eb ve\u00e7ant\u00eb.<\/p>\n<p><\/p>\n<h3 id=\"chto-bylo-udachnym-resheniem\">\u00c7far\u00eb ishte nj\u00eb vendim i suksessh\u00ebm<\/h3>\n<p><\/p>\n<p><strong>Modeli i t\u00eb dh\u00ebnave<\/strong>. Historia tregon se nd\u00ebrsa sh\u00ebrbimi evoluoi, ne p\u00ebrputheshim mjaft sakt\u00eb me modelin e t\u00eb dh\u00ebnave t\u00eb Amazon-it, prandaj mund t\u00eb realizonim ato karakteristika q\u00eb jan\u00eb aty.<\/p>\n<p><\/p>\n<p><strong>Schema e shardingut<\/strong>. Do ta mb\u00ebshtesja ndarjen e nj\u00ebjt\u00eb t\u00eb shardingut n\u00ebp\u00ebr bucket-e, pasi kjo lejon shp\u00ebrndarjen e mir\u00eb t\u00eb k\u00ebrkesave nga bucket-et e ndryshme n\u00eb nj\u00eb klaster t\u00eb madh.<\/p>\n<p><\/p>\n<p><strong>P\u00ebrdorimi i Tarantool<\/strong>. Tarantool ndihmoi shum\u00eb n\u00eb zhvillimin dhe modifikimin e sh\u00ebrbimit, ne punonim leht\u00ebsisht me t\u00eb dh\u00ebnat, transformonim dhe shardonim fondin, pa pasur nevoj\u00eb t\u00eb kalonim n\u00eb nivelin e aplikacionit. <\/p>\n<p><\/p>\n<blockquote><p>Ky raport u shp\u00ebrnda p\u00ebr her\u00eb t\u00eb par\u00eb n\u00eb <noindex><a rel=\"nofollow\" href=\"https:\/\/corp.mail.ru\/ru\/press\/events\/databases-2\/\">@Databases Meetup<\/a><\/noindex> nga Mail.ru Cloud Solutions&amp;Tarantool. Shikoni <noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=jM4hL2u6JM0&amp;list=PLQzTaxmOHjntyNRPWhaWHqlHp8UN_wgQ3\">video<\/a><\/noindex> prezantime t\u00eb tjera dhe abonohuni p\u00ebr njoftimet mbi ngjarjet n\u00eb Telegram <noindex><a rel=\"nofollow\" href=\"https:\/\/t.me\/k8s_mail\">Rreth Kubernetes n\u00eb Mail.ru Group<\/a><\/noindex>.<\/p><\/blockquote>\n<p>Gjithashtu mund t\u00eb shihni prezantimin tim t\u00eb vjet\u00ebr mbi S3 ose t\u00eb lexoni artikullin e kolegut tim mbi ruajtjen me bllok.<\/p>\n<p><\/p>\n<ul>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/www.youtube.com\/watch?v=O0iIADHgBVc\">Inxhinierimi mbrapa i arkitektur\u00ebs Amazon S3 dhe si dukej ruajtja S3 e MCS tre vjet m\u00eb par\u00eb<\/a><\/noindex>.<\/li>\n<li><noindex><a rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/mailru\/blog\/472694\/\">M\u00eb shum\u00eb se Ceph: ruajtja e bllokut t\u00eb MCS<\/a><\/noindex>.<\/li>\n<\/ul>\n<p>Burimi: <a content=\"nofollow\" rel=\"nofollow\" href=\"https:\/\/habr.com\/ru\/company\/mailru\/blog\/513356\/\">habr.com<\/a> <\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"<p>Storage Corridor by St-Pete \u0412\u0441\u0435\u043c \u043f\u0440\u0438\u0432\u0435\u0442! \u042f Mons Anderson, \u0430\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u043e\u0440 \u043f\u043b\u0430\u0442\u0444\u043e\u0440\u043c\u044b Mail.ru Cloud Solutions, \u0440\u0430\u0441\u0441\u043a\u0430\u0436\u0443, \u043a\u0430\u043a \u043c\u044b \u043f\u043e\u0441\u0442\u0440\u043e\u0438\u043b\u0438 \u043d\u0430\u0448\u0435 S3-\u0445\u0440\u0430\u043d\u0438\u043b\u0438\u0449\u0435, \u043a\u0430\u043a \u043e\u043d\u043e \u0440\u0430\u0431\u043e\u0442\u0430\u0435\u0442, \u043a\u0430\u043a\u0438\u0435 \u0440\u0435\u0448\u0435\u043d\u0438\u044f \u043e\u043a\u0430\u0437\u0430\u043b\u0438\u0441\u044c \u0443\u0434\u0430\u0447\u043d\u044b\u043c\u0438, \u0430 \u043a\u0430\u043a\u0438\u0435 \u0441\u0442\u043e\u0438\u043b\u043e \u0438\u0437\u043c\u0435\u043d\u0438\u0442\u044c, \u0435\u0441\u043b\u0438 \u0431\u044b \u043c\u044b \u043d\u0430\u0447\u0430\u043b\u0438 \u0442\u0430\u043a\u043e\u0439 \u0436\u0435 \u043f\u0440\u043e\u0435\u043a\u0442 \u0441 \u043d\u0443\u043b\u044f \u0441\u0435\u0439\u0447\u0430\u0441. \u0421\u0442\u0430\u0442\u044c\u044f \u043f\u043e\u0434\u0433\u043e\u0442\u043e\u0432\u043b\u0435\u043d\u0430 \u043d\u0430 \u043e\u0441\u043d\u043e\u0432\u0435 \u0434\u043e\u043a\u043b\u0430\u0434\u0430 \u043d\u0430 @Databases Meetup by Mail.ru Cloud Solutions &amp; Tarantool. [&hellip;]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":91091,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-91090","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t<!-- All in One SEO 5.0.0.1 - aioseo.com -->\n\t<meta name=\"robots\" content=\"max-image-preview:large\" \/>\n\t<meta name=\"author\" content=\"Yuri Gagarin\"\/>\n\t<link rel=\"canonical\" href=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/arhitektura-s3-3-goda-evolyuczii-mail-ru-cloud-storage\" \/>\n\t<meta name=\"generator\" content=\"All in One SEO (AIOSEO) 5.0.0.1\" \/>\n\t\t<meta property=\"og:locale\" content=\"sq_AL\" \/>\n\t\t<meta property=\"og:site_name\" content=\"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b\" \/>\n\t\t<meta property=\"og:type\" content=\"article\" \/>\n\t\t<meta property=\"og:title\" content=\"\ud83e\udd47\u0410\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u0430 S3: 3 \u0433\u043e\u0434\u0430 \u044d\u0432\u043e\u043b\u044e\u0446\u0438\u0438 Mail.ru Cloud Storage | ProHoster\" \/>\n\t\t<meta property=\"og:url\" content=\"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/arhitektura-s3-3-goda-evolyuczii-mail-ru-cloud-storage\" \/>\n\t\t<meta property=\"og:image\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:secure_url\" content=\"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg\" \/>\n\t\t<meta property=\"og:image:width\" content=\"350\" \/>\n\t\t<meta property=\"og:image:height\" content=\"350\" \/>\n\t\t<meta property=\"article:published_time\" content=\"2020-08-08T11:42:11+00:00\" \/>\n\t\t<meta property=\"article:modified_time\" content=\"2020-08-08T11:42:11+00:00\" \/>\n\t\t<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/prohoster\" \/>\n\t\t<!-- All in One SEO -->\n\n","aioseo_head_json":{"title":"\ud83e\udd47Arkitektura S3: 3 vjet evolucioni i Mail.ru Cloud Storage | ProHoster","description":"","canonical_url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/arhitektura-s3-3-goda-evolyuczii-mail-ru-cloud-storage","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"sq_AL","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd47\u0410\u0440\u0445\u0438\u0442\u0435\u043a\u0442\u0443\u0440\u0430 S3: 3 \u0433\u043e\u0434\u0430 \u044d\u0432\u043e\u043b\u044e\u0446\u0438\u0438 Mail.ru Cloud Storage | ProHoster","og:url":"https:\/\/prohoster.info\/sq\/blog\/administrirovanie\/arhitektura-s3-3-goda-evolyuczii-mail-ru-cloud-storage","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2020-08-08T11:42:11+00:00","article:modified_time":"2020-08-08T11:42:11+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"91090","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":null,"breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 12:34:25","updated":"2022-10-02 13:27:50","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/91090","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/comments?post=91090"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/posts\/91090\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media\/91091"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/media?parent=91090"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/categories?post=91090"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/sq\/wp-json\/wp\/v2\/tags?post=91090"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}