S3 khoom cia hais kom ua txhais ib tsab xov xwm hais txog cov txheej txheem dab tsi tseem ceeb thaum xaiv qhov chaw khaws khoom. Cov hauv qab no yog cov ntawv nyeem los ntawm tus sau qhov kev xav.
Thaum nws los txog rau cov khoom khaws cia, tib neeg feem ntau tsuas xav txog ib yam: tus nqi ib TB / GB. Tau kawg, qhov kev ntsuas no tseem ceeb, tab sis nws ua rau txoj hauv kev ib sab thiab sib npaug ntawm cov khoom cia nrog cov cuab yeej khaws cia. Ntxiv rau, txoj hauv kev no txo ββqhov tseem ceeb ntawm cov khoom khaws cia rau kev lag luam thev naus laus zis pawg.
Thaum xaiv cov khoom cia, koj yuav tsum xyuam xim rau tsib yam ntxwv:
- kev ua tau zoo;
- scalability;
- S3 tau tshaj;
- teb rau kev ua tsis tiav;
- kev ncaj ncees.
Cov tsib yam ntxwv no yog cov kev ntsuas tshiab rau cov khoom khaws cia, nrog rau tus nqi. Cia peb saib lawv txhua tus.
Tsim tau
Cov khw muag khoom ib txwm muaj tsis muaj kev ua haujlwm. Cov chaw muab kev pabcuam tas li txi ββnws mus nrhiav tus nqi qis. Txawm li cas los xij, nrog cov khoom siv niaj hnub khaws khoom sib txawv.
Ntau lub tshuab cia mus kom ze lossis tshaj Hadoop qhov nrawm. Cov kev xav tau niaj hnub no rau kev nyeem thiab sau nrawm: los ntawm 10 GB / s rau hard drives, mus txog 35 GB / s rau NVMe.
Qhov kev hloov pauv no txaus rau Spark, Presto, Tensorflow, Teradata, Vertica, Splunk thiab lwm yam kev suav niaj hnub hauv pawg analytics. Qhov tseeb tias MPP cov ntaub ntawv tau teeb tsa rau cov khoom khaws cia qhia tias nws tau siv los ua thawj qhov chaw cia.
Yog tias koj lub kaw lus cia tsis muab qhov nrawm uas koj xav tau, koj siv tsis tau cov ntaub ntawv thiab rho tawm tus nqi ntawm nws. Txawm hais tias koj khaws cov ntaub ntawv los ntawm cov khoom khaws cia rau hauv lub cim xeeb ua qauv, koj tseem yuav xav tau bandwidth los hloov cov ntaub ntawv mus rau thiab los ntawm lub cim xeeb. Cov khw muag khoom qub txeeg qub teg tsis muaj txaus.
Qhov no yog lub ntsiab lus tseem ceeb: qhov kev ntsuas kev ua tau zoo tshiab yog dhau los, tsis yog latency. Nws yog tsim nyog rau cov ntaub ntawv ntawm qhov ntsuas thiab yog tus qauv hauv cov ntaub ntawv niaj hnub no.
Thaum cov ntsuas ntsuas yog ib txoj hauv kev zoo los txiav txim siab kev ua tau zoo, nws tsis tuaj yeem ntsuas qhov tseeb ua ntej khiav daim ntawv thov hauv ib puag ncig. Tsuas yog tom qab nws tuaj yeem hais tias qhov twg yog qhov khoob khoob: hauv software, disks, network lossis hauv qib xam.
Scalability
Scalability hais txog tus naj npawb ntawm petabytes uas haum rau hauv ib lub npe. Cov neeg muag khoom thov dab tsi yog qhov yooj yim scalability, qhov lawv tsis hais yog tias thaum lawv ntsuas, cov tshuab monolithic loj dhau los ua tsis yooj yim, nyuaj, tsis ruaj khov, thiab kim.
Qhov kev ntsuas tshiab rau scalability yog tus naj npawb ntawm namespaces lossis cov neeg siv khoom koj tuaj yeem ua haujlwm. Cov metric yog coj ncaj qha los ntawm hyperscalers, qhov twg cov tsev cia khoom me me tab sis ntsuas mus txog ntau lab units. Feem ntau, qhov no yog huab metric.
Thaum lub tsev thaiv me me, lawv yooj yim dua rau kev txhim kho kev ruaj ntseg, kev tswj xyuas kev nkag, kev tswj hwm txoj cai, kev tswj xyuas lub neej, thiab kev hloov tshiab tsis cuam tshuam. Thiab thaum kawg xyuas kom productivity. Qhov loj ntawm lub tsev thaiv yog qhov ua haujlwm ntawm kev tswj hwm ntawm thaj tsam tsis ua haujlwm, uas yog qhov ua tau zoo npaum li cas cov tshuab ua haujlwm.
Multi-tenancy muaj ntau yam ntxwv. Thaum lub qhov ntev hais txog yuav ua li cas cov koom haum muab kev nkag mus rau cov ntaub ntawv thiab cov ntawv thov, nws kuj hais txog cov ntawv thov lawv tus kheej thiab cov laj thawj tom qab cais lawv ntawm ib leeg.
Cov yam ntxwv ntawm txoj hauv kev niaj hnub rau ntau tus neeg siv khoom:
- Hauv ib lub sijhawm luv luv, tus naj npawb ntawm cov neeg siv khoom tuaj yeem loj hlob los ntawm ntau pua mus rau ntau lab.
- Cov neeg siv khoom raug cais tawm ntawm ib leeg. Qhov no tso cai rau lawv khiav ntau yam ntawm tib lub software thiab khaws cov khoom sib txawv, kev tso cai, nta, kev ruaj ntseg thiab kev saib xyuas qib. Qhov no yog qhov tsim nyog thaum ntsuas rau cov servers tshiab, hloov tshiab, thiab thaj chaw.
- Kev khaws cia yog elastically scalable, cov peev txheej yog muab los ntawm kev thov.
- Txhua qhov haujlwm yog tswj hwm los ntawm API thiab yog automated yam tsis muaj kev cuam tshuam tib neeg.
- Software tuaj yeem tuav hauv ntim thiab siv cov txheej txheem orchestration xws li Kubernetes.
S3 tau tshaj
Amazon S3 API yog tus qauv de facto rau cov khoom cia. Txhua yam khoom cia software neeg muag khoom thov kev sib raug zoo nrog nws. Compatibility nrog S3 yog binary: txawm nws yog tag nrho siv los yog nws tsis yog.
Hauv kev xyaum, muaj ntau pua lossis ntau txhiab tus ntug qhov xwm txheej uas muaj qee yam tsis raug thaum siv cov khoom cia. Tshwj xeeb tshaj yog los ntawm cov chaw muab kev pabcuam software thiab cov kev pabcuam. Nws cov ntaub ntawv siv tseem ceeb yog ncaj qha archiving lossis thaub qab, yog li muaj ob peb lub laj thawj hu rau API, cov ntaub ntawv siv yog homogeneous.
Qhib qhov chaw software muaj qhov zoo tshaj plaws. Nws npog feem ntau cov xwm txheej, muab qhov loj thiab ntau yam kev siv, kev khiav hauj lwm systems, thiab kho vajtse architectures.
Tag nrho cov no yog qhov tseem ceeb rau cov neeg tsim daim ntawv thov, yog li nws tsim nyog sim daim ntawv thov nrog cov chaw khaws cia. Qhib qhov chaw ua rau cov txheej txheem yooj yim dua - nws yooj yim dua kom nkag siab lub platform twg yog qhov zoo rau koj daim ntawv thov. Tus kws kho mob tuaj yeem siv los ua ib qho kev nkag mus rau hauv kev khaws cia, txhais tau tias nws yuav ua tau raws li koj xav tau.
Qhib qhov chaw txhais tau tias: cov ntawv thov tsis khi rau tus neeg muag khoom thiab pob tshab dua. Qhov no ua kom lub neej ntev ntawm daim ntawv thov.
Thiab ob peb phau ntawv ntxiv txog qhov qhib thiab S3.
Yog tias koj tab tom khiav daim ntawv thov cov ntaub ntawv loj, S3 SELECT txhim kho kev ua haujlwm thiab kev ua haujlwm tau zoo los ntawm qhov kev txiav txim loj. Nws ua qhov no los ntawm kev siv SQL los khaws cov khoom koj xav tau los ntawm kev cia.
Lub ntsiab lus tseem ceeb yog kev txhawb nqa rau cov ntawv ceeb toom thoob. Cov ntawv ceeb toom thoob ua haujlwm yooj yim serverless xam, ib qho tseem ceeb ntawm txhua qhov kev pabcuam microservice uas tau xa los ua kev pabcuam. Muab hais tias cov khoom khaws cia tau zoo huab cia, qhov peev xwm no tseem ceeb heev thaum cov khoom khaws cia siv los ntawm kev siv huab cua.
Thaum kawg, kev siv S3 yuav tsum txhawb nqa Amazon S3 server-sab encryption APIs: SSE-C, SSE-S3, SSE-KMS. Txawm zoo dua, S3 txhawb kev tiv thaiv tamper uas muaj kev ruaj ntseg tiag tiag.
Teb rau kev ua tsis tiav
Ib qho kev ntsuas uas feem ntau tsis saib xyuas yog qhov system ua haujlwm li cas. Kev ua tsis tiav tshwm sim rau ntau yam laj thawj, thiab cov khoom khaws cia yuav tsum tswj lawv txhua tus.
Piv txwv li, muaj ib qho kev ua tsis tiav, qhov ntsuas ntawm qhov no yog xoom.
Hmoov tsis zoo, ntau lub tshuab khaws cia siv cov nodes tshwj xeeb uas yuav tsum tau qhib rau pawg ua haujlwm kom zoo. Cov no suav nrog lub npe nodes lossis metadata servers - qhov no tsim ib qho kev ua tsis tiav.
Txawm tias muaj ntau lub ntsiab lus ntawm kev ua tsis tiav, lub peev xwm los tiv thaiv kev puas tsuaj loj yog qhov tseem ceeb. Disks ua tsis tau, servers poob. Tus yuam sij yog tsim software tsim los daws qhov tsis ua haujlwm raws li qhov xwm txheej ib txwm muaj. Yog tias lub disk lossis node ua tsis tiav, cov software no tseem yuav ua haujlwm ntxiv yam tsis muaj kev hloov pauv.
Ua-nyob rau hauv kev tiv thaiv cov ntaub ntawv erasure thiab cov ntaub ntawv degradation xyuas kom meej tias koj yuav poob ntau npaum li cas disks los yog nodes raws li koj muaj parity blocks - feem ntau yog ib nrab ntawm cov disks. Tsuas yog tom qab ntawd software yuav tsis tuaj yeem rov qab cov ntaub ntawv.
Qhov tsis ua haujlwm yog tsis tshua muaj kev sim nyob rau hauv load, tab sis xws li kev sim yog yuav tsum tau. Simulating ib tug load tsis ua hauj lwm yuav qhia tag nrho cov nqi tshwm sim tom qab tsis ua hauj lwm.
Kev sib raug zoo
Qhov qhab nia sib xws ntawm 100% tseem hu ua nruj sib xws. Kev sib raug zoo yog ib feem tseem ceeb ntawm txhua qhov system cia, tab sis muaj zog sib xws yog tsawg. Piv txwv li, Amazon S3 ListObject tsis nruj me ntsis raws li, nws tsuas yog ua raws li qhov kawg.
Dab tsi yog txhais los ntawm kev nruj nruj? Rau txhua qhov kev ua haujlwm tom qab kev ua haujlwm PUT tau lees paub, cov hauv qab no yuav tsum tshwm sim:
- Tus nqi tshiab yog pom thaum nyeem los ntawm ib qho ntawm cov node.
- Qhov hloov tshiab yog tiv thaiv ntawm node tsis ua hauj lwm redundancy.
Qhov no txhais tau hais tias yog tias koj rub lub ntsaws rau hauv nruab nrab ntawm kev kaw, tsis muaj dab tsi yuav ploj. Lub kaw lus yeej tsis rov qab corrupted los yog outdated cov ntaub ntawv. Qhov no yog qhov bar siab uas tseem ceeb hauv ntau qhov xwm txheej, los ntawm kev hloov pauv daim ntawv thov mus rau thaub qab thiab rov qab.
xaus
Cov no yog cov khoom ntsuas tshiab uas cuam tshuam txog kev siv cov qauv hauv cov koom haum niaj hnub no, qhov kev ua tau zoo, sib xws, scalability, txhaum domains thiab S3 compatibility yog lub tsev blocks rau huab kev siv thiab cov ntaub ntawv loj analytics. Kuv pom zoo kom siv cov npe no ntxiv rau tus nqi thaum tsim cov ntaub ntawv niaj hnub.
Hais txog Mail.ru Huab Solutions khoom cia: .
Yuav nyeem dab tsi ntxiv:
- .
- .
- .
Tau qhov twg los: www.hab.com