Nhasi tichataura nezve yakanakisa kuchengetedza dhata munyika umo chizvarwa chechishanu network, genome scanners uye mota dzinozvityaira dzinoburitsa data rakawanda pazuva kupfuura vanhu vese vakagadzirwa isati yachinja maindasitiri.
Nyika yedu iri kuunza ruzivo rwakawanda uye rwakawanda. Chimwe chikamu chayo chinokurumidza kupera uye chinorasika nokukurumidza sezvachinounganidzwa. Imwe inofanira kuchengetwa kwenguva refu, uye imwe yakatogadzirwa "kwemazana emakore" - zvirinani ndizvo zvatinoona kubva ikozvino. Ruzivo runoyerera runogara munzvimbo dze data nekukurumidza zvekuti chero nzira nyowani, chero tekinoroji yakagadzirirwa kugutsa ichi chisingaperi "chinodikanwa" chinokurumidza kupera.
40 makore ekuvandudzwa kweakagoverwa ekuchengetedza masisitimu
Yekutanga network yekuchengetedza mune fomu yatinoziva nayo yakaonekwa muma1980s. Vazhinji venyu vakasangana neNFS (Network File System), AFS (Andrew File System) kana Coda. Makumi gumi gare gare, fashoni uye tekinoroji zvachinja, uye akagovera mafaira masisitimu akapa nzira kune akaunganidzwa ekuchengetedza masisitimu akavakirwa paGPFS (General Parallel File System), CFS (Clustered File Systems) uye StorNext. Kuchengetera kwechivharo chekirasi yekuvaka yakashandiswa sehwaro, pamusoro payo imwe faira system yakagadzirwa uchishandisa software layer. Izvi uye zvakafanana mhinduro zvichiri kushandiswa, zvinogara niche yavo uye zviri kudiwa zvakanyanya.
Pakutanga kwemireniyamu, iyo yakagoverwa yekuchengetedza paradigm yakachinja neimwe nzira, uye masisitimu ane SN (Shared-Nothing) architecture akatora nzvimbo dzinotungamira. Pakave neshanduko kubva pakuchengetera masango kuenda kuchengetedzo pane imwe node, iyo, sekutonga, yaive yemhando maseva ane software inopa yakavimbika kuchengetedza; Pamitemo yakadaro, iti, HDFS (Hadoop Distributed File System) uye GFS (Global File System) inovakwa.
Padhuze ne2010s, pfungwa dziri pasi pekugoverwa kwekuchengetera masisitimu akawedzera kutanga kuratidzwa mune yakazara-yakazara zvigadzirwa zvekutengesa, seVMware vSAN, Dell EMC Isilon uye yedu. . Kuseri kwemapuratifomu ataurwa hapasisina nharaunda yevanofarira, asi vatengesi chaivo vane basa rekushanda, rutsigiro, uye sevhisi yechigadzirwa uye vanovimbisa kusimudzira kwacho. Mhinduro dzakadaro dzinonyanya kudiwa munzvimbo dzinoverengeka.
Telecom vashandisi
Zvichida mumwe wevatengi vekare veakagoverwa ekuchengetedza masisitimu ndeye telecom vanoshanda. Dhiagiramu inoratidza kuti ndeapi mapoka ezvishandiso anoburitsa huwandu hwe data. OSS (Operations Support Systems), MSS (Management Support Services) uye BSS (Business Support Systems) inomiririra zvikamu zvitatu zvemapurogiramu anowirirana anodiwa kuti ape sevhisi kune vanonyoresa, kushuma kwemari kumupi uye rutsigiro rwekushanda kune mainjiniya anoshanda.
Kakawanda, data yezvikamu izvi inosanganiswa zvakanyanya, uye kuitira kuti udzivise kuunganidza makopi asina kufanira, kuchengetwa kwakagoverwa kunoshandiswa, iyo inounganidza huwandu hwose hwemashoko anobva kune network yekushanda. Iyo yekuchengetedza inosanganiswa kuita dziva rakafanana, rinosvikwa nesevhisi ese.
Maverengero edu anoratidza kuti shanduko kubva kumhando yekuchengetera masisitimu kuvharira masisitimu ekuchengetera inobvumidza iwe kuchengetedza inosvika makumi manomwe muzana ebhajeti chete nekusiya yakatsaurirwa hi-yekupedzisira masisitimu ekuchengetedza uye kushandisa akajairwa echinyakare architecture maseva (kazhinji x70), achishanda pamwe nehunyanzvi. software. Vafambisi venhare vakatotanga kare kutenga zvigadziriso zvakadaro muhuwandu hukuru. Kunyanya, vashandisi veRussia vanga vachishandisa zvigadzirwa zvakadaro kubva kuHuawei kweanopfuura makore matanhatu.
Hongu, akati wandei mabasa haagone kupedzwa uchishandisa akagoverwa masisitimu. Semuenzaniso, nekuwedzera kwekuita zvinodiwa kana kuenderana nemaprotocol ekare. Asi inokwana 70% yedata yakagadziriswa nemushandisi inogona kuwanikwa mune dziva rakagoverwa.
Banking sector
Mune chero bhangi kune akawanda akasiyana maIT masisitimu, kutanga kubva pakugadzirisa uye kupera neautomated banking system. Izvi zvivakwa zvinoshandawo nehuwandu hukuru hweruzivo, nepo mabasa mazhinji asingade kuwedzera kwekuita uye kuvimbika kwekuchengetedza masisitimu, semuenzaniso, kusimudzira, kuyedza, otomatiki maitiro ehofisi, etc. Pano, kushandiswa kwekirasi yekuchengetedza masisitimu kunogoneka, asi gore negore zvishoma uye zvishoma zvishoma. Mukuwedzera, munyaya iyi hapana kushanduka mukushandiswa kwekuchengetedza system zviwanikwa, kushanda kwacho kunotarirwa zvichienderana nepamusoro pepamusoro.
Paunenge uchishandisa akagoverwa ekuchengetedza masisitimu, ma node avo, ayo ari mavhavha akajairwa, anogona kushandurwa chero nguva, semuenzaniso, kuita sevhavha purazi uye anoshandiswa senge computing platform.
Data lakes
Dhiagiramu iri pamusoro inoratidza runyoro rwevatengi vesevhisi . Aya anogona kunge ari e-government masevhisi (somuenzaniso, "Government Services"), makambani edhijitari, masangano emari, nezvimwewo. Vese vanofanirwa kushanda nehuwandu hwakawanda hwemashoko akasiyana.
Kushandisa ekare ekuchengetedza masisitimu ekugadzirisa matambudziko akadai haashande, nekuti zvinoda zvese zvepamusoro-kuita kuwana kune block dhatabhesi uye kuwana nguva dzose kumaraibhurari ezvinyorwa zvakaongororwa zvakachengetwa sezvinhu. Semuenzaniso, iyo yekuraira system kuburikidza newebhu portal inogona zvakare kubatanidzwa pano. Kuti uite zvese izvi papuratifomu yekuchengetedza, iwe unozoda yakakura seti yemidziyo yemabasa akasiyana. Imwe yakachinjika yepasirese yekuchengetera sisitimu inogona kunyatsovhara ese akanyorwa kare mabasa: iwe unongoda kugadzira akati wandei madziva ane akasiyana ekuchengetedza maitiro mairi.
Majenareta eruzivo rutsva
Huwandu hwemashoko akachengetwa munyika huri kukura nezvikamu makumi matatu kubva muzana pagore. Aya ndiwo mashoko akanaka kune vatengesi vekuchengetedza, asi chii chiri uye chichava chikuru che data iyi?
Makore gumi apfuura, masocial network akave majenareta akadaro; izvi zvaida kusikwa kwenhamba huru yealgorithms nyowani, hardware mhinduro, etc. Ikozvino kune matatu madhiraivha makuru ekukura kwemavhoriyamu ekuchengetedza. Yekutanga ndeye cloud computing. Parizvino, angangoita makumi manomwe muzana emakambani anoshandisa makore masevhisi neimwe nzira. Aya anogona kunge ari emagetsi mail masisitimu, makopi ekuchengetedza uye mamwe masangano anoonekwa.
Yechipiri mutyairi ndeyechishanu chizvarwa network. Aya mamhanyiro matsva uye mavhoriyamu matsva ekufambisa data. Zvinoenderana nekufanotaura kwedu, kutorwa kwakapararira kwe5G kuchatungamira mukudonha kwekuda kwema flash memory makadhi. Hazvina mhosva kuti chiyeuchidzo chakawanda sei mufoni, ichiri kupera, uye kana gadget ine 100-megabit channel, hapana chikonzero chekuchengetedza mapikicha munharaunda.
Boka rechitatu rezvikonzero nei kudiwa kwemasisitimu ekuchengetedza kuri kukura kunosanganisira kukurumidza kuvandudzwa kwehungwaru hwekugadzira, shanduko kune hombe data analytics uye maitiro akananga kune universal automation yezvese zvinogoneka.
Chinhu che "new traffic" ndechayo . Isu tinofanirwa kuchengetedza iyi data pasina kutsanangura chimiro chayo chero nzira. Inodiwa chete pakuverenga kunotevera. Semuenzaniso, kuti uone huwandu hwechikwereti huripo, banking scoring system inotarisa pamifananidzo yaunotumira pasocial network, kuona kana iwe uchiwanzoenda kugungwa uye mumaresitorendi, uye panguva imwe chete uchidzidza zvakatorwa kubva mumagwaro ako ekurapa aripo. kwairi. Aya data, kune rumwe rutivi, akazara, asi kune rumwe rutivi, haana homogeneity.
Ocean ye data isina kurongeka
Ndeapi matambudziko anokonzerwa nekubuda kwe "data idzva"? Yokutanga pakati pavo, hongu, ihwo huwandu hwehuwandu hwemashoko uye nguva inofungidzirwa yekuchengetedza kwayo. Mota yemazuva ano isina mutyairi inozvimiririra yega inogadzira anosvika makumi matanhatu terabytes yedata zuva rega rega kubva kune ese masensor ayo nemagadzirirwo. Kugadzira nyowani yekufamba algorithms, ruzivo urwu runofanirwa kugadziriswa mukati mezuva rimwe chete, zvikasadaro ruchatanga kuunganidza. Panguva imwecheteyo, inofanira kuchengetwa kwenguva yakareba kwazvo - makumi emakore. Ipapo chete ndipo pazvichakwanisika mune ramangwana kutora mhedziso zvichibva pane yakakura analytical samples.
Mudziyo mumwe wekududzira kutevedzana kwemajini unoburitsa 6 TB pazuva. Uye iyo data yakaunganidzwa nerubatsiro rwayo hairevi kudzima zvachose, kureva, hypothetically, inofanira kuchengetwa zvachose.
Pakupedzisira, mamwe chizvarwa chechishanu network. Pamusoro peiyo chaiyo inofambiswa ruzivo, network yakadai pachayo ijenareta hombe yedata: matanda echiitiko, marekodhi ekufona, mhedzisiro yepakati-yemuchina-ku-muchina kusangana, nezvimwe.
Zvese izvi zvinoda kuvandudzwa kwemaitiro matsva uye algorithms ekuchengetedza uye kugadzirisa ruzivo. Uye maitiro akadaro ari kubuda.
New era tekinoroji
Kune mapoka matatu emhinduro dzakagadzirirwa kutarisana nezvinodiwa zvitsva zvekuchengetedza ruzivo masisitimu: kuiswa kwehunyanzvi hwekugadzira, kushanduka kwehunyanzvi hwekuchengetedza midhiya uye hutsva mumunda wehurongwa hwekuvaka. Ngatitange neAI.
Mune mitsva yeHuawei mhinduro, hungwaru hwekugadzira hunoshandiswa padanho rekuchengetedza pachayo, iro rakashongedzerwa neAI processor inobvumira sisitimu kuti iongorore yakazvimirira mamiriro ayo uye kufanotaura kukundikana. Kana iyo yekuchengetedza sisitimu yakabatana kune sevhisi gore rine yakakosha komputa hunyanzvi, hungwaru hwekugadzira huchakwanisa kugadzirisa rumwe ruzivo uye kuwedzera huchokwadi hwekufungidzira kwayo.
Pamusoro pekutadza, AI yakadai inogona kufanotaura ramangwana repamusoro uye nguva yasara kusvika simba rapera. Izvi zvinokutendera kuti uwedzere kuita uye kuyera sisitimu zvisati zvaitika chero zvisingadiwe zvaitika.
Iye zvino nezve shanduko yekuchengetedza midhiya. Madhiraivha ekutanga akagadzirwa uchishandisa tekinoroji yeSLC (Single-Level Cell). Midziyo yakavakirwa pairi yaive yakakurumidza, yakavimbika, yakagadzikana, asi yaive nediki uye yaidhura zvakanyanya. Kukura kwevhoriyamu uye kuderedzwa kwemitengo kwakawanikwa kuburikidza nehumwe hunyanzvi hwekubvumidza, nekuda kwekuti kumhanya, kuvimbika uye hupenyu hwebasa hwemadhiraivha hwakaderedzwa. Zvakangodaro, maitiro aya haana kukanganisa masisitimu ekuchengetedza iwo pachawo, ayo, nekuda kweakasiyana matipi ekuvaka, anowanzova ese ari maviri anobudirira uye akavimbika.
Asi nei waida All-Flash kuchengetedza masisitimu? Zvakanga zvisina kukwana here kungotsiva maHDD ekare mune inotoshanda sisitimu nemaSDD matsva eiyo yakafanana fomu chinhu? Izvi zvaidiwa kuitira kushandisa zvinobudirira zviwanikwa zvese zveakasimba-state madhiraivha, izvo zvaive zvisingagoneke mumasisitimu ekare.
Huawei, semuenzaniso, akagadzira akati wandei matekinoroji ekugadzirisa dambudziko iri, rimwe rayo , iyo yakaita kuti zvikwanisike kugadzirisa "disk-controller" kupindirana zvakanyanya sezvinobvira.
Kuzivikanwa kwehungwaru kwakaita kuti zvikwanisike kukanganisa data munzizi dzinoverengeka uye kurarama nehuwandu hwezvinhu zvisingadiwe, senge. (nyora amplification). Panguva imwecheteyo, matsva ekudzoreredza algorithms, kunyanya , yakawedzera kukurumidza kwekuvakazve, kuderedza nguva yayo kusvika kune isingakoshi zvachose.
Kutadza, kuwandisa, kuunganidza marara - izvi zvinhu zvakare hazvichakanganisa mashandiro ehurongwa hwekuchengetedza nekuda kwekugadziriswa kwakakosha kune vanodzora.
Uye block data storages vari kugadzirira kusangana . Ngatiyeukei kuti chirongwa chekare chekuronga kuwana data chakashanda seizvi: processor yakawana RAID controller kuburikidza nePCI Express bhazi. Izvo, zvakare, zvakadyidzana nema mechanical disks kuburikidza neSCSI kana SAS. Kushandiswa kweNVMe kuseri kwemashure kwakamhanyisa zvakanyanya maitiro ese, asi yaive nekamwe dhiraivha: madhiraivha aifanirwa kuve akabatana zvakananga kune processor kuitira kuti aipe iyo yakananga kupinda mundangariro.
Chikamu chinotevera chekuvandudza tekinoroji yatiri kuona ikozvino kushandiswa kweNVMe-oF (NVMe pamusoro peMicheka). Kana iri Huawei block technologies, ivo vanototsigira FC-NVMe (NVMe pamusoro peFiber Channel), uye NVMe pamusoro peRoCE (RDMA pamusoro peConverged Ethernet) iri munzira. Iwo ebvunzo modhi anoshanda chaizvo; kwasara mwedzi yakati wandei isati yasvika yavo yepamutemo mharidzo. Ziva kuti zvese izvi zvichaonekwa mumasisitimu akagoverwa, uko "kurasikirwa kweEthernet" kunenge kuri kudiwa zvikuru.
Imwe nzira yekuwedzera mashandiro ekuchengetedza akagoverwa kwaive kusiiwa zvachose kwedata mirroring. Huawei mhinduro hadzichashandisi n makopi, seyakajairwa RAID 1, uye zvachose chinja kune iyo (Erasure coding). Iyo yakakosha yemasvomhu package inoverengera zvidhinha pane imwe periodicity, iyo inokutendera iwe kudzoreredza data repakati kana uchinge warasikirwa.
Deduplication uye compression michina inove inosungirwa. Kana mumasisitimu ekuchengetedza ekare isu takaganhurirwa nehuwandu hwema processor akaiswa mune controller, saka mune yakagovaniswa yakasimudzwa scalable masisitimu ekuchengetedza, imwe neimwe node ine zvese zvinodiwa: dhisiki, ndangariro, processors uye interconnect. Izvi zviwanikwa zvakaringana kuti ive nechokwadi chekuti deduplication uye kudzvanya zvine mashoma maitiro pakuita.
Uye nezve Hardware optimization nzira. Pano zvaigoneka kuderedza mutoro wepakati processors nerubatsiro rwemamwe machipisi akazvitsaurira (kana mabhuroko akazvitsaurira mu processor pachayo), ayo anoita basa. (TCP/IP Offload Injini) kana kutora mabasa emasvomhu eEC, kudhirowa uye kumanikidza.
Maitiro matsva ekuchengetedza data akaiswa mune yakapatsanurwa (yakagoverwa) dhizaini. Centralized kuchengetedza masisitimu ane sevha fekitori yakabatana kuburikidza neFiber Channel kune ine zvakawanda zvakarongwa. Izvo zvisingabatsiri zveiyi nzira ndeyekuoma kwekuyera uye kuve nechokwadi chehutano hwakavimbiswa (maererano nekuita kana latency). MaHyperconverged masisitimu anoshandisa iwo mauto akafanana ekuchengetedza uye kugadzirisa ruzivo. Izvi zvinopa chiyero chisingaverengeki chekuyera, asi zvinosanganisira mutengo wakakura wekuchengetedza data kuperera.
Kusiyana nezviri pamusoro apa, chivakwa chakapatsanurwa chinoreva kupatsanura sisitimu mumucheka wekombuta uye yakachinjika yekuchengetedza system. Izvi zvinopa mabhenefiti ezvivakwa zvese uye zvinobvumira kunenge kusingagumi kuyera kwechinhu chete chinoshaya kuita.
Kubva pakubatanidzwa kusvika pakusangana
Basa remhando yepamusoro, kukosha kwaro kwakangokura mumakore gumi nemashanu apfuura, kudiwa kwekupa panguva imwe chete bhuroka yekuchengetedza, kuwana mafaira, kuwana zvinhu, kushanda kwepurazi hombe re data, nezvimwe. Iyo icing pakeke inogona zvakare. kuva, semuenzaniso, negadziriro yeparutivi pamusoro magineti tepi.
Padanho rekutanga, manejimendi chete emasevhisi aya anogona kubatanidzwa. Heterogeneous data kuchengetedza masisitimu akabatanidzwa kune imwe nyanzvi software, kuburikidza iyo maneja akagovera zviwanikwa kubva kumadziva aivepo. Asi sezvo madziva aya aive nemidziyo yakasiyana, kutama kwemutoro pakati pawo kwaive kusingaite. Pamwero wepamusoro wekubatanidzwa, kuunganidzwa kwakaitika padanho resuwo. Kana kugovera faira kuripo, yaigona kupihwa kuburikidza nemaprotocol akasiyana.
Iyo yakanyanya kuchinjika nzira iripo ikozvino kwatiri inosanganisira kugadzirwa kwepasi rose hybrid system. Ndizvo chaizvo zvatinofanira kuva zvedu . Kuwanika kwepasirese kunoshandisa zviwanikwa zvehardware zvakafanana, zvine musoro zvakakamurwa kuva madziva akasiyana, asi zvichibvumira kutama kwemutoro. Zvese izvi zvinogona kuitwa kuburikidza nekamwe manejimendi console. Neiyi nzira, takakwanisa kuita iyo pfungwa ye "imwe data data - imwe yekuchengetedza system."
Mutengo wekuchengetedza ruzivo ikozvino unosarudza akawanda ezvivakwa sarudzo. Uye kunyange zvazvo inogona kuiswa zvakachengeteka pamberi, nhasi tiri kukurukura "kurarama" kuchengetedza nekuwana kushanda, saka kushanda kunofanirawo kuverengerwa. Chimwe chinhu chakakosha chechizvarwa chinotevera chakagoverwa masisitimu kubatana. Mushure mezvose, hapana anoda kuve akati wandei akasiyana masisitimu anodzorwa kubva akasiyana consoles. Hunhu hwese uhwu hwakaiswa mumutsara mutsva wezvigadzirwa zveHuawei .
Misa yekuchengetedza system yechizvarwa chitsva
OceanStor Pacific inosangana nezvitanhatu-zvipfumbamwe zvinodiwa zvekuvimbika (99,9999%) uye inogona kushandiswa kugadzira HyperMetro kirasi data nzvimbo. Nechinhambwe pakati penzvimbo mbiri dze data dzinosvika 100 km, masisitimu anoratidza yakawedzera latency ye2 ms, iyo inoita kuti zvikwanise kuvaka pahwaro hwavo chero njodzi inodzivirira njodzi, kusanganisira iyo ine quorum maseva.
Izvo zvitsva zvakatevedzana zvigadzirwa zvinoratidza protocol versatility. Kare, OceanStor 100D inotsigira kuvharika kwekuvhara, kuwana chinhu uye Hadoop kuwana. Kuwana faira kuchaitwa zvakare munguva pfupi iri kutevera. Iko hakuna chikonzero chekuchengeta akawanda makopi edata kana achigona kupihwa kuburikidza neakasiyana maprotocol.
Zvingaite senge, iyo pfungwa ye "isina kurasikirwa network" ine chekuita nekuchengetedza masisitimu? Icho chokwadi ndechekuti akagoverwa ekuchengetedza data masisitimu akavakirwa pahwaro hweiyo inokurumidza network inotsigira yakakodzera algorithms uye RoCE mashandiro. Iyo artificial intelligence system inotsigirwa nekuchinja kwedu inobatsira kuwedzera kuwedzera network kumhanya uye kuderedza latency. . Kuwana mukuchengetedza kuita kana activate AI Fabric inogona kusvika 20%.
Chii chinonzi OceanStor Pacific yakagoverwa yekuchengetedza node? Iyo 5U fomu factor solution inosanganisira 120 madhiraivha uye inogona kutsiva matatu ekirasi node, iyo inopa kanopfuura kaviri kuchengetedza munzvimbo yerack. Nekusachengeta makopi, kugona kwekutyaira kunowedzera zvakanyanya (kusvika +92%).
Isu takajaira chokwadi chekuti software-yakatsanangurwa chengetedzo isoftware yakakosha yakaiswa pane yekirasi server. Asi ikozvino, kuti uwane yakakwana paramita, iyi yekuvaka mhinduro inodawo yakakosha node. Iyo ine maseva maviri akavakirwa pane maARM processors anotonga akatevedzana matatu-inch madhiraivha.
Aya maseva haana kukodzera hyperconverged mhinduro. Chekutanga, kune akati wandei maapplication eARM, uye chechipiri, zvakaoma kuchengetedza chiyero chemutoro. Isu tinokurudzira kutamira kune yakaparadzana chengetedzo: komputa komputa, inomiririrwa neyekare kana rack maseva, inoshanda yakaparadzana, asi yakabatana neOceanStor Pacific yekuchengetedza node, iyo inoitawo mabasa avo akananga. Uye rinozvipembedza.
Semuyenzaniso, ngatitorei yekare hombe yekuchengetera data mhinduro ine hyperconverged system inogara gumi neshanu server racks. Kana iwe ukagovanisa mutoro pakati peakaparadzana komputa maseva uye OceanStor Pacific yekuchengetedza node, uchizvipatsanura kubva kune mumwe nemumwe, iyo nhamba inodiwa racks ichave nepakati! Izvi zvinoderedza dhata rekushandisa mari uye zvinoderedza mutengo wakazara wevaridzi. Munyika umo huwandu hwemashoko akachengetwa huri kukura ne15% pagore, mabhenefiti akadaro haana kukandwa.
***
Iwe unogona kuwana rumwe ruzivo nezve Huawei mhinduro uye maitiro avo ekushandisa pane yedu kana nekubata vamiriri vekambani zvakananga.
Source: www.habr.com