Lero tikambirana za momwe tingasungire bwino deta m'dziko limene maukonde amtundu wachisanu, makina opangira ma genome ndi magalimoto oyendetsa okha amapanga deta yochuluka patsiku kusiyana ndi anthu onse omwe adapangidwa asanasinthe mafakitale.
Dziko lathu lapansi likupanga chidziwitso chochulukirapo. Mbali ina yake ndi yachidule ndipo imatayika mwamsanga pamene yasonkhanitsidwa. Wina uyenera kusungidwa nthawi yayitali, ndipo ina idapangidwanso "kwa zaka mazana ambiri" - ndizo zomwe tikuwona kuyambira pano. Kuyenda kwachidziwitso kumakhazikika m'malo opangira ma data pa liwiro loti njira iliyonse yatsopano, ukadaulo uliwonse wopangidwa kuti ukwaniritse "chofuna" chosathachi chimatha msanga.
Zaka 40 za chitukuko cha machitidwe osungirako ogawidwa
Malo oyamba osungira maukonde mu mawonekedwe omwe timawadziwa adawonekera mu 1980s. Ambiri a inu mwakumana ndi NFS (Network File System), AFS (Andrew File System) kapena Coda. Zaka khumi pambuyo pake, mafashoni ndi zamakono zasintha, ndipo machitidwe ogawa mafayilo apereka njira zosungiramo zosungiramo zambiri zochokera ku GPFS (General Parallel File System), CFS (Clustered File Systems) ndi StorNext. Kusungidwa kwa block ya zomangamanga zakale kunagwiritsidwa ntchito ngati maziko, pamwamba pake pomwe fayilo imodzi idapangidwa pogwiritsa ntchito pulogalamu ya pulogalamu. Mayankho awa ndi ofanana amagwiritsidwabe ntchito, amakhala ndi niche yawo ndipo akufunika kwambiri.
Kumayambiriro kwa zaka chikwi, paradigm yosungidwa yogawidwa inasintha, ndipo machitidwe omwe ali ndi zomangamanga za SN (Shared-Nothing) adatenga malo otsogolera. Pakhala kusintha kuchokera kusungirako masango kupita kusungirako pa mfundo zaumwini, zomwe, monga lamulo, zinali ma seva akale omwe ali ndi mapulogalamu omwe amapereka zosungirako zodalirika; Pa mfundo zoterezi, nenani, HDFS (Hadoop Distributed File System) ndi GFS (Global File System) amamangidwa.
Pafupifupi zaka za m'ma 2010, malingaliro omwe amayang'aniridwa ndi makina osungira omwe amagawidwa adayamba kuwonekera pazogulitsa zonse, monga VMware vSAN, Dell EMC Isilon ndi athu. . Kumbuyo kwa nsanja zomwe zatchulidwazi kulibenso gulu la anthu okonda, koma ogulitsa enieni omwe ali ndi udindo pakugwira ntchito, chithandizo, ndi ntchito za malonda ndikutsimikiziranso chitukuko chake. Zothetsera zoterezi ndizofunikira kwambiri m'malo angapo.
Othandizira pa telecom
Mwina m'modzi mwa ogula akale kwambiri pamakina osungira omwe amagawidwa ndi ogwiritsa ntchito ma telecom. Chithunzichi chikuwonetsa magulu a mapulogalamu omwe amapanga zambiri. OSS (Operations Support Systems), MSS (Management Support Services) ndi BSS (Business Support Systems) zimayimira zigawo zitatu zowonjezera mapulogalamu zomwe zimafunika kuti zipereke chithandizo kwa olembetsa, malipoti azachuma kwa opereka chithandizo ndi chithandizo chogwira ntchito kwa akatswiri oyendetsa ntchito.
Nthawi zambiri, zidziwitso za zigawozi zimasakanizidwa kwambiri, ndipo pofuna kupewa kusonkhanitsa makope osafunika, kusungidwa kogawidwa kumagwiritsidwa ntchito, komwe kumasonkhanitsa chidziwitso chonse chochokera pa intaneti. Zosungirazo zimaphatikizidwa kukhala dziwe wamba, lomwe limapezeka ndi mautumiki onse.
Mawerengedwe athu akuwonetsa kuti kusintha kuchokera ku makina osungira akale kuti mutseke zosungirako kumakupatsani mwayi wosunga mpaka 70% ya bajeti pokhapokha mutasiya njira zosungirako zodzipatulira ndikugwiritsa ntchito ma seva odziwika bwino (nthawi zambiri x86), akugwira ntchito molumikizana ndi akatswiri apadera. mapulogalamu. Ogwiritsa ntchito ma cellular ayamba kale kugula njira zoterezi mochulukira. Makamaka, ogwira ntchito ku Russia akhala akugwiritsa ntchito zinthu zoterezi kuchokera ku Huawei kwa zaka zoposa zisanu ndi chimodzi.
Inde, ntchito zingapo sizingatheke pogwiritsa ntchito machitidwe ogawidwa. Mwachitsanzo, ndi kuchuluka kwa magwiridwe antchito kapena kugwirizana ndi ma protocol akale. Koma osachepera 70% ya deta yokonzedwa ndi wogwira ntchitoyo ikhoza kukhala mu dziwe logawidwa.
Mabanki gawo
Mu banki iliyonse pali machitidwe osiyanasiyana a IT, kuyambira pakukonza ndi kutha ndi makina osungira mabanki. Zomangamangazi zimagwiranso ntchito ndi chidziwitso chochuluka, pamene ntchito zambiri sizifuna kuwonjezeka kwa ntchito ndi kudalirika kwa machitidwe osungiramo zinthu, mwachitsanzo, chitukuko, kuyesa, makina opangira maofesi, ndi zina zotero. koma chaka chilichonse zimakhala zochepa komanso zosapindulitsa. Kuonjezera apo, mu nkhaniyi palibe kusinthasintha pakugwiritsa ntchito zipangizo zosungirako zosungirako, zomwe zimawerengedwa potengera katundu wapamwamba.
Pogwiritsira ntchito machitidwe osungirako ogawidwa, ma node awo, omwe ali ma seva wamba, akhoza kusinthidwa nthawi iliyonse, mwachitsanzo, kukhala famu ya seva ndikugwiritsidwa ntchito ngati kompyuta.
Data Lakes
Chithunzi pamwambapa chikuwonetsa mndandanda wa ogula omwe ali nawo . Izi zikhoza kukhala mautumiki a boma (mwachitsanzo, "Government Services"), makampani opanga digito, mabungwe azachuma, ndi zina zotero.
Kugwiritsa ntchito njira zakale zosungiramo zosungirako kuti athetse mavuto ngati amenewa sikuthandiza, chifukwa pamafunika mwayi wopezeka pazida zotsekera komanso mwayi wopezeka m'malaibulale a zolemba zosungidwa ngati zinthu. Mwachitsanzo, dongosolo loyitanitsa kudzera pa intaneti litha kulumikizidwanso pano. Kuti mugwiritse ntchito zonsezi pa nsanja yosungiramo zakale, mudzafunika zida zazikulu zogwirira ntchito zosiyanasiyana. Njira imodzi yopingasa yosungiramo zinthu zonse imatha kuphimba zonse zomwe zalembedwa kale: mumangofunika kupanga maiwe angapo okhala ndi mawonekedwe osiyanasiyana osungiramo.
Majenereta a chidziwitso chatsopano
Kuchuluka kwa chidziwitso chomwe chimasungidwa padziko lapansi chikukulirakulira pafupifupi 30% pachaka. Iyi ndi nkhani yabwino kwa ogulitsa osungira, koma ndi chiyani ndipo chidzakhala gwero lalikulu la deta iyi?
Zaka khumi zapitazo, malo ochezera a pa Intaneti adakhala opanga majenereta; izi zimafuna kuti pakhale ma algorithms atsopano, mayankho a hardware, ndi zina zotero. Choyamba ndi cloud computing. Pakadali pano, pafupifupi 70% yamakampani amagwiritsa ntchito ntchito zamtambo mwanjira ina. Izi zitha kukhala maimelo apakompyuta, zosunga zobwezeretsera ndi mabungwe ena owoneka bwino.
Dalaivala wachiwiri ndi maukonde a m'badwo wachisanu. Awa ndi ma liwiro atsopano komanso ma voliyumu atsopano otengera deta. Malinga ndi zolosera zathu, kufalikira kwa 5G kudzetsa kuchepa kwa kufunikira kwa makadi okumbukira flash. Ziribe kanthu kuchuluka kwa kukumbukira komwe kuli mu foni, kumathabe, ndipo ngati chidacho chili ndi njira ya 100-megabit, palibe chifukwa chosungira zithunzi kwanuko.
Gulu lachitatu lazifukwa zomwe kufunikira kwa makina osungirako kukukulirakulira kumaphatikizapo kukula mwachangu kwa luntha lochita kupanga, kusintha kwa ma analytics akulu a data komanso mayendedwe opita kuzinthu zonse zomwe zingatheke.
Mbali ya "magalimoto atsopano" ndi ake . Tiyenera kusunga deta iyi popanda kufotokoza maonekedwe ake mwanjira iliyonse. Zimangofunika pakuwerenga kotsatira. Mwachitsanzo, kuti mudziwe kuchuluka kwa ngongole yomwe ilipo, mabanki amapeza ziwongola dzanja amawona zithunzi zomwe mumayika pa malo ochezera a pa Intaneti, kudziwa ngati mumapita nthawi zambiri kunyanja ndi kumalo odyera, ndipo nthawi yomweyo phunzirani zomwe zachokera muzolemba zanu zamankhwala zomwe zilipo. ku izo. Deta iyi, kumbali imodzi, ndi yokwanira, koma kumbali inayo, ilibe homogeneity.
Nyanja ya data yosakhazikika
Kodi kutulukira kwa "zatsopano" kumabweretsa mavuto otani? Choyamba mwa izo, ndithudi, ndi kuchuluka kwa chidziwitso ndi nthawi yowerengeka ya kusungidwa kwake. Galimoto yamakono yopanda dalaivala yokha imapanga data yofikira ma 60 terabytes tsiku lililonse kuchokera ku masensa ake onse ndi makina ake. Kuti mupange ma aligorivimu atsopano, chidziwitsochi chiyenera kukonzedwa mkati mwa tsiku lomwelo, apo ayi chidzayamba kudziunjikira. Pa nthawi yomweyi, iyenera kusungidwa kwa nthawi yayitali kwambiri - zaka zambiri. Pokhapokha padzakhala zotheka m'tsogolomu kuti tipeze mfundo zochokera ku zitsanzo zazikulu zowunikira.
Chida chimodzi chodziwira momwe majini amayendera chimapanga pafupifupi 6 TB patsiku. Ndipo zomwe zasonkhanitsidwa mothandizidwa sizikutanthauza kuchotsedwa konse, ndiko kuti, mongoyerekeza, ziyenera kusungidwa kwamuyaya.
Pomaliza, maukonde m'badwo wachisanu womwewo. Kuphatikiza pa chidziwitso chenichenicho, maukonde oterowo ndi jenereta yayikulu ya data: zipika zantchito, ma rekodi oyimba, zotsatira zapakatikati zamakina a makina ndi makina, ndi zina zambiri.
Zonsezi zimafuna kukhazikitsidwa kwa njira zatsopano ndi ma aligorivimu posungira ndi kukonza zidziwitso. Ndipo njira zoterezi zikuwonekera.
Tekinoloje yatsopano yanthawi
Pali magulu atatu a mayankho omwe amapangidwa kuti athe kuthana ndi zofunikira zatsopano zamakina osungira zidziwitso: kukhazikitsidwa kwanzeru zopangira, kusinthika kwaukadaulo wama media osungiramo zinthu ndi zatsopano pazomangamanga zamakina. Tiyeni tiyambe ndi AI.
M'mayankho atsopano a Huawei, luntha lochita kupanga limagwiritsidwa ntchito pamlingo wosungirako womwewo, womwe uli ndi purosesa ya AI yomwe imalola kuti dongosololi lizisanthula mozama momwe zilili ndikulosera zolephera. Ngati makina osungiramo zinthu akugwirizanitsidwa ndi mtambo wautumiki womwe uli ndi mphamvu zazikulu zamakompyuta, luntha lochita kupanga lidzatha kukonza zambiri ndikuwonjezera kulondola kwa malingaliro ake.
Kuphatikiza pa zolephera, AI yotere imatha kulosera zam'tsogolo zam'tsogolo komanso nthawi yotsalira mpaka mphamvu itatheratu. Izi zimakuthandizani kuti muwongolere magwiridwe antchito ndikukulitsa dongosolo pasanachitike zovuta zilizonse.
Tsopano za kusinthika kwa media media. Ma drive ama flash oyamba adapangidwa pogwiritsa ntchito ukadaulo wa SLC (Single-Level Cell). Zipangizo zochokera pa izo zinali zofulumira, zodalirika, zokhazikika, koma zinali ndi mphamvu zochepa ndipo zinali zodula kwambiri. Kukula kwa voliyumu ndi kutsika kwamitengo kunapezedwa kudzera muzovomerezeka zina zaukadaulo, chifukwa chake kuthamanga, kudalirika ndi moyo wautumiki wamagalimoto zidachepetsedwa. Komabe, machitidwewa sanakhudze machitidwe osungira okha, omwe, chifukwa cha zidule zosiyanasiyana zamamangidwe, nthawi zambiri amakhala opindulitsa komanso odalirika.
Koma chifukwa chiyani mumafunikira makina osungira a All-Flash? Kodi sikunali kokwanira kungosintha ma HDD akale mumakina ogwiritsira ntchito kale ndi ma SSD atsopano amtundu womwewo? Izi zinali zofunika kuti agwiritse ntchito bwino zonse zomwe zili mumayendedwe atsopano olimba, zomwe zinali zosatheka m'machitidwe akale.
Mwachitsanzo, Huawei wapanga matekinoloje angapo kuti athetse vutoli, imodzi mwazomwe ndi , zomwe zidapangitsa kuti zitheke kukhathamiritsa kuyanjana kwa "disk-controller" momwe ndingathere.
Kuzindikiritsa mwanzeru kunapangitsa kuti zitheke kuwononga deta m'mitsinje ingapo ndikuthana ndi zochitika zingapo zosafunikira, monga (lembani kukulitsa). Nthawi yomweyo, ma aligorivimu atsopano ochira, makamaka , kuonjezera liwiro la kumanganso, kuchepetsa nthawi yake kukhala yochepa kwambiri.
Kulephera, kuchulukirachulukira, kusonkhanitsa zinyalala - zinthu izi sizikhudzanso magwiridwe antchito osungira chifukwa chakusintha kwapadera kwa owongolera.
Ndipo block data storages akukonzekera kukumana . Tikumbukenso kuti dongosolo lachikale lokonzekera kupeza deta linagwira ntchito motere: purosesa inapeza wolamulira wa RAID kudzera pa basi ya PCI Express. Izi, zimalumikizana ndi ma disks amakina kudzera pa SCSI kapena SAS. Kugwiritsiridwa ntchito kwa NVMe pa backend kwambiri kufulumizitsa ndondomeko yonseyi, koma inali ndi drawback imodzi: ma drive amayenera kulumikizidwa mwachindunji ndi purosesa kuti apereke mwayi wokumbukira.
Gawo lotsatira lachitukuko chaukadaulo chomwe tikuwona tsopano ndikugwiritsa ntchito NVMe-oF (NVMe over Fabrics). Ponena za matekinoloje a Huawei block, amathandizira kale FC-NVMe (NVMe over Fiber Channel), ndipo NVMe over RoCE (RDMA over Converged Ethernet) ili m'njira. Mitundu yoyesera imagwira ntchito bwino; kwatsala miyezi ingapo kuti awonetsedwe. Zindikirani kuti zonsezi zidzawonekera mu machitidwe ogawidwa, kumene "Ethernet yotayika" idzafunika kwambiri.
Njira yowonjezera yokwaniritsira ntchito yosungidwa yogawidwa inali kusiyidwa kwathunthu kwa magalasi a data. Mayankho a Huawei sagwiritsanso ntchito makope a n, monga mwachizolowezi RAID 1, ndikusintha kwathunthu ku (Erasure coding). Phukusi lapadera la masamu limawerengera zoletsa pa periodicity inayake, zomwe zimakulolani kuti mubwezeretse deta yapakatikati ngati mutatayika.
Njira zochepetsera komanso zopondereza zimakhala zovomerezeka. Ngati m'makina osungiramo akale timachepetsedwa ndi kuchuluka kwa mapurosesa omwe amaikidwa mu olamulira, ndiye kuti mumagawidwe osungiramo osakanikirana, node iliyonse imakhala ndi zonse zofunika: disks, memory, processors ndi interconnect. Zothandizira izi ndizokwanira kuwonetsetsa kuti kuchotsera ndi kuponderezana kumakhala ndi zotsatira zochepa pakuchita.
Komanso za njira kukhathamiritsa hardware. Apa zinali zotheka kuchepetsa katundu pa mapurosesa apakati ndi thandizo la tchipisi odzipatulira owonjezera (kapena midadada odzipereka mu purosesa yokha), yomwe imagwira ntchito. (TCP/IP Offload Engine) kapena kutenga masamu a EC, kuchotsera ndi kukakamiza.
Njira zatsopano zosungira deta zikuphatikizidwa muzomangamanga zogawanika (zogawidwa). Makina osungira apakati ali ndi fakitale ya seva yolumikizidwa kudzera pa Fiber Channel kupita ndi masanjidwe ambiri. Zoyipa za njirayi ndizovuta kukulitsa ndikuwonetsetsa kuti mulingo wotsimikizika wautumiki (mwakuchita kapena latency). Machitidwe a Hyperconverged amagwiritsa ntchito makamu omwewo posungira ndi kukonza zambiri. Izi zimapereka mwayi wopanda malire pakukulitsa, koma zimatengera mtengo wokwera posunga kukhulupirika kwa data.
Mosiyana ndi zonse zomwe zili pamwambapa, zomanga zogawanika zimatanthauza kugawaniza dongosolo mu nsalu ya computing ndi njira yosungiramo yopingasa. Izi zimapereka phindu lazomangamanga zonse ziwiri ndipo zimalola pafupifupi makulitsidwe opanda malire a chinthu chokhacho chomwe chilibe ntchito.
Kuchokera pakuphatikizana mpaka kuphatikizika
Ntchito yachikale, kufunikira kwake komwe kwakula zaka 15 zapitazi, ndikufunika kupereka nthawi imodzi yosungiramo chipika, kupeza mafayilo, kupeza zinthu, kugwiritsa ntchito famu yaikulu ya deta, ndi zina zotero. kukhala, mwachitsanzo, dongosolo kubwerera pa maginito tepi.
Pa gawo loyamba, kasamalidwe ka mautumikiwa okha ndi omwe angagwirizane. Njira zosungiramo deta zosawerengeka zinalumikizidwa ndi mapulogalamu ena apadera, omwe woyang'anira adagawira zothandizira kuchokera ku maiwe omwe alipo. Koma popeza maiwewa anali ndi zida zosiyanasiyana, kusamuka kwa katundu pakati pawo kunali kosatheka. Pakuphatikiza kwakukulu, kuphatikizika kunachitika pachipata. Ngati kugawana mafayilo kunalipo, kumatha kutumizidwa kudzera munjira zosiyanasiyana.
Njira yapamwamba kwambiri yolumikizirana yomwe ikupezeka pano kwa ife ikukhudza kupanga makina osakanizidwa padziko lonse lapansi. Ndendende zomwe zathu ziyenera kukhala . Kufikira kwa Universal kumagwiritsa ntchito zida za Hardware zomwezo, zogawika bwino m'madziwe osiyanasiyana, koma kulola kusamuka kwa katundu. Zonsezi zitha kuchitika kudzera mu imodzi mwama management console. Mwanjira imeneyi, tinatha kugwiritsa ntchito lingaliro la "malo amodzi a data - njira imodzi yosungirako."
Mtengo wosungira zidziwitso tsopano umasankha zosankha zambiri zamamangidwe. Ndipo ngakhale kuti ikhoza kuyikidwa bwino patsogolo, lero tikukambirana zosungirako "zamoyo" ndi mwayi wogwira ntchito, choncho ntchito iyeneranso kuganiziridwa. Chinthu china chofunikira cha machitidwe ogawidwa a m'badwo wotsatira ndikugwirizanitsa. Kupatula apo, palibe amene akufuna kukhala ndi machitidwe osiyanasiyana owongolera kuchokera kuzinthu zosiyanasiyana. Makhalidwe onsewa ali mu mndandanda watsopano wazinthu za Huawei .
Misa yosungirako dongosolo la m'badwo watsopano
OceanStor Pacific imakwaniritsa zofunikira zisanu ndi chimodzi zodalirika (99,9999%) ndipo ingagwiritsidwe ntchito kupanga malo opangira deta a HyperMetro. Pokhala ndi mtunda pakati pa malo awiri a data mpaka 100 km, machitidwewa amasonyeza kuti pali latency yowonjezera ya 2 ms, yomwe imapangitsa kuti pakhale njira zothetsera mavuto, kuphatikizapo omwe ali ndi ma seva a quorum.
Zogulitsa zatsopano zikuwonetsa kusinthasintha kwa ma protocol. Kale, OceanStor 100D imathandizira kulowa kwa block, kupeza zinthu ndi mwayi wa Hadoop. Kufikira kwamafayilo kudzakhazikitsidwanso posachedwa. Palibe chifukwa chosungira makope angapo a data ngati atha kuperekedwa kudzera munjira zosiyanasiyana.
Zikuwoneka kuti, lingaliro la "network yopanda kutaya" likugwirizana bwanji ndi makina osungira? Chowonadi ndi chakuti machitidwe osungiramo deta omwe amagawidwa amamangidwa pamaziko a netiweki yachangu yomwe imathandizira ma aligorivimu oyenera ndi makina a RoCE. Dongosolo lanzeru lochita kupanga lomwe limathandizidwa ndi ma switch athu limathandizira kukulitsa liwiro la netiweki ndikuchepetsa latency. . Kupindula pakusungirako mukatsegula AI Fabric kumatha kufika 20%.
Kodi malo osungira atsopano a OceanStor Pacific ndi chiyani? Yankho la 5U form factor limaphatikizapo ma drive 120 ndipo amatha kusintha ma node atatu apamwamba, omwe amapereka ndalama zopitilira kawiri mu rack space. Posasunga makope, mphamvu zamagalimoto zimakula kwambiri (mpaka + 92%).
Tidazolowera kuti kusungirako komwe kumatanthauzidwa ndi mapulogalamu ndi mapulogalamu apadera omwe amaikidwa pa seva yapamwamba. Koma tsopano, kuti mukwaniritse magawo abwino, njira yomanga iyi imafunikiranso ma node apadera. Imakhala ndi ma seva awiri kutengera ma processor a ARM omwe amayendetsa ma drive angapo mainchesi atatu.
Ma seva awa sali oyenera mayankho a hyperconverged. Choyamba, pali mapulogalamu angapo a ARM, ndipo chachiwiri, ndizovuta kusunga bwino katundu. Tikupangira kusamukira kumalo osungirako osiyana: gulu la makompyuta, loyimiridwa ndi ma seva akale kapena oyika, limagwira ntchito padera, koma limalumikizidwa ndi malo osungira a OceanStor Pacific, omwe amachitanso ntchito zawo zachindunji. Ndipo icho chimadzilungamitsa chokha.
Mwachitsanzo, tiyeni titenge njira yayikulu yosungira deta yokhala ndi ma hyperconverged system omwe amakhala ndi ma seva 15. Mukagawira katunduyo pakati pa ma seva apakompyuta osiyana ndi malo osungira a OceanStor Pacific, kuwalekanitsa wina ndi mzake, kuchuluka kwa ma racks ofunikira kuchepetsedwa! Izi zimachepetsa ndalama zogwirira ntchito za data center ndikuchepetsa mtengo wonse wa umwini. M'dziko lomwe kuchuluka kwa chidziwitso chosungidwa chikukula ndi 30% pachaka, zabwino zotere sizimaponyedwa mozungulira.
***
Mutha kudziwa zambiri za mayankho a Huawei ndi mawonekedwe awo ogwiritsira ntchito patsamba lathu kapena polumikizana ndi oyimira kampani mwachindunji.
Source: www.habr.com