A yau za mu yi magana game da yadda mafi kyawun adana bayanai a cikin duniyar da cibiyoyin sadarwa na ƙarni na biyar, na'urorin binciken kwayoyin halitta da motoci masu tuka kansu ke samar da ƙarin bayanai a cikin yini fiye da duk ɗan adam da aka samar kafin juyin juya halin masana'antu.
Duniyarmu tana samar da ƙarin bayanai. Wasu daga ciki mai wucewa ne kuma suna ɓacewa da sauri kamar yadda aka tattara. Sauran ya kamata a adana tsawon lokaci, ɗayan kuma an tsara shi gaba ɗaya "har tsawon ƙarni" - aƙalla yadda muke gani daga yanzu. Matsakaicin bayanai yana daidaitawa a cibiyoyin bayanai cikin sauri ta yadda kowace sabuwar hanya, duk wata fasaha da aka ƙera don biyan wannan "buƙata" mara iyaka tana saurin zama tsohuwa.
Shekaru 40 na ci gaban ajiya da aka rarraba
Ma'ajiyar cibiyar sadarwa ta farko a cikin hanyar da muka saba da ita sun bayyana a cikin 1980s. Yawancinku sun ci karo da NFS (Tsarin Fayil na Yanar Gizo), AFS (Tsarin Fayil na Andrew) ko Coda. Shekaru goma bayan haka, salo da fasaha sun canza, kuma tsarin fayil da aka rarraba sun ba da hanya zuwa tsarin ajiya mai tushe bisa GPFS (Tsarin Fayil na Fayil na Gabaɗaya), CFS (Clustered File Systems) da StorNext. A matsayin tushe, an yi amfani da toshe ma'ajiyar gine-gine na gargajiya, wanda a saman sa an ƙirƙiri tsarin fayil guda ɗaya ta amfani da Layer na software. Ana amfani da waɗannan da makamantan hanyoyin warware su, sun mamaye al'amuransu kuma suna cikin buƙata sosai.
A ƙarshen karni, tsarin ajiya da aka rarraba ya ɗan canza, kuma tsarin tare da tsarin gine-ginen SN (Raba-Ba komai) ya jagoranci. An sami canji daga ma'ajiyar tari zuwa ajiya akan nodes daban-daban, waɗanda, a matsayin mai mulkin, sune sabobin gargajiya tare da software wanda ke ba da ingantaccen ajiya; Irin waɗannan ka'idodin an gina su, ka ce, HDFS (Hadoop Distributed File System) da GFS (Global File System).
Kusa da 2010, dabarun da ke tattare da tsarin ajiya da aka rarraba sun ƙara fara nunawa a cikin cikakkun samfuran kasuwanci, kamar VMware vSAN, Dell EMC Isilon, da namu. . Bayan dandali da aka ambata ba wata al'umma ce ta masu sha'awa ba, amma takamaiman dillalai waɗanda ke da alhakin ayyuka, tallafi, kula da sabis na samfurin kuma suna ba da garantin ci gaba. Irin waɗannan mafita sun fi buƙata a wurare da yawa.
Ma'aikatan sadarwa
Wataƙila ɗaya daga cikin tsofaffin masu amfani da tsarin ajiya da aka rarraba sune masu gudanar da sadarwa. Jadawalin ya nuna waɗanne ƙungiyoyin aikace-aikacen ke samar da mafi yawan bayanai. OSS (Tsarin Tallafin Ayyuka), MSS (Sabis na Tallafi na Gudanarwa) da BSS (Tsarin Tallafin Kasuwanci) matakan software ne guda uku da ake buƙata don isar da sabis ga masu biyan kuɗi, bayar da rahoton kuɗi ga mai samarwa, da tallafin aiki ga injiniyoyin mai aiki.
Sau da yawa, bayanan waɗannan yadudduka suna haɗuwa da juna sosai, kuma don guje wa tarin kwafin da ba dole ba, ana amfani da ma'ajin da aka rarraba wanda ke tara dukkanin bayanan da ke fitowa daga hanyar sadarwa mai aiki. An haɗa ma'ajiyar a cikin wani tafkin gama gari, wanda duk sabis ke shiga.
Ƙididdigar mu sun nuna cewa sauyawa daga classic zuwa toshe tsarin ajiya yana ba ku damar adana har zuwa 70% na kasafin kuɗi kawai ta hanyar watsar da tsarin ajiyar hi-karshen sadaukarwa da amfani da sabobin gine-gine na al'ada (yawanci x86), aiki tare da software na musamman. Masu aiki da wayar salula sun daɗe suna samun irin waɗannan mafita a cikin adadi mai yawa. Musamman ma, kamfanonin Rasha sun yi amfani da irin waɗannan samfuran daga Huawei sama da shekaru shida.
Ee, ba za a iya yin adadin ayyuka ta amfani da tsarin rarrabawa ba. Misali, tare da ƙarin buƙatun aiki ko dacewa tare da tsoffin ka'idoji. Amma aƙalla 70% na bayanan da ma'aikatan ke aiwatarwa za a iya sanya su a cikin tafkin da aka rarraba.
Banki
A kowane banki, akwai tsarin IT iri-iri iri-iri, kama daga sarrafawa zuwa tsarin banki mai sarrafa kansa. Wannan kayan aikin kuma yana aiki tare da adadi mai yawa na bayanai, yayin da yawancin ayyukan ba sa buƙatar ƙara yawan aiki da amincin tsarin ajiya, kamar haɓakawa, gwaji, sarrafa kayan aiki na ofis, da sauransu. , amma duk shekara yana raguwa kuma yana raguwa. Bugu da ƙari, a cikin wannan yanayin, babu sassauci a cikin ciyar da albarkatun ajiya, wanda aka ƙididdige aikin da aka yi daga mafi girman nauyin.
Lokacin amfani da tsarin ajiya da aka rarraba, nodes ɗin su, waɗanda a zahiri sabar sabar na yau da kullun, ana iya canza su a kowane lokaci, misali, zuwa gonar uwar garken kuma ana amfani da su azaman dandamali na kwamfuta.
Tafkunan bayanai
Hoton da ke sama yana nuna jerin masu amfani da sabis na yau da kullun. . Waɗannan na iya zama sabis na e-gwamnati (misali, "Gosuslugi"), kamfanoni waɗanda suka yi aikin dijital, tsarin kuɗi, da sauransu. Dukkanin su suna buƙatar yin aiki tare da manyan kundin bayanai daban-daban.
Ayyukan tsarin ajiya na al'ada don magance irin waɗannan matsalolin ba su da inganci, tun da duka damar yin aiki mai girma don toshe bayanan bayanai da samun damar kai-tsaye zuwa ɗakunan karatu na takaddun bayanan da aka adana kamar yadda ake buƙatar abubuwa. Anan, alal misali, ana iya ɗaure tsarin umarni ta hanyar tashar yanar gizo. Don aiwatar da duk wannan akan dandamalin ajiya na gargajiya, kuna buƙatar babban saiti na kayan aiki don ayyuka daban-daban. Tsarin ajiya na duniya ɗaya a kwance yana iya sauƙi rufe duk ayyukan da aka jera a baya: kawai kuna buƙatar ƙirƙirar wuraren waha da yawa a ciki tare da halaye daban-daban na ajiya.
Masu samar da sabbin bayanai
Adadin bayanan da aka adana a duniya yana haɓaka da kusan 30% a kowace shekara. Wannan labari ne mai kyau ga masu sayar da ajiya, amma menene kuma zai zama babban tushen wannan bayanan?
Shekaru goma da suka wuce, cibiyoyin sadarwar jama'a sun zama irin wannan janareta, wanda ya buƙaci ƙirƙirar adadi mai yawa na sababbin algorithms, hardware mafita, da dai sauransu Yanzu akwai manyan direbobi uku na girma ajiya. Na farko shine Cloud Computing. A halin yanzu, kusan 70% na kamfanoni suna amfani da sabis na girgije ta wata hanya ko wata. Waɗannan na iya zama tsarin imel, madogara, da sauran abubuwan da aka ƙima.
Cibiyoyin sadarwa na ƙarni na biyar suna zama direba na biyu. Waɗannan sabbin saurin gudu ne da sabbin juzu'i na canja wurin bayanai. Dangane da hasashen mu, karɓar 5G da yawa zai haifar da raguwar buƙatun katunan ƙwaƙwalwar walƙiya. Komai memorin wayar da ke cikin wayar, har yanzu tana ƙarewa, kuma idan na'urar tana da tashar megabit 100, babu buƙatar adana hotuna a cikin gida.
Rukuni na uku na dalilan da ya sa buƙatun tsarin ajiya ke haɓaka sun haɗa da haɓakar haɓakar haɓakar hankali na wucin gadi, sauye-sauye zuwa manyan ƙididdigar bayanai da kuma yanayin sarrafa sarrafa kansa na duniya na duk abin da zai yiwu.
Siffar "sabon zirga-zirga" ita ce ta . Muna buƙatar adana wannan bayanan ba tare da bayyana tsarinsa ta kowace hanya ba. Ana buƙatar kawai don karatu na gaba. Misali, tsarin nuna maki na banki don tantance girman rancen da ake da shi zai duba hotunan da kuka saka a shafukan sada zumunta, yana tantance sau nawa kuke zuwa teku da gidajen cin abinci, kuma a lokaci guda nazarin abubuwan da aka samo daga takaddun likitan ku da ke akwai. Wadannan bayanai, a daya bangaren, suna da cikakkun bayanai, kuma a daya bangaren, ba su da kamanni.
Tekun bayanan da ba a tsara su ba
Menene matsalolin da fitowar "sabbin bayanai" ya kunsa? Na farko a cikinsu, ba shakka, shi ne adadin bayanan da kansa da kuma kiyasin lokacin ajiyarsa. Mota mai cin gashin kanta ta zamani ita kaɗai tana samar da bayanai har zuwa 60 TB a kowace rana daga duk na'urori masu auna firikwensin da tsarinta. Don haɓaka sabon algorithms motsi, dole ne a sarrafa wannan bayanin a cikin rana ɗaya, in ba haka ba zai fara tarawa. A lokaci guda, ya kamata a adana shi na dogon lokaci - shekarun da suka gabata. Sa'an nan ne kawai za a iya yanke shawara bisa manyan samfuran nazari a nan gaba.
Na'urar guda ɗaya don tantance jerin kwayoyin halitta tana samar da kusan terabytes 6 kowace rana. Kuma bayanan da aka tattara tare da taimakonsa ba ya nufin gogewa kwata-kwata, wato, a zahiri, ya kamata a adana su har abada.
A ƙarshe, duk cibiyoyin sadarwa iri ɗaya na ƙarni na biyar. Baya ga bayanan da kanta ke watsawa, irin wannan hanyar sadarwar ita kanta babbar janareta ce: rajistan ayyukan, bayanan kira, sakamakon tsaka-tsaki na hulɗar na'ura zuwa na'ura, da sauransu.
Duk wannan yana buƙatar haɓaka sabbin hanyoyi da algorithms don adanawa da sarrafa bayanai. Kuma irin wadannan hanyoyin suna tasowa.
Fasaha na sabon zamani
Ƙungiyoyi uku na mafita waɗanda aka tsara don magance sababbin buƙatun don tsarin adana bayanai za a iya bambanta: gabatarwar basirar wucin gadi, juyin fasaha na kafofin watsa labaru na ajiya, da sababbin abubuwa a cikin tsarin gine-ginen tsarin. Bari mu fara da AI.
A cikin sababbin hanyoyin Huawei, an riga an yi amfani da basirar wucin gadi a matakin ajiyar kanta, wanda aka sanye da na'ura mai sarrafa AI wanda ke ba da damar tsarin don nazarin yanayinsa da kansa da kuma hasashen gazawar. Idan tsarin ajiya yana da alaƙa da gajimaren sabis wanda ke da mahimmin damar kwamfuta, hankali na wucin gadi zai iya aiwatar da ƙarin bayani kuma ya inganta daidaiton hasashen sa.
Bugu da ƙari ga gazawar, irin wannan AI yana iya yin tsinkaya babban nauyin gaba na gaba da kuma lokacin da ya rage har sai ƙarfin ya ƙare. Wannan yana ba ku damar haɓaka aiki da sikelin tsarin kafin duk abubuwan da ba'a so su faru.
Yanzu game da juyin halitta na masu ɗaukar bayanai. An yi filasha ta farko ta amfani da fasahar SLC (Single-Level Cell). Na'urorin da aka dogara da su sun kasance masu sauri, abin dogara, kwanciyar hankali, amma suna da ƙananan ƙarfi kuma suna da tsada sosai. An samu karuwar girma da raguwar farashin ta hanyar wasu fasahohin fasaha, saboda abin da aka rage gudu, aminci da rayuwar tafiyarwa. Duk da haka, yanayin bai shafi tsarin ajiya da kansu ba, wanda, saboda daban-daban dabaru na gine-gine, a gaba ɗaya, ya zama duka biyu mafi inganci kuma mafi aminci.
Amma me yasa kuke buƙatar tsarin ajiya ajin All-Flash? Shin bai isa kawai don maye gurbin tsoffin HDDs a cikin tsarin da ya riga ya gudana tare da sabbin SSDs na nau'i iri ɗaya ba? Ana buƙatar wannan don yin amfani da ingantaccen amfani da duk albarkatun sabbin SSDs, wanda ba zai yiwu ba a cikin tsofaffin tsarin.
Misali Huawei ya samar da fasahohi da dama don magance wannan matsala, daya daga cikinsu shine , wanda ya sa ya yiwu a inganta hulɗar masu sarrafa faifai kamar yadda zai yiwu.
Ganewa na hankali ya ba da damar rarraba bayanai zuwa rafuka da yawa da kuma jimre da yawan abubuwan da ba a so, kamar su. (rubuta ƙarawa). A lokaci guda, sabon dawo da algorithms, musamman , ya ƙara saurin sake ginawa, yana rage lokacinsa zuwa ƙimar da ba ta da mahimmanci.
Rashin gazawa, cunkoson jama'a, tarin datti - waɗannan abubuwan kuma ba sa yin tasiri kan aikin tsarin ajiya saboda gyare-gyare na musamman na masu sarrafawa.
Kuma wuraren ajiyar bayanai suna shirye-shiryen saduwa . Ka tuna cewa tsarin da ya dace don tsara hanyar samun bayanai ya yi aiki kamar haka: mai sarrafawa ya sami dama ga mai sarrafa RAID ta hanyar bas ɗin PCI Express. Wannan, bi da bi, yana hulɗa tare da faifan inji ta hanyar SCSI ko SAS. Amfani da NVMe akan bangon baya yana haɓaka aikin gabaɗaya, amma yana ɗaukar koma baya: dole ne a haɗa faifai kai tsaye zuwa na'urar don samar da shi ta hanyar samun damar ƙwaƙwalwar ajiya kai tsaye.
Mataki na gaba na ci gaban fasaha da muke gani yanzu shine amfani da NVMe-oF (NVMe akan Fabrics). Dangane da fasahar toshe Huawei, sun riga sun goyi bayan FC-NVMe (NVMe akan Tashar Fiber), kuma NVMe akan RoCE (RDMA akan Haɗin Ethernet) yana kan hanya. Samfuran gwajin suna aiki sosai, 'yan watanni sun rage kafin gabatar da su a hukumance. Lura cewa duk wannan kuma zai bayyana a cikin tsarin da aka rarraba, inda "Ethernet ba tare da asara" zai kasance cikin buƙata mai girma ba.
Ƙarin hanyar da za a inganta aikin ma'ajiyar da aka rarraba ita ce cikakkiyar kin amincewa da madubin bayanai. Maganin Huawei ba sa amfani da kwafin n, kamar yadda yake a cikin RAID 1 na yau da kullun, kuma gaba ɗaya ya canza zuwa injin (Goge codeing). Kunshin lissafi na musamman yana ƙididdige tubalan sarrafawa tare da takamaiman mitar, wanda ke ba ku damar dawo da bayanan tsaka-tsaki idan aka yi hasara.
Hanyoyin ƙetare da matsawa sun zama wajibi. Idan a cikin tsarin ajiya na gargajiya muna iyakance ta adadin masu sarrafawa da aka shigar a cikin masu sarrafawa, to, a cikin tsarin ajiya mai ma'auni mai rarraba a kwance, kowane kumburi yana ƙunshe da duk abin da kuke buƙata: diski, ƙwaƙwalwar ajiya, masu sarrafawa da haɗin kai. Waɗannan albarkatun sun isa don ƙaddamarwa da matsawa don yin tasiri kaɗan akan aiki.
Kuma game da hanyoyin inganta kayan aiki. Anan, yana yiwuwa a rage nauyi a kan na'urori masu sarrafawa ta tsakiya tare da taimakon ƙarin microcircuits kwazo (ko tubalan da aka keɓe a cikin injin ɗin kanta), waɗanda ke taka rawa. (TCP/IP Offload Engine) ko ɗaukar ayyukan lissafi na EC, ƙaddamarwa, da matsawa.
Sabbin hanyoyin da za a adana bayanai suna kunshe a cikin tsarin gine-ginen da aka rarraba (rabawa). A cikin tsarin ajiya na tsakiya, akwai masana'antar uwar garken da aka haɗa ta tashar Fiber zuwa tare da adadi mai yawa. Rashin lahani na wannan hanyar shine matsaloli tare da ƙima da samar da ingantaccen matakin sabis (dangane da aiki ko jinkiri). Tsarukan da aka haɗu da hyperconverged suna amfani da runduna iri ɗaya don duka ajiya da sarrafa bayanai. Wannan yana ba da kusan iyaka mara iyaka don ƙira, amma yana haifar da tsada mai tsada don kiyaye amincin bayanai.
Ba kamar duka na sama ba, ƙaƙƙarfan gine-gine yana nufin raba tsarin zuwa masana'antar lissafi da tsarin ajiya a kwance. Wannan yana ba da fa'idodin gine-ginen duka biyun kuma yana ba da damar ƙima mara iyaka kawai na kashi wanda aikinsa bai isa ba.
Daga haɗin kai zuwa haɗuwa
A classic aiki, da dacewa da wanda kawai girma a cikin past 15 shekaru, shi ne bukatar lokaci guda samar da block ajiya, fayil damar yin amfani da abubuwa, da aiki na gona don babban bayanai, da dai sauransu The icing a kan cake iya. Har ila yau, zama, misali, tsarin ajiya zuwa tef ɗin maganadisu.
A mataki na farko, gudanarwar waɗannan ayyuka ne kaɗai za a iya haɗa kai. An rufe tsarin adana bayanai iri-iri ga wasu ƙwararrun software, ta inda mai gudanarwa ke rarraba albarkatu daga wuraren tafki. Amma tun da waɗannan wuraren tafki sun bambanta a cikin kayan aiki, ƙaura da nauyi a tsakanin su ba zai yiwu ba. A matsayi mafi girma na haɗin kai, ƙarfafawa ya faru a matakin ƙofar. Idan akwai hanyar shiga fayil ɗin da aka raba, ana iya ba da shi ta hanyar ladabi daban-daban.
Hanyar haɗin kai mafi ci gaba da ke akwai a gare mu yanzu ta ƙunshi ƙirƙirar tsarin haɗaɗɗiyar duniya. Kamar yadda ya kamata namu ya kasance . Samun dama ga duniya yana amfani da albarkatun kayan masarufi iri ɗaya, a hankali an raba su zuwa wuraren tafki daban-daban, amma yana ba da izinin ƙaura. Ana iya yin duk wannan ta hanyar na'ura mai sarrafa kwamfuta guda ɗaya. Ta wannan hanyar, mun gudanar da aiwatar da manufar "cibiyar bayanai ɗaya - tsarin ajiya ɗaya."
Kudin ajiyar bayanai yanzu yana ƙayyade yawancin yanke shawara na gine-gine. Kuma kodayake ana iya sanya shi cikin aminci a sahun gaba, muna tattaunawa akan ajiyar "rayuwa" tare da samun damar aiki a yau, don haka dole ne a yi la'akari da aikin. Wani muhimmin dukiya na tsarin rarraba na gaba mai zuwa shine haɗin kai. Bayan haka, babu wanda yake son samun tsarin tsarin daban-daban da aka sarrafa daga na'urori daban-daban. Duk waɗannan halayen suna cikin sabon jerin samfuran Huawei. .
Ma'ajiyar taro na gaba na gaba
OceanStor Pacific ya hadu da nine shida (99,9999%) abubuwan dogaro kuma ana iya amfani dashi don ƙirƙirar cibiyar bayanan aji na HyperMetro. Tare da nisa tsakanin cibiyoyin bayanai guda biyu har zuwa kilomita 100, tsarin yana nuna ƙarin jinkiri na 2 ms, wanda ya sa ya yiwu a gina duk wata hanyar da za ta tabbatar da bala'i dangane da su, ciki har da waɗanda ke da sabar quorum.
Samfuran sabon jerin suna nuna versatility cikin sharuddan ladabi. Tuni, OceanStor 100D yana goyan bayan toshe damar shiga, damar abu, da damar Hadoop. Za a aiwatar da hanyar shiga fayil nan gaba kadan. Babu buƙatar adana kwafi da yawa na bayanan idan ana iya fitar da su ta hanyar ka'idoji daban-daban.
Zai zama alama, menene manufar "cibiyar sadarwa mara hasara" tana da alaƙa da ajiya? Gaskiyar ita ce, an gina tsarin ajiya mai rarraba akan tushen hanyar sadarwa mai sauri wanda ke goyan bayan algorithms masu dacewa da tsarin RoCE. Tsarin hankali na wucin gadi wanda ke tallafawa masu sauya mu yana taimakawa don ƙara haɓaka saurin cibiyar sadarwa da rage jinkiri. . Ribar aikin tsarin ajiya lokacin da aka kunna AI Fabric zai iya kaiwa 20%.
Menene sabon kullin ajiya da aka rarraba na OceanStor Pacific? Maganin nau'in nau'in nau'in 5U ya haɗa da tafiyarwa 120 kuma yana iya maye gurbin nodes na gargajiya guda uku, fiye da ninka sararin tara. Saboda ƙin adana kwafin, ingancin tuƙi yana ƙaruwa sosai (har zuwa + 92%).
An yi amfani da mu don gaskiyar cewa ma'auni na software software ce ta musamman da aka sanya akan sabar gargajiya. Amma yanzu, don cimma ingantattun sigogi, wannan tsarin ginin kuma yana buƙatar nodes na musamman. Ya ƙunshi sabobin guda biyu dangane da na'urori masu sarrafa ARM waɗanda ke sarrafa tsararrun inci uku.
Waɗannan sabar ba su dace da hanyoyin magance rikice-rikice ba. Da fari dai, akwai 'yan aikace-aikace don ARM, kuma na biyu, yana da wuya a kula da ma'aunin nauyi. Muna ba da shawarar canjawa zuwa keɓance ma'adana: gungu na kwamfuta, wanda ke wakilta ta sabar sabar gargajiya ko rack, tana aiki daban, amma an haɗa shi da nodes ɗin ajiya na OceanStor Pacific, waɗanda kuma suke yin ayyukansu kai tsaye. Kuma yana baratar da kansa.
Misali, bari mu dauki wani babban bayani mai cike da rudani wanda ya mamaye rumbun sabar 15. Idan kun rarraba nauyin tsakanin ɗayan ɗayan OceanStor Pacific na lissafin sabar da nodes ɗin ajiya, raba su da juna, adadin raƙuman da ake buƙata za su ragu! Wannan yana rage farashin sarrafa cibiyar bayanai kuma yana rage jimillar kuɗin mallakar. A cikin duniyar da adadin bayanan da aka adana ke girma a 30% a kowace shekara, irin waɗannan fa'idodin ba su warwatse ba.
***
Don ƙarin bayani game da mafita na Huawei da yanayin aikace-aikacen su, da fatan za a ziyarci mu ko kuma ta hanyar tuntuɓar wakilan kamfanin kai tsaye.
source: www.habr.com