Hello Habr! Ma seti a data a Big Data ndi kuphunzira pamakina akukula kwambiri ndipo tikuyenera kupitiliza nawo. Zolemba zathu zokhudzana ndiukadaulo wina wazopanga zamakompyuta apamwamba kwambiri (HPC, High Performance Computing), zowonetsedwa pabwalo la Kingston ku . Uku ndikugwiritsira ntchito Hi-End data storage systems (SDS) mu maseva okhala ndi graphic processing units (GPU) ndi GPUDirect Storage bus technology. Chifukwa cha kusinthanitsa kwachindunji kwa data pakati pa makina osungira ndi GPU, kudutsa CPU, kutsitsa deta mu ma accelerator a GPU kumachulukitsidwa ndi dongosolo la ukulu, kotero kuti ntchito za Big Data zimathamanga kwambiri zomwe ma GPU amapereka. Momwemonso, opanga makina a HPC ali ndi chidwi ndi kupita patsogolo kwa makina osungira omwe ali ndi liwiro lapamwamba kwambiri la I / O, monga omwe amapangidwa ndi Kingston.
Kuchita kwa GPU kumaposa kutsitsa kwa data
Popeza CUDA, makina opangira ma GPU opangidwa ndi GPU ndi mapulogalamu ofananirako opangira ntchito zanthawi zonse, adapangidwa mu 2007, kuthekera kwa Hardware kwa ma GPU pawokha kwakula modabwitsa. Masiku ano, ma GPU akugwiritsidwa ntchito kwambiri muzinthu za HPC monga Big Data, kuphunzira makina (ML), ndi kuphunzira mozama (DL).
Zindikirani kuti ngakhale kufanana kwa mawu, awiri omalizira ndi ntchito zosiyana mwadongosolo. ML imaphunzitsa makompyuta kutengera deta yokhazikika, pamene DL imaphunzitsa makompyuta potengera ndemanga zochokera ku neural network. Chitsanzo chothandizira kumvetsetsa kusiyana kwake ndi kosavuta. Tiyeni tiyerekeze kuti kompyuta iyenera kusiyanitsa zithunzi za amphaka ndi agalu omwe amanyamula kuchokera ku yosungirako. Kwa ML, muyenera kutumiza zithunzi zokhala ndi ma tag ambiri, chilichonse chomwe chimatanthawuza chinthu chimodzi cha nyama. Kwa DL, ndikokwanira kukweza zithunzi zambiri, koma ndi tag imodzi yokha "uyu ndi mphaka" kapena "uyu ndi galu". DL ndi yofanana kwambiri ndi momwe ana aang'ono amaphunzitsidwira - amangowonetsedwa zithunzi za agalu ndi amphaka m'mabuku ndi m'moyo (nthawi zambiri, popanda ngakhale kufotokoza kusiyana kwakukulu), ndipo ubongo wa mwanayo umayamba kudziwa mtundu wa nyama pambuyo pake. chiwerengero chovuta cha zithunzi kuyerekeza ( Malinga ndi kuyerekezera, tikukamba za zana kapena ziwiri ziwonetsero mu ubwana wonse). Ma algorithms a DL sanakwaniritsidwebe: kuti neural network igwirenso ntchito bwino pakuzindikiritsa zithunzi, ndikofunikira kudyetsa ndi kukonza mamiliyoni a zithunzi mu GPU.
Chidule cha mawu oyambira: kutengera ma GPU, mutha kupanga mapulogalamu a HPC m'munda wa Big Data, ML ndi DL, koma pali vuto - ma seti a data ndi akulu kwambiri kotero kuti nthawi yomwe idagwiritsidwa ntchito kutsitsa deta kuchokera ku yosungirako kupita ku GPU. akuyamba kuchepetsa ntchito yonse ya ntchito. Mwanjira ina, ma GPU othamanga amakhalabe osagwiritsidwa ntchito pang'onopang'ono chifukwa cha data yapang'onopang'ono ya I/O yochokera kuzinthu zina. Kusiyanitsa kwa liwiro la I/O la GPU ndi basi kupita ku CPU / kachitidwe kosungirako kumatha kukhala dongosolo la kukula.
Kodi ukadaulo wa GPUDirect Storage umagwira ntchito bwanji?
Njira ya I/O imayang'aniridwa ndi CPU, monganso njira yotsitsa deta kuchokera ku yosungirako kupita ku GPUs kuti ipitirire. Izi zidapangitsa kuti pempho laukadaulo lomwe lingapereke mwayi wolumikizana mwachindunji pakati pa ma GPU ndi ma drive a NVMe kuti azilumikizana mwachangu. NVIDIA anali woyamba kupereka ukadaulo wotere ndipo adautcha GPUDirect Storage. M'malo mwake, uku ndikusintha kwaukadaulo wa GPUDirect RDMA (Remote Direct Memory Address) omwe adapanga kale.
Jensen Huang, CEO wa NVIDIA, awonetsa GPUDirect Storage ngati mtundu wa GPUDirect RDMA ku SC-19. Gwero: NVIDIA
Kusiyana pakati pa GPUDirect RDMA ndi GPUDirect Storage kuli pazida zomwe ma adilesi amachitikira. Ukadaulo wa GPUDirect RDMA wakonzedwanso kuti usunthire deta mwachindunji pakati pa makadi akutsogolo a netiweki (NIC) ndi kukumbukira kwa GPU, ndipo GPUDirect Storage imapereka njira yachindunji pakati pa zosungirako zakumalo kapena zakutali monga NVMe kapena NVMe over Fabric (NVMe-oF) ndi GPU kukumbukira.
Onse awiri a GPUDirect RDMA ndi GPUDirect Storage amapewa kusuntha kosafunikira kwa data kudzera mu buffer mu kukumbukira kwa CPU ndikulola makina ofikira (DMA) kuti asunthire deta kuchokera pa netiweki khadi kapena posungira molunjika kapena kuchokera kukumbukira GPU - zonse popanda katundu pakatikati CPU. Kwa GPUDirect Storage, malo osungira alibe kanthu: ikhoza kukhala NVME disk mkati mwa GPU unit, mkati mwa choyikapo, kapena yolumikizidwa pa netiweki monga NVMe-oF.
Ndondomeko ya ntchito ya GPUDirect Storage. Gwero: NVIDIA
Makina osungira a Hi-End pa NVMe akufunika pamsika wa HPC
Pozindikira kuti pakubwera GPUDirect Storage, chidwi cha makasitomala akuluakulu chidzakopeka ndikupereka makina osungira omwe ali ndi liwiro la I / O lofanana ndi momwe GPU ikuyendera, pachiwonetsero cha SC-19 Kingston adawonetsa chiwonetsero cha dongosolo lopangidwa ndi makina osungira otengera ma disks a NVMe ndi gawo lokhala ndi GPU, lomwe limasanthula masauzande azithunzi za satellite pamphindikati. Talemba kale za makina osungira oterowo kutengera ma drive 10 DC1000M U.2 NVMe .
Dongosolo losungirako lochokera pa 10 DC1000M U.2 NVMe ma drive amakwaniritsa mokwanira seva yokhala ndi ma graphic accelerators. Gwero: Kingston
Dongosolo losungirali limapangidwa ngati 1U kapena rack unit yayikulu ndipo imatha kuwonjezedwa kutengera kuchuluka kwa ma drive a DC1000M U.2 NVMe, iliyonse ili ndi mphamvu ya 3.84-7.68 TB. DC1000M ndiye mtundu woyamba wa NVMe SSD mu mawonekedwe a U.2 mu mzere wa Kingston wama data center. Ili ndi kupirira (DWPD, Drive imalemba patsiku), kulola kuti ilembenso deta kuti ikwaniritsidwe kamodzi patsiku kuti ikhale ndi moyo wotsimikizika wagalimoto.
Mu mayeso a fio v3.13 pa Ubuntu 18.04.3 LTS opareting'i sisitimu, Linux kernel 5.0.0-31-generic, chitsanzo chosungiramo chiwonetsero chinawonetsa liwiro la kuwerenga (Sustained Read) la 5.8 miliyoni IOPS yokhala ndi kutulutsa kokhazikika (Sustained Bandwidth). 23.8 Gbit / s.
Ariel Perez, woyang'anira bizinesi ya SSD ku Kingston, adanena za machitidwe atsopano osungiramo zinthu: "Tili okonzeka kukonzekeretsa mbadwo wotsatira wa ma seva ndi U.2 NVMe SSD zothetsera kuthetsa mavuto ambiri otumizira deta omwe akhala akugwirizana ndi kusunga. Kuphatikiza kwa ma drive a NVMe SSD ndi premium Server Premier DRAM yathu kumapangitsa Kingston kukhala m'modzi mwa opereka mayankho omaliza mpaka-mapeto pamakampani.
Mayeso a gfio v3.13 adawonetsa kutulutsa kwa 23.8 Gbps pamakina osungira owonera pamagetsi a DC1000M U.2 NVMe. Gwero: Kingston
Kodi machitidwe amtundu wa HPC angawoneke bwanji pogwiritsa ntchito GPUDirect Storage kapena ukadaulo wofananira? Izi ndi zomangamanga zomwe zimakhala ndi mayunitsi olekanitsa mkati mwa choyikamo: gawo limodzi kapena awiri a RAM, ena angapo a GPU ndi CPU computing node, ndi gawo limodzi kapena zingapo zosungirako.
Ndi chilengezo cha GPUDirect Storage komanso kutheka kwa matekinoloje ofananawo kuchokera kwa ogulitsa ma GPU ena, kufuna kwa Kingston kwa makina osungira opangidwa kuti azigwiritsidwa ntchito pamakompyuta ochita bwino kwambiri kukukulirakulira. Cholembacho chidzakhala liwiro la kuwerenga deta kuchokera kumalo osungiramo zinthu, kufanana ndi kutulutsa kwa makadi a 40- kapena 100-Gbit pa khomo la chigawo cha computing ndi GPU. Chifukwa chake, makina osungirako othamanga kwambiri, kuphatikiza NVMe yakunja kudzera pa Nsalu, adzachoka pakukhala achilendo kupita kuzinthu zambiri zamapulogalamu a HPC. Kuphatikiza pa kuwerengera kwa sayansi ndi zachuma, apeza ntchito m'malo ena ambiri othandiza, monga machitidwe achitetezo ku Safe City metropolitan level kapena malo oyang'anira mayendedwe, komwe kuzindikira ndi kuzindikiritsa kuthamanga kwa mamiliyoni a zithunzi za HD pamphindikati zimafunikira, "adatero. kagawo kakang'ono kamsika kapamwamba ka Storage System
Zambiri zazinthu za Kingston zitha kupezeka pa kampani.
Source: www.habr.com