Molo Habr! Iiseti zedatha yeDatha enkulu kunye nokufunda koomatshini zikhula ngokukhawuleza kwaye kufuneka siqhubeke nazo. Isithuba sethu malunga nenye itekhnoloji entsha kwicandelo lekhompyuter yokusebenza okuphezulu (i-HPC, i-High Performance Computing), eboniswe kwi-Kingston booth . Oku kukusetyenziswa kweenkqubo zokugcinwa kwedatha ye-Hi-End (SDS) kwiiseva ezineeyunithi zegraphic processing (GPU) kunye ne-GPUDirect Storage bus technology. Ndiyabulela utshintshiselwano lwedatha ngokuthe ngqo phakathi kwenkqubo yokugcina kunye ne-GPU, ukudlula i-CPU, ukulayisha idatha kwi-accelerator ye-GPU ikhawuleza ngomyalelo wobukhulu, ngoko ke usetyenziso lweDatha enkulu lusebenza ngowona msebenzi uphezulu obonelelwa yi-GPU. Emva koko, abaphuhlisi benkqubo ye-HPC banomdla kwinkqubela phambili kwiinkqubo zokugcina ezinesantya esiphezulu se-I / O, njengalezo eziveliswe nguKingston.
Ukusebenza kweGPU kugqithise ukulayisha idatha
Ukusukela ukuba i-CUDA, i-GPU-based hardware kunye nesoftware parallel computing architecture yokuphuhlisa usetyenziso lwenjongo jikelele, yadalwa ngo-2007, ubunakho behardware bee-GPU ngokwazo bukhule ngendlela emangalisayo. Namhlanje, ii-GPU zisetyenziswa kakhulu kwizicelo ze-HPC ezifana neDatha enkulu, ukufunda ngomatshini (ML), kunye nokufunda okunzulu (DL).
Qaphela ukuba ngaphandle kokufana kwamagama, ezimbini zokugqibela ziyimisebenzi eyahlukileyo ngokwe-algorithmically. I-ML iqeqesha ikhomputha ngokusekelwe kwidatha ehleliweyo, ngelixa i-DL iqeqesha ikhomputha ngokusekelwe kwimpendulo evela kwinethiwekhi ye-neural. Umzekelo wokunceda ukuqonda umahluko ulula kakhulu. Makhe sicinge ukuba ikhompyuter kufuneka yahlule phakathi kweefoto zeekati kunye nezinja ezilayishwe kwinkqubo yokugcina. Kwi-ML, kufuneka ungenise iseti yemifanekiso enethegi ezininzi, nganye kuzo ichaza uphawu olulodwa lwesilwanyana. Kwi-DL, kwanele ukulayisha inani elikhulu kakhulu lemifanekiso, kodwa ngethegi enye "le yikati" okanye "le yinja". I-DL ifana kakhulu nendlela abafundiswa ngayo abantwana abancinci - baboniswa nje imifanekiso yezinja kunye neekati ezincwadini nasebomini (izihlandlo ezininzi, ngaphandle kokuchaza umahluko ocacileyo), kwaye ingqondo yomntwana ngokwayo iqala ukugqiba uhlobo lwesilwanyana emva kokuzalwa. inani elithile elibalulekileyo lemifanekiso yokuthelekisa (Ngokoqikelelo, sithetha ngekhulu okanye ezimbini kuphela imiboniso ebuntwaneni bokuqala). Ii-algorithms ze-DL azikagqibeleli kangako: ukuze inethiwekhi ye-neural nayo isebenze ngempumelelo ekuchongeni imifanekiso, kuyafuneka ukondla kunye nokwenza izigidi zemifanekiso kwiGPU.
Isishwankathelo sentshayelelo: ngokusekelwe kwi-GPUs, unokwakha izicelo ze-HPC kwintsimi yeDatha enkulu, i-ML kunye ne-DL, kodwa kukho ingxaki - iiseti zedatha zikhulu kangangokuba ixesha elichithwe ukulayisha idatha ukusuka kwinkqubo yokugcina kwi-GPU. iqala ukunciphisa ukusebenza ngokubanzi kwesicelo. Ngamanye amagama, ii-GPU ezikhawulezayo zihlala zingasetyenziswanga ngenxa yokucotha kwedatha ye-I/O evela kwezinye ii-subsystems. Umahluko kwisantya se-I / O se-GPU kunye nebhasi kwi-CPU / inkqubo yokugcina ingaba ngumyalelo wobukhulu.
Isebenza njani iteknoloji yoGcino lweGPUDirect?
Inkqubo ye-I / O ilawulwa yi-CPU, njengenkqubo yokulayisha idatha ukusuka kwisitoreji ukuya kwi-GPU yokuqhubela phambili. Oku kukhokelele kwisicelo setekhnoloji eya kubonelela ukufikelela ngokuthe ngqo phakathi kwe-GPUs kunye ne-NVMe drives ukunxibelelana ngokukhawuleza omnye nomnye. I-NVIDIA yaba yeyokuqala ukubonelela ngetekhnoloji enjalo kwaye wayibiza ngokuba yi-GPUDirect Storage. Ngapha koko, olu lutshintsho lwetekhnoloji ye-GPUDirect RDMA (idilesi ethe ngqo yeMemori) abayiphuhlise ngaphambili.
UJensen Huang, i-CEO ye-NVIDIA, uza kubonisa i-GPUDirect Storage njengokwahluka kwe-GPUDirect RDMA kwi-SC-19. Umthombo: NVIDIA
Umahluko phakathi kwe-GPUDirect RDMA kunye ne-GPUDirect Storage kwizixhobo phakathi kwedilesi eyenziwayo. Ubuchwephesha be-GPUDirect RDMA buyinjongo yokuhambisa idatha ngokuthe ngqo phakathi kwekhadi lojongano lwenethiwekhi yangaphambili (NIC) kunye nememori ye-GPU, kwaye i-GPUDirect Storage ibonelela ngendlela ethe ngqo yedatha phakathi kogcino lwasekhaya okanye olukude olufana neNVMe okanye iNVMe ngaphezulu kweFabric (NVMe-oF) kunye Imemori yeGPU.
Zombini i-GPUDirect RDMA kunye ne-GPUDirect Storage zinqanda ukunyakaza kwedatha ngokungeyomfuneko nge-buffer kwimemori ye-CPU kwaye zivumele ufikelelo lwememori ethe ngqo (i-DMA) indlela yokuhambisa idatha ukusuka kwikhadi lenethiwekhi okanye ukugcinwa ngokuthe ngqo ukuya okanye kwimemori ye-GPU - konke ngaphandle komthwalo kumbindi we-CPU. Kwi-GPUDirect Storage, indawo yokugcina ayinamsebenzi: ingaba yidiski ye-NVME ngaphakathi kwiyunithi ye-GPU, ngaphakathi kwe-rack, okanye iqhagamshelwe kwinethiwekhi njenge-NVMe-oF.
Inkqubo yokusebenza yeGPUDirect Storage. Umthombo: NVIDIA
Iinkqubo zokugcina ze-Hi-End kwi-NVMe zifunwa kwimarike yesicelo se-HPC
Ukuqonda ukuba ngokufika kwe-GPUDirect Storage, umdla wabathengi abakhulu baya kutsalwa ekunikezeni iinkqubo zokugcina ezinezantya ze-I / O ezihambelana ne-GPU, kumboniso we-SC-19 u-Kingston ubonise idemo yenkqubo ebandakanya inkqubo yokugcina esekelwe kwiidiski ze-NVMe kunye neyunithi ene-GPU, ehlalutye amawaka emifanekiso yesathelayithi ngomzuzwana. Sele sibhale malunga nenkqubo yokugcina enjalo esekwe kwi-10 DC1000M U.2 NVMe drives. .
Inkqubo yokugcina esekelwe kwi-10 DC1000M U.2 NVMe drives incedisa ngokufanelekileyo iseva kunye ne-accelerators yemizobo. Umthombo: Kingston
Le nkqubo yokugcina yenzelwe njenge-1U okanye iyunithi ye-rack enkulu kwaye inokulinganiswa ngokuxhomekeke kwinani le-DC1000M U.2 NVMe drives, nganye inomthamo we-3.84-7.68 TB. I-DC1000M yimodeli yokuqala ye-NVMe SSD kwi-U.2 yefom yefom kwi-Kingston's line of data center drives. Inomlinganiselo wokunyamezela (i-DWPD, i-Drive ibhala ngosuku), ivumela ukuba iphinde ibhale idatha kwisakhono sayo esipheleleyo kanye ngosuku kubomi obuqinisekisiweyo bokuqhuba.
Kuvavanyo lwe-fio v3.13 kwi-Ubuntu 18.04.3 LTS inkqubo yokusebenza, i-Linux kernel 5.0.0-31-generic, isampuli yokugcina umboniso ibonise isantya sokufunda (Ukufunda okuQinisekisiweyo) kwe-5.8 yezigidi ze-IOPS kunye ne- throughput ezinzileyo (iBandwidth eqhubekayo ) ye-23.8 Gbit/s.
U-Ariel Perez, umphathi wezoshishino we-SSD e-Kingston, uthe malunga neenkqubo ezintsha zokugcina: "Sikulungele ukuxhobisa isizukulwana esilandelayo samaseva kunye ne-U.2 NVMe SSD isisombululo sokuphelisa ezininzi iibhotile zokudluliswa kwedatha eziye zadibaniswa ngokuqhelekileyo kunye nokugcinwa. Ukudityaniswa kwe-NVMe SSD drives kunye ne-premium Server yethu yeNkulumbuso i-DRAM yenza i-Kingston ibe ngomnye wababoneleli bezisombululo zedatha yokuphela kokuphela kweshishini.
Uvavanyo lwe-gfio v3.13 lubonise i-throughput ye-23.8 Gbps yenkqubo yokugcina i-demo kwi-DC1000M U.2 NVMe drives. Umthombo: Kingston
Ingaba inkqubo eqhelekileyo yezicelo ze-HPC ijongeka njani usebenzisa i-GPUDirect Storage okanye iteknoloji efanayo? Olu luyilo olunolwahlulo olubonakalayo lweeyunithi ezisebenzayo ngaphakathi kwe-rack: iyunithi enye okanye ezimbini ze-RAM, ezininzi ngakumbi ze-GPU kunye ne-CPU zekhompiyutha node, kunye neyunithi enye okanye ngaphezulu kwiinkqubo zokugcina.
Ngokubhengezwa koGcino lweGPUDirect kunye nokuvela kobuchwepheshe obufanayo obuvela kwabanye abathengisi be-GPU, imfuno yeKingston yeenkqubo zokugcina ezenzelwe ukusetyenziswa kwikhompyuter ephezulu yokusebenza iyakhula. Umakishi uya kuba yisantya sokufunda idatha kwinkqubo yokugcina, efaniswa nokugqithiswa kwe-40- okanye i-100-Gbit yamakhadi womnatha wenethiwekhi ekungeneni kwiyunithi yekhompyutheni kunye ne-GPU. Ngaloo ndlela, iinkqubo zokugcina i-ultra-high-speed storage, ezibandakanya i-NVMe yangaphandle ngeFabric, ziya kuhamba zisuka kwi-exotic ukuya kwi-mainstream yezicelo ze-HPC. Ukongeza kwizibalo zesayensi kunye nezemali, baya kufumana isicelo kwezinye iindawo ezininzi ezisebenzayo, ezifana neenkqubo zokhuseleko kwinqanaba le-Safe City Metropolitan okanye amaziko okuhlola ezothutho, apho ukuqaphela kunye nokuchongwa kwezantya zezigidi zemifanekiso ye-HD ngesekhondi kufuneka, "ichazwe. i-niche yemarike yenkqubo yokuGcina ephezulu
Ulwazi oluthe kratya malunga neemveliso zeKingston lunokufumaneka apha nkampani.
umthombo: www.habr.com