Ndewo Habr! Ntọala data maka Big Data na mmụta igwe na-eto nke ukwuu ma anyị kwesịrị ijigide ha. Ozi anyị gbasara teknụzụ ọhụrụ ọhụrụ n'ọhịa nke ịgbakọ arụmọrụ dị elu (HPC, Computing High Performance Computing), nke egosiri na ụlọ ntu Kingston na . Nke a bụ iji sistemụ nchekwa data Hi-End (SDS) na sava nwere nkeji nhazi eserese (GPU) yana teknụzụ ụgbọ ala nchekwa GPUDirect. Ekele maka mgbanwe data n'etiti sistemụ nchekwa na GPU, na-agafe CPU, itinye data n'ime ngwa ngwa GPU na-eme ngwa ngwa site n'usoro ịdị ukwuu, yabụ ngwa Big Data na-arụ ọrụ kachasị nke GPU na-enye. N'aka nke ya, ndị na-emepụta usoro HPC na-enwe mmasị na ọganihu na usoro nchekwa nke nwere ọsọ I / O kachasị elu, dị ka nke Kingston mepụtara.
Ọrụ GPU karịrị nbudata data
Ebe ọ bụ na CUDA, ngwaike dabere na GPU na ngwa nrụkọmpụta kọmpụta sọftụwia maka mmepe ngwa ebumnuche izugbe, ka emepụtara na 2007, ike ngwaike nke GPU n'onwe ha etoola nke ukwuu. Taa, a na-ejikarị GPU eme ihe na ngwa HPC dị ka Big Data, mmụta igwe (ML), na mmụta miri emi (DL).
Rịba ama na n'agbanyeghị myirịta nke okwu, abụọ ikpeazụ bụ algọridim dị iche iche ọrụ. ML na-azụ kọmpụta ahụ dabere na data ahaziri ahazi, ebe DL na-azụ kọmpụta dabere na nzaghachi sitere na netwọkụ akwara. Ihe atụ iji nyere aka ịghọta ndịiche dị nnọọ mfe. Ka anyị were ya na kọmpụta ga-amata ọdịiche dị n'etiti foto nwamba na nkịta na-ejuru na sistemụ nchekwa. Maka ML, ị ga-edobe ihe onyonyo nwere ọtụtụ mkpado, nke ọ bụla na-akọwa otu akụkụ nke anụmanụ ahụ. Maka DL, ọ bụ iji bulite ọnụọgụ onyonyo buru ibu, mana jiri naanị otu mkpado “nke a bụ pusi” ma ọ bụ “nke a bụ nkịta”. DL yiri nnọọ otú a na-akụziri ụmụaka ihe - ha na-egosi nanị foto nke nkịta na nwamba na akwụkwọ na ndụ (ọtụtụ mgbe, na-enweghị ọbụna na-akọwa nkọwa zuru ezu dị iche), na ụbụrụ nwa n'onwe ya na-amalite ikpebi ụdị anụmanụ mgbe. a ụfọdụ oké egwu ọnụ ọgụgụ nke foto maka ntụnyere ( Dị ka atụmatụ si kwuo, anyị na-ekwu banyere nanị otu narị ma ọ bụ abụọ na-egosi n'oge nwata). Algọridim DL erubeghị nke ọma: ka netwọkụ akwara wee rụọ ọrụ nke ọma n'ịchọpụta onyonyo, ọ dị mkpa inye nri na hazie nde onyonyo n'ime GPU.
Nchịkọta okwu mmalite: dabere na GPUs, ị nwere ike wulite ngwa HPC n'ọhịa nke Big Data, ML na DL, mana enwere nsogbu - usoro data ahụ buru ibu nke na oge ejiri na-ebu data sitere na sistemụ nchekwa na GPU. amalite ibelata n'ozuzu arụmọrụ nke ngwa. N'ikwu ya n'ụzọ ọzọ, GPU ngwa ngwa na-anọgide na-ejighi ya n'ihi data I/O na-abịa site na sistemụ ndị ọzọ. Ọdịiche dị na ọsọ I/O nke GPU na ụgbọ ala na sistemụ CPU/nchekwa nwere ike ịbụ usoro ịdị ukwuu.
Kedu ka teknụzụ nchekwa GPUDirect si arụ ọrụ?
Usoro I/O na-achịkwa CPU, dịkwa ka usoro nbudata data sitere na nchekwa na GPU maka nhazi ọzọ. Nke a butere arịrịọ maka teknụzụ ga-enye ohere ozugbo n'etiti GPUs na draịva NVMe ka ha na ibe ha na-akpakọrịta ngwa ngwa. NVIDIA bụ onye mbụ nyere teknụzụ dị otú ahụ wee kpọọ ya GPUDirect Storage. N'ezie, nke a bụ mgbanwe nke teknụzụ GPUDirect RDMA (Remote Direct Memory Address) nke ha mepụtara na mbụ.
Jensen Huang, Onye isi oche nke NVIDIA, ga-eweta GPUDirect Nchekwa dị ka iche nke GPUDirect RDMA na SC-19. Isi mmalite: NVIDIA
Ọdịiche dị n'etiti GPUDirect RDMA na GPUDirect Storage dị na ngwaọrụ n'etiti nke a na-eme okwu. A na-emegharị teknụzụ GPUDirect RDMA iji bugharịa data ozugbo n'etiti kaadị interface netwọk n'ihu (NIC) na ebe nchekwa GPU, yana GPUDirect Nchekwa na-enye ụzọ data ozugbo n'etiti nchekwa mpaghara ma ọ bụ nke dịpụrụ adịpụ dị ka NVMe ma ọ bụ NVMe n'elu Fabric (NVMe-oF) na GPU ebe nchekwa.
Ma GPUDirect RDMA na GPUDirect Nchekwa na-ezere mmegharị data na-enweghị isi site na nchekwa nchekwa CPU ma kwe ka usoro nchekwa nchekwa ozugbo (DMA) bugharịa data site na kaadị netwọkụ ma ọ bụ nchekwa ozugbo na ma ọ bụ site na ebe nchekwa GPU - ha niile na-enweghị ibu na etiti CPU. Maka nchekwa GPUDirect, ebe nchekwa ahụ adịghị mkpa: ọ nwere ike ịbụ diski NVME n'ime otu GPU, n'ime rack, ma ọ bụ jikọọ na netwọk dịka NVMe-oF.
Atụmatụ ọrụ nke nchekwa GPUDirect. Isi mmalite: NVIDIA
Sistemụ nchekwa Hi-End na NVMe na-achọ n'ahịa ngwa HPC
N'ịghọta na na ọbịbịa nke GPUDirect Storage, mmasị nke nnukwu ndị ahịa ga-adọrọ mmasị ịnye usoro nchekwa na ngwa ngwa I / O kwekọrọ na ntinye nke GPU, na ihe ngosi SC-19 Kingston gosipụtara ngosi ngosi nke usoro nke nwere a usoro nchekwa dabere na diski NVMe yana otu nwere GPU, nke nyochara puku kwuru puku onyonyo satịlaịtị kwa nkeji. Anyị edeelarị maka usoro nchekwa dị otú ahụ dabere na 10 DC1000M U.2 NVMe draịva. .
Sistemu nchekwa dabere na 10 DC1000M U.2 NVMe draịva na-emeju ihe nkesa nke ọma na ndị na-eme ngwa ngwa eserese. Isi mmalite: Kingston
Emebere usoro nchekwa a ka ọ bụrụ otu 1U ma ọ bụ nnukwu rack ma nwee ike tụọ ya dabere na ọnụọgụ DC1000M U.2 NVMe, nke ọ bụla nwere ikike nke 3.84-7.68 TB. DC1000M bụ ihe mbụ NVMe SSD ụdị n'ụdị U.2 na ahịrị Kingston nke data center draịva. Ọ nwere ọkwa ntachi obi (DWPD, Drive na-ede kwa ụbọchị), na-enye ya ohere idegharị data n'ụzọ zuru ezu otu ugboro n'ụbọchị maka ndụ draịvụ ahụ ejiri n'aka.
N'ime ule fio v3.13 na Ubuntu 18.04.3 LTS sistemụ arụmọrụ, Linux kernel 5.0.0-31-generic, ihe ngosi nchekwa ihe ngosi gosipụtara ọsọ agụ (Agụgide agụ) nke 5.8 nde IOPS na ntinye na-adigide (bandwidth na-adịgide adịgide). ) nke 23.8 Gbit/s.
Ariel Perez, onye njikwa azụmaahịa SSD na Kingston, kwuru banyere sistemụ nchekwa ọhụrụ: “Anyị dị njikere ịkwado ọgbọ na-esote nke sava na ngwọta U.2 NVMe SSD iji kpochapụ ọtụtụ n'ime ihe mgbochi mbufe data nke ejirila na nchekwa. Ngwakọta nke NVMe SSD draịva na Premier Premier DRAM anyị na-eme Kingston ka ọ bụrụ otu n'ime ndị na-eweta ihe ngwọta njedebe njedebe na njedebe nke ụlọ ọrụ ahụ. "
Nnwale gfio v3.13 gosipụtara ntinye nke 23.8 Gbps maka sistemụ nchekwa ngosi na draịva DC1000M U.2 NVMe. Isi mmalite: Kingston
Kedu otu usoro maka ngwa HPC ga-adị ka iji GPUDirect Storage ma ọ bụ teknụzụ yiri ya? Nke a bụ ihe owuwu nwere nkewa anụ ahụ nke nkeji arụ ọrụ n'ime rack: otu nkeji ma ọ bụ abụọ maka RAM, ọtụtụ ndị ọzọ maka GPU na ọnụ ọnụ CPU, yana otu ma ọ bụ karịa maka sistemụ nchekwa.
Site na nkwuputa nke nchekwa GPUDirect yana mpụta teknụzụ yiri nke ahụ sitere na ndị na-ere GPU ndị ọzọ, ọchịchọ Kingston maka sistemụ nchekwa nke emebere maka iji na kọmpụta dị elu na-agbasa. Ihe nrịbama ga-abụ ọsọ nke ịgụ data sitere na sistemụ nchekwa, ma e jiri ya tụnyere ntinye nke kaadị netwọkụ 40- ma ọ bụ 100-Gbit n'ọnụ ụzọ otu mgbakọ nwere GPU. Ya mere, sistemụ nchekwa ngwa ngwa dị elu, gụnyere NVMe mpụga site na Fabric, ga-esi na-apụ apụ gaa na isi maka ngwa HPC. Na mgbakwunye na ngụkọ sayensị na ego, ha ga-ahụ ngwa n'ọtụtụ mpaghara ndị ọzọ bara uru, dị ka sistemụ nchekwa na Safe City mepere emepe ma ọ bụ ebe a na-ahụ maka njem, ebe achọrọ nnabata na njirimara ọsọ nke nde ihe onyonyo HD kwa nkeji. niche ahịa nke usoro Nchekwa kacha elu
Enwere ike ịchọta ozi ndị ọzọ gbasara ngwaahịa Kingston na ụlọ ọrụ.
isi: www.habr.com