{"id":54686,"date":"2020-01-01T00:00:00","date_gmt":"2019-12-31T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/5-8-mln-iops-zachem-tak-mnogo"},"modified":"2020-02-18T14:02:44","modified_gmt":"2020-02-18T11:02:44","slug":"5-8-mln-iops-zachem-tak-mnogo","status":"publish","type":"post","link":"https:\/\/prohoster.info\/da\/blog\/administrirovanie\/5-8-mln-iops-zachem-tak-mnogo","title":{"rendered":"5,8 millioner IOPS: hvorfor s\u00e5 mange?","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Hej Habr! Datas\u00e6t til Big Data og maskinl\u00e6ring vokser eksponentielt, og vi skal n\u00e5 at behandle dem. Vores indl\u00e6g handler om endnu en innovativ teknologi inden for h\u00f8jtydende computing (HPC, High Performance Computing), pr\u00e6senteret p\u00e5 Kingston's stand p\u00e5 Supercomputing-2019<\/a><\/noindex>. Dette er anvendelsen af hi-end datalagringssystemer (SAS) i servere med grafiske processorer (GPU) og GPUDirect Storage-bussystemteknologi. Takket v\u00e6re direkte dataudveksling mellem SAS og GPU, der omg\u00e5r CPU, kan dataindl\u00e6sningen til GPU-acceleratorer betydeligt accelereres, hvilket g\u00f8r, at Big Data-applikationer k\u00f8rer p\u00e5 maksimal ydeevne, som GPU'er kan levere. HPC-systemudviklere er derimod interesseret i fremskridt inden for SAS med ekstremt h\u00f8j ind- og udgangshastighed \u2014 s\u00e5som dem, Kingston producerer.<\/p>\n

\"5,8
\n<\/a><\/noindex><\/p>\n

GPU-ydeevne overg\u00e5r dataindl\u00e6sning <\/h2>\n

\nSiden CUDA blev skabt i 2007 \u2014 en software-hardware-arkitektur for parallelle beregninger baseret p\u00e5 GPU til udvikling af generelle applikationer, er de hardwarem\u00e6ssige muligheder for GPU'erne steget utrolig meget. I dag finder GPU'er anvendelse i flere og flere HPC-applikationer, s\u00e5som big data, maskinl\u00e6ring (ML, machine learning) og dyb l\u00e6ring (DL, deep learning). <\/p>\n

Det er v\u00e6rd at bem\u00e6rke, at selvom termerne kan synes at v\u00e6re ens, er de to sidste algoritmisk forskellige opgaver. ML tr\u00e6ner computeren baseret p\u00e5 strukturerede data, mens DL g\u00f8r det p\u00e5 baggrund af respons fra et neuralt netv\u00e6rk. Et eksempel p\u00e5 at forst\u00e5 forskellene er ret simpelt. Lad os antage, at computeren skal skelne mellem billeder af katte og hunde, som uploades fra et lagersystem. For ML skal der pr\u00e6senteres et s\u00e6t billeder med mange tags, der hver is\u00e6r definerer en bestemt egenskab ved dyret. For DL er det nok at uploade et langt st\u00f8rre antal billeder, men kun med \u00e9n tag som \u201cdette er en kat\u201d eller \u201cdette er en hund\u201d. DL svarer meget til, hvordan sm\u00e5 b\u00f8rn l\u00e6rer \u2014 de f\u00e5r bare vist billeder af hunde og katte i b\u00f8ger og i virkeligheden (oftest uden at f\u00e5 forklaret detaljerne), og barnets hjerne begynder selv at skelne mellem dyretyper efter et kritisk antal billeder til sammenligning (if\u00f8lge estimater drejer det sig kun om et hundrede til to hundrede visninger i hele den tidlige barndom). DL-algoritmerne er stadig ikke s\u00e5 komplette: for at en neuralt netv\u00e6rk kan fungere s\u00e5 godt i billedgenkendelse, skal der pr\u00e6senteres og behandles millioner af billeder i GPU.<\/p>\n

Konklusionen p\u00e5 forordet: P\u00e5 baggrund af GPU kan der opbygges HPC-applikationer inden for Big Data, ML og DL, men der er et problem \u2014 datam\u00e6ngderne er s\u00e5 store, at den tid, der bruges p\u00e5 at indl\u00e6se data fra lagersystemet til GPU, begynder at p\u00e5virke den samlede ydeevne af applikationen negativt. Med andre ord forbliver de hurtige grafiske processorer underudnyttede p\u00e5 grund af den langsomme ind- og udgang af data fra andre subsystemer. Forskellen i hastighed mellem ind-\/uddata fra GPU og bussen til CPU\/lager kan v\u00e6re p\u00e5 en st\u00f8rrelsesorden.<\/p>\n

Hvordan fungerer teknologien GPUDirect Storage?<\/h2>\n

\nInd- og uddata processen styres af CPU'en, ligesom dataindl\u00e6sning fra lageret til grafiske processorer til yderligere behandling. Deraf opstod behovet for en teknologi, der kunne sikre direkte adgang mellem GPU og NVMe-diske for hurtigere interaktion mellem dem. Den f\u00f8rste teknologi, der tilb\u00f8d dette, var fra NVIDIA, og de kaldte den GPUDirect Storage. I bund og grund er det en variant af deres tidligere udviklede teknologi GPUDirect RDMA (Remote Direct Memory Address).<\/p>\n

\"5,8
\nJensen Huang, administrerende direkt\u00f8r for NVIDIA, pr\u00e6senterer GPUDirect Storage som en variant af GPUDirect RDMA p\u00e5 messen S\u0421-19. Kilde: NVIDIA<\/i><\/p>\n

Forskellen mellem GPUDirect RDMA og GPUDirect Storage ligger i de enheder, som adresseres. GPUDirect RDMA-teknologien er designet til at flytte data direkte mellem netv\u00e6rksinterfacekortet (NIC) og GPU-hukommelsen, mens GPUDirect Storage giver en direkte dataoverf\u00f8rsel mellem lokal eller fjernlagring, s\u00e5som NVMe eller NVMe over Fabric (NVMe-oF), og GPU-hukommelsen.<\/p>\n

Begge muligheder, GPUDirect RDMA og GPUDirect Storage, undg\u00e5r un\u00f8dvendige dataflytninger gennem CPU-hukommelsesbufferen og giver DMA-mekanismen mulighed for at flytte data fra netv\u00e6rkskortet eller lagringen direkte til GPU-hukommelsen eller fra den \u2014 alt sammen uden belastning af CPU'en. For GPUDirect Storage er placeringen af lagringen ligegyldig: det kan v\u00e6re en NVMe-disk inde i en GPU-enhed, i et rack eller tilsluttet over netv\u00e6rket som NVMe-oF. <\/p>\n

\"5,8
\nHvordan GPUDirect Storage fungerer. Kilde: NVIDIA<\/i><\/p>\n

Hi-End SAN p\u00e5 NVMe er eftertragtede p\u00e5 markedet for HPC-applikationer.<\/h2>\n

\nVed at forst\u00e5, at interessen fra store kunder vil fokusere p\u00e5 lagringssystemer med input\/output-hastigheder, der matcher GPU'ens b\u00e5ndbredde med GPUDirect Storage, viste Kingston p\u00e5 SC-19-messen en demo af et system best\u00e5ende af en SAN baseret p\u00e5 NVMe-diske og en GPU-enhed, hvor tusindvis af satellitbilleder blev analyseret per sekund. Vi har allerede skrevet om denne SAN baseret p\u00e5 10 DC1000M U.2 NVMe-drev. i rapporten fra supercomputerudstillingen.<\/a><\/noindex>. <\/p>\n

\"5,8
\nSAN baseret p\u00e5 10 DC1000M U.2 NVMe-drev passer perfekt til en server med grafiske acceleratorer. Kilde: Kingston.<\/i><\/p>\n

Denne SAN udf\u00f8res som en 1U eller st\u00f8rre rack-enhed og kan skaleres afh\u00e6ngigt af antallet af DC1000M U.2 NVMe-diske, hvor hver har en kapacitet p\u00e5 3,84-7,68 TB. DC1000M er den f\u00f8rste model af NVMe SSD i U.2 formfaktoren i Kingston's datacenter lagerlinje. Den har en holdbarhedsvurdering (DWPD, Drive writes per day), der tillader omwrite af data til fuld kapacitet \u00e9n gang om dagen i l\u00f8bet af lagringsenhedens garanterede levetid. <\/p>\n

I fio v3.13 testen p\u00e5 Ubuntu 18.04.3 LTS, Linux kernel 5.0.0-31-generic, viste udstillingspr\u00f8ven af SAN en l\u00e6sehastighed (Sustained Read) p\u00e5 5,8 millioner IOPS ved en stabil b\u00e5ndbredde (Sustained Bandwidth) p\u00e5 23,8 Gbit\/s.<\/p>\n

Ariel Perez, business manager for SSD at Kingston, described the new storage systems as follows: \"We are ready to supply the next generation of servers with U.2 NVMe SSD solutions to eliminate many of the data transfer bottlenecks traditionally associated with storage systems. The combination of NVMe SSDs and our premium Server Premier DRAM makes Kingston one of the most comprehensive providers of integrated data processing solutions in the industry.\"<\/p>\n

\"5,8
\nThe gfio v3.13 test showed a throughput of 23.8 Gbit\/s for the demonstration storage system on DC1000M U.2 NVMe drives. Source: Kingston<\/i><\/p>\n

What would a typical system for HPC applications look like, where GPUDirect Storage technology or a similar technology is implemented? It is an architecture with a physical separation of functional blocks within the rack: one or two units for RAM, several for GPU and CPU compute nodes, and one or more units for storage systems.<\/p>\n

With the announcement of GPUDirect Storage and the potential emergence of similar technologies from other GPU vendors, demand for storage systems optimized for high-performance computing is expanding for Kingston. A marker will be the speed of data reading from storage systems equivalent to the throughput of 40 or 100 Gbit network cards at the input to the GPU compute unit. Thus, ultra-high-speed storage systems, including external NVMe through Fabric, will transition from exotic to mainstream for HPC applications. Aside from science and financial calculations, they will find applications in many other practical fields, such as metropolitan security systems, Safe City, or transport monitoring centers, where recognition and identification speed at the level of millions of HD images per second is required,\" noted the market niche for top storage systems.<\/p>\n

For more information about Kingston products, visit den officielle hjemmeside<\/a><\/noindex> the company.
\n
Kilde: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u041f\u0440\u0438\u0432\u0435\u0442 \u0425\u0430\u0431\u0440! \u041d\u0430\u0431\u043e\u0440\u044b \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f Big Data \u0438 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043e\u043d\u0435\u043d\u0446\u0438\u0430\u043b\u044c\u043d\u043e \u0440\u0430\u0441\u0442\u0443\u0442 \u0438 \u043d\u0430\u0434\u043e \u0443\u0441\u043f\u0435\u0432\u0430\u0442\u044c \u0438\u0445 \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c. \u041d\u0430\u0448 \u043f\u043e\u0441\u0442 \u043e \u0435\u0449\u0435 \u043e\u0434\u043d\u043e\u0439 \u0438\u043d\u043d\u043e\u0432\u0430\u0446\u0438\u043e\u043d\u043d\u043e\u0439 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0438 \u0432 \u043e\u0431\u043b\u0430\u0441\u0442\u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0439 (HPC, High Performance Computing), \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u043d\u043e\u0439 \u043d\u0430 \u0441\u0442\u0435\u043d\u0434\u0435 Kingston \u043d\u0430 Supercomputing-2019. \u042d\u0442\u043e \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u0435 Hi-End \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445 (\u0421\u0425\u0414) \u0432 \u0441\u0435\u0440\u0432\u0435\u0440\u0430\u0445 \u0441 \u0433\u0440\u0430\u0444\u0438\u0447\u0435\u0441\u043a\u0438\u043c\u0438 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u0430\u043c\u0438 (GPU) \u0438 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0435\u0439 \u0448\u0438\u043d\u044b GPUDirect […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-54686","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd475.8 million IOPS: Why so much? | ProHoster","description":"Hello Habr! Datasets for Big Data and machine learning are growing exponentially, and we need to keep up with processing them.","canonical_url":"https:\/\/prohoster.info\/da\/blog\/administrirovanie\/5-8-mln-iops-zachem-tak-mnogo","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"da_DK","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd475.8 \u043c\u043b\u043d IOPS: \u0437\u0430\u0447\u0435\u043c \u0442\u0430\u043a \u043c\u043d\u043e\u0433\u043e? | ProHoster","og:description":"\u041f\u0440\u0438\u0432\u0435\u0442 \u0425\u0430\u0431\u0440! \u041d\u0430\u0431\u043e\u0440\u044b \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f Big Data \u0438 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043e\u043d\u0435\u043d\u0446\u0438\u0430\u043b\u044c\u043d\u043e \u0440\u0430\u0441\u0442\u0443\u0442 \u0438 \u043d\u0430\u0434\u043e \u0443\u0441\u043f\u0435\u0432\u0430\u0442\u044c \u0438\u0445 \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c.","og:url":"https:\/\/prohoster.info\/da\/blog\/administrirovanie\/5-8-mln-iops-zachem-tak-mnogo","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-12-31T21:00:00+00:00","article:modified_time":"2020-02-18T11:02:44+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"54686","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 12:22:25","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 20:00:11","updated":"2026-01-24 12:22:25","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/posts\/54686","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/comments?post=54686"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/posts\/54686\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/media?parent=54686"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/categories?post=54686"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/da\/wp-json\/wp\/v2\/tags?post=54686"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}