{"id":54686,"date":"2020-01-01T00:00:00","date_gmt":"2019-12-31T21:00:00","guid":{"rendered":"https:\/\/prohoster.info\/blog\/blog_prohoster\/5-8-mln-iops-zachem-tak-mnogo"},"modified":"2020-02-18T14:02:44","modified_gmt":"2020-02-18T11:02:44","slug":"5-8-mln-iops-zachem-tak-mnogo","status":"publish","type":"post","link":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/5-8-mln-iops-zachem-tak-mnogo","title":{"rendered":"5.8 million IOPS: why so much?","gt_translate_keys":[{"key":"rendered","format":"text"}]},"content":{"rendered":"

Hello Habr! Datasets for Big Data and machine learning are growing exponentially, and we need to keep up with processing them. Our post discusses another innovative technology in the field of high-performance computing (HPC) showcased at Kingston's booth at Supercomputing-2019<\/a><\/noindex>. This is the application of Hi-End storage systems in servers with graphics processors (GPU) and the GPUDirect Storage bus technology. With direct data exchange between the storage systems and GPUs, bypassing the CPU, data loading into GPU accelerators is significantly accelerated, allowing Big Data applications to operate at peak performance supported by GPUs. HPC system developers, in turn, are interested in advancements in storage systems with the highest input\/output speeds, such as those produced by Kingston.<\/p>\n

\"5.8
\n<\/a><\/noindex><\/p>\n

GPU performance outpaces data loading <\/h2>\n

\nSince the creation of CUDA in 2007\u2014a software-hardware architecture for parallel computing based on GPUs for developing general-purpose applications\u2014the hardware capabilities of GPUs have grown immensely. Today, GPUs are increasingly used in HPC applications such as big data, machine learning (ML), and deep learning (DL). <\/p>\n

It is worth noting that despite the similarity of terms, the last two represent algorithmically different tasks. ML trains a computer based on structured data, while DL relies on feedback from a neural network. A simple example helps to understand these differences. Suppose a computer must distinguish between pictures of cats and dogs uploaded from a storage system. For ML, a set of images should be provided with multiple tags, each defining a specific feature of the animal. For DL, a much larger number of images can be uploaded, but only with a single tag: \"this is a cat\" or \"this is a dog.\" DL closely resembles how young children are taught\u2014they are shown images of dogs and cats in books and real life (often without detailed explanations), and the child's brain begins to identify the type of animal after a certain critical number of images for comparison (estimates suggest it's just a hundred or so exposures throughout early childhood). DL algorithms are not yet perfect: for a neural network to effectively recognize patterns, millions of images must be processed and fed into the GPU.<\/p>\n

The conclusion of the preface: HPC applications in the fields of Big Data, ML, and DL can be built on GPUs, but there is a problem\u2014the datasets are so large that the time taken to load data from storage into the GPU starts to diminish the overall performance of the application. In other words, fast graphics processors remain underutilized due to the slow input\/output of data from other subsystems. The difference in input\/output speed between the GPU and the CPU\/storage bus can be an order of magnitude.<\/p>\n

How does the GPUDirect Storage technology work?<\/h2>\n

\nThe input\/output process is controlled by the CPU, as is the process of loading data from storage into graphics processors for subsequent processing. Hence, the demand arose for a technology that would provide direct access between the GPU and NVMe drives for rapid interaction. The first such technology was proposed by NVIDIA and named GPUDirect Storage. Essentially, it's a variant of their previously developed GPUDirect RDMA (Remote Direct Memory Address) technology.<\/p>\n

\"5.8
\nJensen Huang, CEO of NVIDIA, presents GPUDirect Storage as a variant of GPUDirect RDMA at the S\u0421-19 conference. Source: NVIDIA<\/i><\/p>\n

The difference between GPUDirect RDMA and GPUDirect Storage lies in the devices that are being addressed. The GPUDirect RDMA technology is designed for transferring data directly between the network interface card (NIC) and GPU memory, while GPUDirect Storage provides a direct data transfer path between local or remote storage, such as NVMe or NVMe over Fabric (NVMe-oF), and GPU memory.<\/p>\n

Both options, GPUDirect RDMA and GPUDirect Storage, avoid unnecessary data transfers through CPU memory buffers, allowing the direct memory access (DMA) mechanism to move data from the network card or storage directly to GPU memory, and vice versa \u2014 all without burdening the central processor. For GPUDirect Storage, the location of the storage doesn't matter: it can be an NVMe drive within the GPU unit, within a rack, or connected over the network as NVMe-oF. <\/p>\n

\"5.8
\nThe operation scheme of GPUDirect Storage. Source: NVIDIA<\/i><\/p>\n

High-end storage systems based on NVMe are in demand in the HPC application market.<\/h2>\n

\nRecognizing that with the advent of GPUDirect Storage, major clients will focus on storage system offerings that match the I\/O speed corresponding to GPU bandwidth, at the SC-19 exhibition Kingston showcased a demo system consisting of an NVMe-based storage system and a GPU unit, where thousands of satellite images were analyzed per second. We have already written about such a storage system based on 10 DC1000M U.2 NVMe drives. in a report from the supercomputing exhibition.<\/a><\/noindex>. <\/p>\n

\"5.8
\nA storage system based on 10 DC1000M U.2 NVMe drives complements a server with graphics accelerators well. Source: Kingston<\/i><\/p>\n

This storage system is designed as a 1U rack unit or larger and can scale depending on the number of DC1000M U.2 NVMe drives, with each drive having a capacity of 3.84-7.68 TB. The DC1000M is the first NVMe SSD model in the U.2 form factor in Kingston's lineup for data centers. It has a durability rating (DWPD, Drive writes per day) that allows data to be rewritten to full capacity once per day throughout the drive's warranty period. <\/p>\n

In the fio v3.13 test on Ubuntu 18.04.3 LTS operating system, Linux kernel 5.0.0-31-generic, the showcased storage system achieved a read speed (Sustained Read) of 5.8 million IOPS with a sustained bandwidth of 23.8 Gbps.<\/p>\n

Ariel Perez, Business Manager for SSD at Kingston, described the new storage systems: \u201cWe are ready to supply the next generation of servers with U.2 NVMe SSD solutions to eliminate many of the data transfer bottlenecks traditionally associated with storage systems. The combination of NVMe SSD drives and our premium Server Premier DRAM makes Kingston one of the most comprehensive providers of integrated data processing solutions in the industry.\u201d<\/p>\n

\"5.8
\nThe gfio v3.13 test showed a throughput of 23.8 Gbps for a demo storage system using DC1000M U.2 NVMe drives. Source: Kingston<\/i><\/p>\n

What would a typical system for HPC applications look like where GPUDirect Storage technology or a similar one is implemented? It's an architecture with physical separation of functional blocks within the rack: one or two units for memory, several for GPU and CPU compute nodes, and one or more units for storage systems.<\/p>\n

With the announcement of GPUDirect Storage and the potential emergence of similar technologies from other GPU vendors, the demand for storage systems designed for high-performance computing is growing for Kingston. A marker will be the data reading speed from the storage systems, comparable to the throughput of 40 or 100 Gbps network cards entering the compute unit with GPU. Thus, ultra-fast storage systems, including external NVMe over Fabric, will transition from exotic to mainstream for HPC applications. Beyond science and financial computations, they will find applications in many other practical areas, such as metropolitan security systems like Safe City or transportation monitoring centers, where the speed of recognition and identification at the level of millions of HD images per second is required,\u201d highlighted the market niche for top-tier storage systems.<\/p>\n

Additional information about Kingston products can be found at the official website<\/a><\/noindex> of the company.
\n
Source: habr.com<\/a><\/p>","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"excerpt":{"rendered":"

\u041f\u0440\u0438\u0432\u0435\u0442 \u0425\u0430\u0431\u0440! \u041d\u0430\u0431\u043e\u0440\u044b \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f Big Data \u0438 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043e\u043d\u0435\u043d\u0446\u0438\u0430\u043b\u044c\u043d\u043e \u0440\u0430\u0441\u0442\u0443\u0442 \u0438 \u043d\u0430\u0434\u043e \u0443\u0441\u043f\u0435\u0432\u0430\u0442\u044c \u0438\u0445 \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c. \u041d\u0430\u0448 \u043f\u043e\u0441\u0442 \u043e \u0435\u0449\u0435 \u043e\u0434\u043d\u043e\u0439 \u0438\u043d\u043d\u043e\u0432\u0430\u0446\u0438\u043e\u043d\u043d\u043e\u0439 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0438 \u0432 \u043e\u0431\u043b\u0430\u0441\u0442\u0438 \u0432\u044b\u0441\u043e\u043a\u043e\u043f\u0440\u043e\u0438\u0437\u0432\u043e\u0434\u0438\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0432\u044b\u0447\u0438\u0441\u043b\u0435\u043d\u0438\u0439 (HPC, High Performance Computing), \u043f\u043e\u043a\u0430\u0437\u0430\u043d\u043d\u043e\u0439 \u043d\u0430 \u0441\u0442\u0435\u043d\u0434\u0435 Kingston \u043d\u0430 Supercomputing-2019. \u042d\u0442\u043e \u043f\u0440\u0438\u043c\u0435\u043d\u0435\u043d\u0438\u0435 Hi-End \u0441\u0438\u0441\u0442\u0435\u043c \u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f \u0434\u0430\u043d\u043d\u044b\u0445 (\u0421\u0425\u0414) \u0432 \u0441\u0435\u0440\u0432\u0435\u0440\u0430\u0445 \u0441 \u0433\u0440\u0430\u0444\u0438\u0447\u0435\u0441\u043a\u0438\u043c\u0438 \u043f\u0440\u043e\u0446\u0435\u0441\u0441\u043e\u0440\u0430\u043c\u0438 (GPU) \u0438 \u0442\u0435\u0445\u043d\u043e\u043b\u043e\u0433\u0438\u0435\u0439 \u0448\u0438\u043d\u044b GPUDirect […]<\/p>\n","protected":false,"gt_translate_keys":[{"key":"rendered","format":"html"}]},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[688],"tags":[],"class_list":["post-54686","post","type-post","status-publish","format-standard","hentry","category-administrirovanie"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\n","aioseo_head_json":{"title":"\ud83e\udd475.8 million IOPS: why so much? | ProHoster","description":"Hello Habr! Datasets for Big Data and machine learning are growing exponentially, and we need to keep up with processing them.","canonical_url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/5-8-mln-iops-zachem-tak-mnogo","robots":"max-image-preview:large","keywords":"","webmasterTools":{"miscellaneous":""},"schema":null,"og:locale":"en_US","og:site_name":"ProHoster | \u041a\u0443\u043f\u0438\u0442\u044c \u043d\u0430\u0434\u0435\u0436\u043d\u044b\u0439 \u0445\u043e\u0441\u0442\u0438\u043d\u0433 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432 \u0441 \u0437\u0430\u0449\u0438\u0442\u043e\u0439 \u043e\u0442 DDoS, VPS VDS \u0441\u0435\u0440\u0432\u0435\u0440\u044b","og:type":"article","og:title":"\ud83e\udd475.8 \u043c\u043b\u043d IOPS: \u0437\u0430\u0447\u0435\u043c \u0442\u0430\u043a \u043c\u043d\u043e\u0433\u043e? | ProHoster","og:description":"\u041f\u0440\u0438\u0432\u0435\u0442 \u0425\u0430\u0431\u0440! \u041d\u0430\u0431\u043e\u0440\u044b \u0434\u0430\u043d\u043d\u044b\u0445 \u0434\u043b\u044f Big Data \u0438 \u043c\u0430\u0448\u0438\u043d\u043d\u043e\u0433\u043e \u043e\u0431\u0443\u0447\u0435\u043d\u0438\u044f \u044d\u043a\u0441\u043f\u043e\u043d\u0435\u043d\u0446\u0438\u0430\u043b\u044c\u043d\u043e \u0440\u0430\u0441\u0442\u0443\u0442 \u0438 \u043d\u0430\u0434\u043e \u0443\u0441\u043f\u0435\u0432\u0430\u0442\u044c \u0438\u0445 \u043e\u0431\u0440\u0430\u0431\u0430\u0442\u044b\u0432\u0430\u0442\u044c.","og:url":"https:\/\/prohoster.info\/en\/blog\/administrirovanie\/5-8-mln-iops-zachem-tak-mnogo","og:image":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:secure_url":"https:\/\/prohoster.info\/wp-content\/uploads\/2021\/11\/logo-350.jpg","og:image:width":350,"og:image:height":350,"article:published_time":"2019-12-31T21:00:00+00:00","article:modified_time":"2020-02-18T11:02:44+00:00","article:publisher":"https:\/\/www.facebook.com\/prohoster","article:author":"https:\/\/www.facebook.com\/prohoster"},"aioseo_meta_data":{"post_id":"54686","title":null,"description":null,"keywords":null,"keyphrases":null,"primary_term":null,"canonical_url":null,"og_title":null,"og_description":null,"og_object_type":"default","og_image_type":"default","og_image_url":null,"og_image_width":null,"og_image_height":null,"og_image_custom_url":null,"og_image_custom_fields":null,"og_video":null,"og_custom_url":null,"og_article_section":null,"og_article_tags":null,"twitter_use_og":false,"twitter_card":"default","twitter_image_type":"default","twitter_image_url":null,"twitter_image_custom_url":null,"twitter_image_custom_fields":null,"twitter_title":null,"twitter_description":null,"schema":{"blockGraphs":[],"customGraphs":[],"default":{"data":{"Article":[],"Course":[],"Dataset":[],"FAQPage":[],"Movie":[],"Person":[],"Product":[],"ProductReview":[],"Car":[],"Recipe":[],"Service":[],"SoftwareApplication":[],"WebPage":[]},"graphName":"","isEnabled":true},"graphs":[]},"schema_type":null,"schema_type_options":null,"pillar_content":false,"robots_default":true,"robots_noindex":false,"robots_noarchive":false,"robots_nosnippet":false,"robots_nofollow":false,"robots_noimageindex":false,"robots_noodp":false,"robots_notranslate":false,"robots_max_snippet":null,"robots_max_videopreview":null,"robots_max_imagepreview":"large","priority":null,"frequency":null,"local_seo":null,"seo_analyzer_scan_date":"2026-01-24 12:22:25","breadcrumb_settings":null,"limit_modified_date":false,"reviewed_by":null,"ai":null,"created":"2021-02-28 20:00:11","updated":"2026-01-24 12:22:25","focus_keyword":null,"additional_keywords":null,"truseo_locale":null},"gt_translate_keys":[{"key":"link","format":"url"}],"_links":{"self":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/54686","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/comments?post=54686"}],"version-history":[{"count":0,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/posts\/54686\/revisions"}],"wp:attachment":[{"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/media?parent=54686"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/categories?post=54686"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prohoster.info\/en\/wp-json\/wp\/v2\/tags?post=54686"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}