MAX Data will integrate with Optane DC for efficient operation with AI and ML systems.
Photo - - Unsplash
According to research from MIT Sloan Management Review and The Boston Consulting Group shows that 85% of three thousand surveyed managers believe that AI systems will help their companies gain a competitive edge in the market. However, only 39% of companies have attempted to implement something similar in practice.
One of the reasons for this situation is that effective data management and optimization of resources for machine learning tasks is no easy task. According to IDC, the situation could be resolved by a new technology based on Persistent Memory (PMEM).
This technology was proposed by NetApp and Intel, NetApp Memory Accelerated (MAX) Data and Intel Optane DC Persistent Memory to create a product for local data storage based on non-volatile memory.
How it works
MAX Data is a server technology that enhances application performance by leveraging PMEM or DRAM, but does not require a complete software architecture overhaul.
It implements principles of automated tiered storage, distributing data across tiers and storage based on usage frequency—less frequently accessed data is stored on more accessible storage, while frequently used data remains readily available in Persistent Memory, minimizing latency when working with such data.
Version 1.1 utilizes DRAM and . Both implementations have their drawbacks—relative loss of efficiency and high memory costs, respectively—compared to Optane DCPMM. A diagram providing a comparative assessment of latency is presented (p. 4).
Technology and and working with the semantics of block or file systems. Data protection and recovery at the storage level is implemented using MAX Snap and MAX Recovery. These technologies utilize snapshots, the SnapMirror tool, and other protective mechanisms from ONTAP.
The schematic representation looks as follows:
This diagram currently does not include PMEM, but developers promise to add support for this type of memory by the end of the year. For now, Max Data operates with DRAM and DIMM.
Solution Potential
According to IDC, there will be more developments like MAX Data in the coming years, as the volume of enterprise data continues to grow, and companies lack the capacity to process it efficiently. The technology It will be useful in large-scale cloud environments and for working with resource-intensive tasks such as training neural networks. It will find applications on trading platforms, in security systems, and in any other software products where constant and fast access to large volumes of information is required.
There is also a chance that the technology may not immediately take off in the market. As we noted earlier, only about a third of companies worldwide are currently utilizing AI systems in one form or another. From this perspective, the emergence of MAX Data may be considered premature by many, who will focus their attention on more accessible infrastructure that addresses current tasks.
Our other materials on IT infrastructure:
Source: habr.com
