MAX Data will be combined with Optane DC to effectively work with AI and ML systems.
A photo - — unsplash
On research by MIT Sloan Management Review and The Boston Consulting Group, 85% of the three thousand managers surveyed believe that AI systems will help their companies gain a competitive advantage in the market. However, only 39% of companies have tried to implement something similar in practice.
One of the reasons for this situation is that efficiently working with data and optimizing the use of capacities for machine learning tasks is not an easy job. At IDC that a new technology based on persistent memory (Persistent Memory, PMEM) could resolve the situation.
This technology was proposed by NetApp and Intel, NetApp Memory Accelerated (MAX) Data and Intel Optane DC Persistent Memory for a local storage product based on persistent memory.
How it works
MAX Data is a server technology that improves application performance through the use of PMEM or DRAM, but does not require software re-architecting.
It implements the principles of automated tiered storage, distributing data into levels and storages depending on the frequency of use - for "cold" data, more accessible storage is used, and frequently used ones are "at hand" - in Persistent Memory, which minimizes the delay when working with such data .
Version 1.1 uses DRAM and . Both implementations have their drawbacks - the relative loss of efficiency and the high cost of memory, respectively - compared to Optane DCPMM. A chart giving a comparative assessment of the delay is presented (p. 4).
Technology и and work with the semantics of block or file systems. Storage-level data protection and recovery is implemented with MAX Snap and MAX Recovery. These technologies use snapshots, the SnapMirror tool, and other ONTAP security mechanisms.
Schematically, the implementation looks like this:
There is no PMEM on this scheme yet, but the developers promise to add support for this type of memory before the end of the year. So far, Max Data works with DRAM and DIMM.
Solution Potential
At IDC that in the coming years there will be more developments like MAX Data, as the volume of corporate data is constantly growing, and companies do not have enough capacity to effectively process it. Technology useful in a large-scale cloud environment and for working with resource-intensive tasks like training neural networks. It will find application on trading platforms, information security systems and any other software products that require constant and fast access to large amounts of information.
There is also a degree of probability that the technology will not immediately take root in the market. As we noted above, only a third of companies worldwide work with AI systems in one form or another. From this point of view, the emergence of MAX Data may be considered premature by many and will focus their attention on a more accessible infrastructure that allows them to solve current problems.
Our other materials about IT infrastructure:
Source: habr.com