A team of engineers at MIT developed an object-oriented memory hierarchy to work with data more efficiently. In the article we understand how it is arranged.
/ /PD
As you know, the performance growth of modern CPUs is not accompanied by a corresponding decrease in latency when accessing memory. The difference in the change in indicators from year to year can reach up to 10 times (). As a result, a bottleneck appears, which does not allow the full use of available resources and slows down data processing.
The performance penalty is the so-called decompression delay. In some cases, preparatory data decompression can take up to 64 processor cycles.
For comparison: addition and multiplication of floating point numbers no more than ten cycles. The problem is that memory works with data blocks of a fixed size, while applications operate with objects that can contain different types of data and differ from each other in size. To solve the problem, MIT engineers developed an object-oriented memory hierarchy that optimizes data processing.
How the technology works
The solution is based on three technologies: Hotpads, Zippads and the COCO compression algorithm.
Hotpads are a software-driven hierarchy of scratchpad register memory (). These registers are called pads (pads) and there are three of them - from L1 to L3. They store objects of different sizes, metadata, and arrays of pointers.
In essence, the architecture is a cache system, but sharpened to work with objects. The level of the pad at which the object is located depends on how often it is used. If one of the levels "overflows", the system triggers a mechanism similar to "garbage collectors" in Java or Go. It analyzes which objects are used less often than others and automatically moves them between levels.
Zippads works on the basis of Hotpads - archives and unzips data that enters or leaves the last two levels of the hierarchy - the L3 pad and main memory. In the first and second pads, the data is stored unchanged.
Zippads compresses objects up to 128 bytes. Larger objects are divided into parts, which are then placed in different areas of memory. As the developers write, this approach increases the coefficient of effectively used memory.
To compress objects, the COCO (Cross-Object COmpression) algorithm is used, which we will discuss later, although the system is able to work with or . The COCO algorithm is a variation of differential compression (). It compares objects to "base" and removes duplicate bits - see diagram below:
According to MIT engineers, their object-oriented memory hierarchy is 17% faster than classical approaches. It is much closer in its structure to the architecture of modern applications, so the new method has potential.
It is expected that first of all, companies that work with big data and machine learning algorithms can start using the technology. Another potential direction is cloud platforms. IaaS providers will be able to work more efficiently with virtualization, storage systems and computing resources.
Our additional resources and sources:
Source: habr.com