Engineers from Facebook (banned in the Russian Federation) published a report on the implementation last year of TMO (Transparent Memory Offloading) technology, which significantly conserves RAM on servers by offloading unnecessary secondary data to cheaper storage options like NVMe SSDs. According to Facebook's estimates, TMO can save between 20 to 32% of RAM on each server. This solution is aimed at infrastructures where applications run in isolated containers. TMO components that operate at the kernel level are already included in the Linux kernel.
The functionality of TMO at the Linux kernel level is supported by the PSI (Pressure Stall Information) subsystem, which has been available since version 4.20. PSI is already used in various memory shortage handlers and helps analyze information about the waiting time for various resources (CPU, memory, I/O). With PSI, user-space handlers can more accurately assess the system's load and the nature of slowdowns, allowing them to identify anomalies at an early stage when they do not yet significantly impact performance.
In user space, TMO is handled by the Senpai component, which dynamically adjusts the memory limit for application containers based on data obtained from PSI through cgroup2. Senpai analyzes signs of resource shortages via PSI, evaluates the sensitivity of applications to memory access delays, and attempts to determine the minimally required memory size for the container that keeps the necessary working data in RAM while offloading accompanying data, which is resident in the file cache or not being directly used at that moment, to swap space.
Thus, the essence of TMO is to keep processes on a strict diet in terms of memory consumption, forcibly transferring unused memory pages to the swap area, whose eviction does not noticeably affect performance (for example, pages with code used only during initialization and data used sporadically in disk cache). Unlike moving information to the swap in response to memory shortage, TMO evicts data based on proactive forecasting.
One of the criteria for eviction is the absence of access to a memory page for 5 minutes. Such pages are referred to as cold memory pages and on average account for about 35% of application memory (depending on the type of applications, the range varies from 19% to 65%). When evicting, the activity related to anonymous memory pages (memory allocated by applications) and memory used for file caching (allocated by the kernel) is taken into account. In some applications, primary consumption is associated with anonymous memory, while in others, the file cache is also significant. To avoid imbalance when evicting memory to cache, TMO uses a new paging algorithm that proportionally evicts anonymous pages and pages associated with the file cache.
Evicting rarely used pages to slower memory does not significantly impact performance but allows for substantial reductions in hardware costs. Data is evicted to SSD drives or into a compressed swap area in RAM. In terms of storage costs per byte of data, using NVMe SSDs is up to 10 times cheaper than using compression in RAM.
Source: opennet.ru
