One of the members of the social network development team Facebook, Roman Gushchin, offered in the developers mailing list a set of Linux kernel patches, aimed at improving the work with memory through the implementation of a new memory controller - slab (slab memory controller).
slab distribution is a memory management mechanism designed to allocate memory more efficiently and eliminate significant fragmentation. The basis of this algorithm is to store allocated memory containing an object of a certain type and reuse this memory the next time it is allocated for an object of the same type. This technique was first introduced in SunOS by Jeff Bonwick and is now widely used in the kernels of many Unix operating systems, including FreeBSD and Linux.
The new controller is based on moving slab accounting from the memory page level to the kernel object level, which makes it possible to share a single slab page across different cgroups, instead of dedicating a separate cache for each cgroup.
According to the test results, it follows that the proposed method of memory management allows to increase efficiency slab usage up to 45%, as well as reduce the overall memory consumption of the OS kernel. Also, by reducing the number of pages allocated for slab, memory fragmentation as a whole is reduced, which cannot but affect the system performance.
The new controller has been tested on Facebook working servers for several months, and so far this testing can be called successful: in the absence of performance losses and an increase in the number of errors, a clear decrease in memory consumption has been noticed - on some servers up to 1GB. This number is quite subjective, for example, earlier tests showed slightly lower results:
- 650-700 MB on the web frontend
- 750-800 MB on server with database cache
- 700 MB on DNS server
Source: linux.org.ru