Ingo Molnar, a well-known Linux kernel developer and author of the Completely Fair Scheduler (CFS), has proposed a series of patches for discussion on the Linux kernel developers mailing list. These patches affect more than half of all files in the kernel source code and provide an increase in the speed of a full kernel rebuild by 50-80%, depending on the settings. The optimization is notable for being associated with the addition of the largest set of changes in kernel development history — a total of 2297 patches have been proposed, changing over 25,000 files (10,000 header files in the 'include/' and 'arch/*/include/' directories and 15,000 source files).
The performance gain is achieved by changing the method of processing header files. It is noted that after thirty years of kernel development, the state of header files has become deplorable due to a large number of cross-dependencies among files. The work on restructuring the header files took over a year and required significant reworking of the hierarchy and dependencies. During the restructuring, effort was made to separate type and API definitions for different kernel subsystems.
Among the changes made: separating high-level header files from each other, excluding linking headers from inline functions, allocating header files for types and APIs, ensuring isolated building of header files (about 80 files had interfering indirect dependencies exposed through other header files), automatic dependency addition to '.h' and '.c' files, step-by-step optimization of header files, using the 'CONFIG_KALLSYMS_FAST=y' mode, and selective consolidation of C files into build blocks to reduce the number of object files.
As a result, the work done has allowed for a reduction in the size of header files processed during the post-processing stage by one to two orders of magnitude. For example, prior to optimization, the use of the header file 'linux/gfp.h' added 13,543 lines of code and included 303 dependent header files, whereas after optimization, the size was reduced to 181 lines and 26 dependent files. Another example: during the preprocessing of the file 'kernel/pid.c', without the patch, 94,000 lines of code were connected, most of which were not used in pid.c. The separation of header files enabled a threefold reduction in the amount of code processed, decreasing the number of processed lines to 36,000.
When fully rebuilding the kernel with the command 'make -j96 vmlinux' on the test system, the application of patches showed a reduction in build time for branch v5.16-rc7 from 231.34 to 129.97 seconds (from 15.5 to 27.7 builds per hour), as well as increased CPU core efficiency during the build. In incremental builds, the effect of optimization is even more noticeable—the time for rebuilding the kernel after changes to the header files has decreased significantly (from 112% to 173%, depending on the changed header file). Optimization is currently only available for ARM64, MIPS, Sparc, and x86 (32-bit and 64-bit) architectures.
Source: opennet.ru
