Release of Linux Kernel 5.18

After two months of development, Linus Torvalds announced the release of Linux kernel 5.18. Among the most notable changes are: a significant cleanup of legacy functionality, the deprecation of the Reiserfs filesystem, the implementation of trace events for user processes, added support for Intel IBT exploit mitigation, enabled buffer overflow detection mode when using the memcpy() function, introduced fprobe function call tracking, improved CPU scheduler performance on AMD Zen, included a driver to manage Intel CPU functionality (SDS), integrated parts of patches for restructuring header files, and approved the adoption of the C11 standard.

The new version incorporates 16,206 fixes from 2,127 developers (the previous release had 14,203 fixes from 1,995 developers), with a patch size of 108 MB (changes affected 14,235 files, adding 1,340,982 lines of code and removing 593,836 lines). About 44% of all changes presented in 5.18 are related to device drivers, approximately 16% pertain to code updates specific to hardware architectures, 11% is related to the network stack, 3% to filesystems, and 3% to internal kernel subsystems.

Key innovations in kernel 5.18:

• Disk subsystem, input/output, and file systems
  • In the Btrfs filesystem, support for passing compressed data during send and receive operations has been added. Previously, when using send/receive, the sending side would uncompress the data stored in a compressed format, while the receiving side would recompress it before writing. In kernel 5.18, user-space applications using send/receive calls are now enabled to transmit compressed data without re-packaging. This functionality is made possible through new ioctl operations BTRFS_IOC_ENCODED_READ and BTRFS_IOC_ENCODED_WRITE, allowing direct reading and writing of information in extents.

    Additionally, Btrfs has improved fsync performance. Support for deduplication and reflink (cloning file metadata with the creation of a link to already existing data without actual duplication) has been added for the entire storage, not limited to mount points.

  • In Direct I/O mode, access to encrypted files is enabled when using fscrypt inline encryption, where encryption and decryption operations are performed by the storage controller rather than the kernel. In traditional kernel-based encryption, access to encrypted files using Direct I/O remains impossible, as the file access bypasses the buffering mechanism in the kernel.
  • By default, the NFS server supports the NFSv3 protocol, which no longer requires separate activation and is available with general NFS activation. NFSv3 is considered the primary and always-supported version of NFS, while support for NFSv2 may be discontinued in the future. The efficiency of reading directory contents has significantly improved.
  • The ReiserFS filesystem has been classified as obsolete and is expected to be removed in 2025. Decommissioning ReiserFS will reduce maintenance efforts related to shared changes across filesystems concerning support for the new API for mounting, iomap, and volumes.
  • For F2FS, the ability to map user IDs of mounted filesystems has been implemented, which is used to associate files of a specific user on a mounted foreign partition with a different user in the current system.
  • The code for calculating statistics in Device-mapper handlers has been reworked, significantly improving accounting accuracy in handlers such as dm-crypt.
  • Support for 64-bit checksums for integrity verification has been implemented for NVMe devices.
  • A new mounting option 'keep_last_dots' has been proposed for the exfat filesystem, which prohibits the removal of trailing dots in file names (in Windows, trailing dots are removed by default).
  • In EXT4, the performance of the fast_commit mode has been improved, and scalability has increased. The mounting option 'mb_optimize_scan', which helps enhance performance under high filesystem fragmentation, has been adapted for working with extent files.
  • Support for write streams in the subsystem managing block devices has been discontinued. This feature was proposed for SSDs but did not gain traction, and there are currently no devices that support this mode, making it unlikely that they will appear in the future.
• Memory and system services
  • Integration of a patch set has begun, significantly reducing the time for kernel rebuilds by restructuring the hierarchy of header files and minimizing cross-dependencies. The 5.18 kernel includes patches that optimize the structure of the task scheduler's header files (kernel/sched). Compared to the previous release, the CPU time consumption during the build of the kernel/sched code has decreased by 61%, and the actual time has reduced by 3.9% (from 2.95 to 2.84 seconds).
  • The kernel code now allows the use of the C11 standard, published in 2011. Previously added code to the kernel had to comply with the ANSI C specification (C89), established in 1989. In the build scripts for kernel 5.18, the option ‘—std=gnu89’ has been replaced with ‘—std=gnu11 -Wno-shift-negative-value’. The possibility of using the C17 standard was considered, but this would have necessitated raising the minimum supported version of GCC; however, adding C11 support aligns with the current GCC version requirements (5.1).
  • Task scheduling performance on AMD processors with Zen microarchitecture has been improved, where multiple last-level caches (LLCs) are provided for each node with local memory channels. The new version addresses LLC balance issues between NUMA nodes, resulting in significant performance gains under certain types of workloads.
  • Tools for tracing user-space applications have been expanded. The new kernel version adds the capability for user processes to create tracing events and record data in a tracing buffer, which can be viewed through typical kernel tracing utilities like ftrace and perf. User-space tracing events are isolated from kernel tracing events. The status of events can be checked via the file /sys/kernel/debug/tracing/user_events_status, while event registration and data recording are managed through the file /sys/kernel/debug/tracing/user_events_data.
  • A tracking mechanism for function calls has been added — fprobe. The fprobe API is based on ftrace but is limited to the ability to attach callback handlers to function entry and exit points. Unlike kprobes and kretprobes, the new mechanism allows using one handler for several functions simultaneously.
  • Support for older ARM processors (ARMv4 and ARMv5), which lack a memory management unit (MMU), has been discontinued. Support for ARMv7-M systems without an MMU has been retained.
  • Support for the RISC-like NDS32 architecture, used in processors from Andes Technologies, has been discontinued. The code was removed due to lack of maintenance and low demand for NDS32 support in the main Linux kernel (remaining users rely on specialized kernel builds from hardware manufacturers).
  • By default, kernel builds with support for the a.out executable format for the alpha and m68k architectures, where this format is still in use, have been disabled. It is likely that support for the outdated a.out format will be completely removed from the kernel soon. Plans to remove the a.out format have been discussed since 2019.
  • Minimal support for the vDSO (virtual dynamic shared objects) mechanism has been implemented for the PA-RISC architecture, providing a limited set of system calls available in user space without context switching. vDSO support has enabled the ability to run with a non-executable stack.
  • Support for the Intel HFI (Hardware Feedback Interface) mechanism has been added, allowing hardware to convey information about the current performance and energy efficiency of each CPU to the kernel.
  • A driver for the Intel SDSi (Software-Defined Silicon) mechanism has been added, allowing management of enabling additional features in the processor (for example, specialized instructions and additional cache). The idea is that lower-priced chips may be supplied with locked extended features, which can later be 'purchased' and activated without hardware replacement of the chip.
  • The amd_hsmp driver has been added to support the AMD HSMP (Host System Management Port) interface, providing access to processor management features through a set of special registers available in AMD EPYC server processors starting from the Fam19h generation. For example, HSMP can provide data on power consumption and temperature, set frequency limits, activate various performance boost modes, and manage memory operation parameters.
  • The asynchronous I/O interface io_uring has implemented the IORING_SETUP_SUBMIT_ALL option for registering a set of file descriptors in the ring buffer, as well as the IORING_OP_MSG_RING operation, which allows signaling from one ring buffer to another.
  • The DAMOS (Data Access Monitoring-based Operation Schemes) mechanism, which allows for memory release based on access frequency, has expanded capabilities for monitoring memory operations from user space.
  • The third series of patches has been integrated, implementing the concept of page folios, which resemble compound pages but offer improved semantics and a clearer organizational approach. Utilizing folios speeds up memory management in certain kernel subsystems. The proposed patches involved transitioning internal memory management functions to folios, including variations of the get_user_pages() function. Support for creating large folios within data prefetching code has been provided.
  • The build system has introduced support for the USERCFLAGS and USERLDFLAGS environment variables, allowing the passing of additional flags to the compiler and linker.
  • In the eBPF subsystem, the BTF (BPF Type Format) mechanism, which provides type checking information in BPF pseudocode, has implemented the ability to add annotations to variables that reference memory areas in user space. Annotations help the BPF code verification system to more effectively identify and validate memory access.
  • A new memory allocation handler for storing loaded BPF programs has been proposed, which allows for more efficient memory usage in situations where a large number of BPF programs are loaded.
  • The madvise() system call, which provides means for optimizing memory management of a process, has been enhanced with the MADV_DONTNEED_LOCKED flag. This flag complements the existing MADV_DONTNEED flag, allowing the kernel to be informed in advance about the upcoming release of a memory block, indicating that this block is no longer needed and can be reused by the kernel. Unlike MADV_DONTNEED, the MADV_DONTNEED_LOCKED flag can be used for locked pages in RAM, which, when madvise is called, are evicted without changing their locked status, and if there is a subsequent access to the block resulting in a page fault, they return while maintaining their binding. Additionally, a change has been made to allow the MADV_DONTNEED flag to be used with large memory pages in HugeTLB.
• Virtualization and Security
  • Support for the Intel IBT (Indirect Branch Tracking) execution flow protection mechanism has been added for the x86 architecture. This mechanism prevents the use of exploit construction techniques utilizing Return-Oriented Programming (ROP), where an exploit is formed as a chain of calls to existing machine instruction fragments in memory, ending with a return control instruction (typically, function endings). The essence of the implemented protection method lies in blocking indirect jumps into the function body by adding a special ENDBR instruction at the beginning of the function and allowing execution through an indirect jump only if it leads to this instruction (an indirect call through JMP and CALL must always hit the ENDBR instruction, which is placed at the very start of the function).
  • Stricter boundary checks have been enabled for the memcpy(), memmove(), and memset() functions, which are performed at compile time when CONFIG_FORTIFY_SOURCE mode is enabled. The added change checks for out-of-bounds accesses to structure elements whose sizes are known. It is noted that this implemented feature could block all buffer overflows related to memcpy() in the kernel, identified over the past three years at least.
  • The second part of the code has been added to the updated implementation of the RDRAND pseudorandom number generator, which is responsible for the operation of the /dev/random and /dev/urandom devices. The new implementation is notable for unifying the operation of /dev/random and /dev/urandom, adding protection against the appearance of duplicates in the stream of random numbers at startup. of virtual machines Additionally, it transitions to the use of the BLAKE2s hash function instead of SHA1 for entropy mixing operations. This change enhances the security of the pseudorandom number generator by eliminating the problematic SHA1 algorithm and preventing the overwriting of the RNG initialization vector. Since the BLAKE2s algorithm outperforms SHA1 in terms of performance, its application has also positively impacted overall performance.
  • Support for the new pointer authentication algorithm 'QARMA3' has been added for the ARM64 architecture, which is faster than the QARMA algorithm while maintaining an adequate level of security. This technology allows for the use of specialized ARM64 instructions to verify return addresses using digital signatures stored in the unused upper bits of the pointer itself.
  • Support for building with the inclusion of GCC 12's return address protection mechanism has been implemented for the ARM64 architecture to prevent return address overwriting in case of stack buffer overflow. The essence of the protection lies in saving the return address to a separate 'shadow' stack after transferring control to the function and extracting this address before exiting the function.
  • A new key storage (keyring) named 'machine' has been added, containing the keys of the system owner (MOK, Machine Owner Keys) supported by the shim bootloader. The specified keys can be used to sign components of the kernel loaded post-initial boot with a digital signature (for example, kernel modules).
  • Support for asymmetric private keys for TPM, proposed in the outdated version of TPM, has been removed due to known security issues and the lack of practical adoption.
  • Protection against integer overflow for data types of size_t has been added. The code utilizes handlers size_mul(), size_add(), and size_sub(), allowing for safe multiplication, addition, and subtraction of sizes of type size_t.
  • The kernel compilation includes the flags "-Warray-bounds" and "-Wzero-length-bounds", which generate warnings when an index goes out of array bounds and when zero-length arrays are used.
  • The virtio-crypto device has added support for encryption using the RSA algorithm.
• Network subsystem
  • The implementation of network bridges has added support for port binding mode (locked mode), where a user can send traffic through a port only with a permitted MAC address. Additionally, multiple structures for assessing the state of the STP (Spanning Tree Protocol) have been added. Previously, VLANs could only be directly bound to STP (1:1), where each VLAN was managed independently. The new version introduces the mst_enable parameter, which, when enabled, allows VLAN states to be controlled by the MST (Multiple Spanning Trees) module, enabling VLAN bindings to conform to the M:N model.
  • Work has continued on integrating tools into the network stack for tracking reasons for packet drops (reason codes). The reason code is transmitted during the memory release associated with the packet and allows accounting for situations such as packet drops due to header field fill errors, spoofing detection by the rp_filter, incorrect checksums, memory shortages, triggering of IPSec XFRM rules, incorrect TCP sequence numbers, etc.
  • The ability to transmit network packets from BPF programs running in user space in BPF_PROG_RUN mode has been provided, where BPF programs execute in the kernel but return results to user space. Packets are transmitted using the XDP (eXpress Data Path) subsystem. Live packet processing mode is supported, where the XDP handler can dynamically redirect network packets to the network stack or other devices. It is also possible to create software generators of external traffic or to inject network frames into the network stack.
  • For BPF programs attached to network cgroups, auxiliary functions have been proposed to explicitly set the return value from system calls, allowing for more complete information on the reasons for blocking the system call.
  • The XDP (eXpress Data Path) subsystem has added support for fragmented packets distributed across multiple buffers, allowing for the processing of Jumbo frames in XDP and the application of TSO/GRO (TCP Segmentation Offload/Generic Receive Offload) for XDP_REDIRECT.
  • The process of removing network namespaces has been significantly accelerated, which was in demand for some large systems with high traffic volumes.
• Hardware
  • In the amdgpu driver, the FreeSync adaptive synchronization technology is now enabled by default, allowing the refresh rate of the display to be adjusted, ensuring smoothness and the absence of screen tearing during gaming and video viewing. Stable support for the Aldebaran GPU has been announced.
  • The i915 driver has added support for Intel Alderlake N chips and discrete Intel DG2-G12 graphics cards (Arc Alchemist).
  • The nouveau driver now supports higher bit rates for DP/eDP interfaces and supports lttprs (Link-Training Tunable PHY Repeaters) cable extenders.
  • In the drm (Direct Rendering Manager) subsystem, support for the nomodeset parameter has been added in drivers for armada, exynos, gma500, hyperv, imx, ingenic, mcde, mediatek, msm, omap, rcar-du, rockchip, sprd, sti, tegra, tilcdc, xen, and vc4, allowing the disabling of video mode switching at the kernel level and the use of hardware rendering acceleration, keeping only the functionality associated with the system framebuffer.
  • Support has been added for ARM SoCs including Qualcomm Snapdragon 625/632 (used in LG Nexus 5X and Fairphone FP3), Samsung Exynos 850, Samsung Exynos 7885 (used in Samsung Galaxy A8), Airoha (Mediatek/EcoNet) EN7523, Mediatek mt6582 (Prestigio PMT5008 3G tablet), Microchip Lan966, Renesas RZ/G2LC, RZ/V2L, Tesla FSD, TI K3/AM62, and i.MXRTxxxx.
  • Support has been added for ARM devices and boards from Broadcom (Raspberry Pi Zero 2 W), Qualcomm (Google Herobrine R1 Chromebook, SHIFT6mq, Samsung Galaxy Book2), Rockchip (Pine64 PineNote, Bananapi-R2-Pro, STM32 Emtrion emSBS, Samsung Galaxy Tab S, Prestigio PMT5008 3G tablet), Allwinner (A20-Marsboard), Amlogic (Amediatek X96-AIR, CYX A95XF3-AIR, Haochuangy H96-Max, Amlogic AQ222 and OSMC Vero 4K+), Aspeed (Quanta S6Q, ASRock ROMED8HM3), Marvell MVEBU/Armada (Ctera C200 V1 and V2 NAS), Mstar (DongShanPiOne, Miyoo Mini), and NXP i.MX (Protonic PRT8MM, emCON-MX8M Mini, Toradex Verdin, Gateworks GW7903).
  • Support has been added for AMD PDM, Atmel PDMC, Awinic AW8738, i.MX TLV320AIC31xx, Intel CS35L41, ESSX8336, Mediatek MT8181, nVidia Tegra234, Qualcomm SC7280, Renesas RZ/V2L, and Texas Instruments TAS585M audio systems and codecs. An initial implementation of the audio driver for the Intel AVS DSP chip has been added. Driver support for Intel ADL and Tegra234 has been updated, and changes have been made to improve audio support on devices from Dell, HP, Lenovo, ASUS, Samsung, and Clevo.

  Meanwhile, the Latin American Free Software Foundation has released a fully free kernel version 5.18 — Linux-libre 5.18-gnu, stripped of firmware and driver elements containing non-free components or sections of code, the scope of which is restricted by the manufacturer. This new release includes driver cleaning for MIPI DBI panels, VPU Amphion, WiFi MediaTek MT7986 WMAC, Mediatek MT7921U (USB), and Realtek 8852a/8852c, as well as sound chips Intel AVS and Texas Instruments TAS5805M. Additionally, DTS files have been cleaned for various Qualcomm SoCs with processors based on the AArch64 architecture. The code for cleaning blobs in AMD GPU, MediaTek MT7915, Silicon Labs WF200+ WiFi, Mellanox Spectrum Ethernet, Realtek rtw8852c, Qualcomm Q6V5, Wolfson ADSP, and MediaTek HCI UART drivers and subsystems has also been updated.

Source: opennet.ru