Release of Linux Kernel 5.16

After two months of development, Linus Torvalds has released the Linux kernel 5.16. Among the most notable changes are: the system call futex_waitv to improve the performance of Windows games in Wine, error tracking in the filesystem via fanotify, the introduction of folios in memory management, support for AMX processor instructions, the ability to reserve memory for network sockets, support in netfilter for packet classification during the 'egress' stage, activation of the DAMON subsystem for proactive eviction of unused memory areas, enhanced handling of overloads during high-volume write operations, and support for multi-drive hard disks.

The new version includes 15,415 fixes from 2,105 developers, with a patch size of 45 MB (changes affected 12,023 files, added 685,198 lines of code, and removed 263,867 lines). About 44% of all changes presented in 5.16 relate to device drivers, approximately 16% pertain to updates specific to hardware architecture, 16% are related to the network stack, 4% involve filesystems, and 4% concern internal kernel subsystems.

Key innovations in kernel 5.16:

• Disk subsystem, input/output, and file systems
  • The fanotify mechanism has been enhanced with tools for monitoring the state of the filesystem and tracking errors. Error information is transmitted using a new type of event—FAN_FS_ERROR—which can be intercepted by user-space monitoring systems for timely alerts to the administrator or triggering recovery processes. In the event of a cascading series of errors, fanotify ensures delivery of the first error message along with a total problem counter to facilitate subsequent diagnosis of the failure causes. Error tracking support is currently implemented only for the Ext4 filesystem.
  • Improved handling of write congestion that occurs when the volume of write operations exceeds the storage bandwidth, forcing the system to block write process requests until the completion of already submitted requests. The new version completely redesigns the kernel mechanism used to gather information about the occurrence of congestion and task blocking, as the old implementation showed issues with the coupling of write congestion handling with memory page swapping in case of system memory shortages.
  • Btrfs now supports Zoned Namespace technology, used in hard drives or NVMe SSDs for dividing storage space into zones, comprising groups of blocks or sectors where only sequential data addition is permitted, with entire groups of blocks needing to be updated. Additionally, minor optimizations in inode logging have been made, leading to a 3% increase in throughput in the dbench test and reducing latency by 11%. The directory logging mechanism has been redesigned, reducing the number of search operations and locks in the tree for improved efficiency. Element insertion into the btree structure in batch mode has been accelerated (bulk insertion time has decreased by 4%, while deletions have improved by 12%). Limited support for compression during the write of incomplete pages has been added, along with the capability for subpage defragmentation. Preparations have been made for enabling support for the second version of the protocol for the 'send' command.
  • In the XFS file system, memory consumption has been reduced by utilizing separate slab caches for frequently used elements and optimizing some data structures.
  • In the Ext4 file system, only bug fixes and more accurate calculations of delayed initialization parameters for the Inode table have been noted.
  • At the block device level, optimizations have been implemented that significantly enhance the efficiency of binding operations to CPU cores.
  • Initial support for hard drives with multiple independent actuators has been added, enabling simultaneous access to multiple sectors in different zones of the magnetic platter.
  • A new ioctl command CDROM_TIMED_MEDIA_CHANGE has been added to detect media change events in optical drives.
  • The EROFS (Enhanced Read-Only File System) now supports operating over multiple storage devices. Different devices can be mapped into a single 32-bit block address space. Compression support using the LZMA algorithm has also been added.
  • The F2FS file system has introduced mount options to manage file fragmentation when placing in storage (for example, for debugging optimizations with fragmented storage).
  • CEPH now includes the ability to create and delete directories in asynchronous mode by default (to return to the old behavior, use the ‘-o wsync’ flag when mounting). Metrics tracking external object copy operations have been added.
  • CIFS has added a mounting parameter tcpnodelay, which sets the tcp_sock_set_nodelay mode for the network socket, disabling queue fill waiting in the TCP stack. Support for nested DFS Links (Distributed File System) when remounting has also been added.
  • Support has been added for batching block device request completions. Testing of the change has shown an increase in random read operation intensity from Optane storage from 6.1 to 6.6 million IOPS on a single CPU core.
• Memory and system services
  • A new system call futex_waitv has been introduced, allowing the state of multiple futexes to be monitored with a single system call. This feature resembles the Windows functionality WaitForMultipleObjects, whose emulation through futex_waitv can be beneficial for enhancing the performance of Windows games running under Wine or Proton. Additionally, simultaneous futex waiting can also be applied to optimize performance for native Linux game builds.
  • The concept of page folios has been implemented, which will allow for faster memory management under standard loads in certain subsystems of the kernel. Currently, the main memory management subsystem in the kernel and the page cache implementation have already been transitioned to folios, and future plans include transitioning file systems. Additionally, support for multi-page folios is planned for future releases of the kernel.

    Folios resemble compound pages but are distinguished by improved semantics and a more comprehensible organizational structure. For managing system memory, available RAM is divided into memory pages whose size depends on the architecture, typically measured in kilobytes (usually 4096 bytes) on x86 systems. Modern systems are equipped with tens of gigabytes of RAM, complicating memory management due to the need to handle vast numbers of memory pages. To reduce the number of pages, the concept of compound pages was previously implemented in the kernel, using structures that encompass more than one physical memory page. However, the API for manipulating compound pages left much to be desired and resulted in additional overhead.

  • A handler has been added to the task scheduler that takes cache clustering in the CPU into account. In some processors, such as the Kunpeng 920 (ARM) and Intel Jacobsville (x86), a certain number of CPU cores, typically 4, can share L3 or L2 cache. Considering such topologies can significantly enhance the efficiency of task distribution across CPU cores in the task scheduler, as moving tasks within a single CPU cluster increases memory access throughput and reduces cache contention.
  • Support for Advanced Matrix Extensions (AMX) instructions has been added, which are implemented in the upcoming Intel Xeon Scalable server processors, codenamed Sapphire Rapids. AMX offers new customizable registers TMM "TILE" and instructions for manipulating data in these registers, such as TMUL (Tile matrix MULtiply) for matrix multiplication.
  • Several new features have been implemented, based on the DAMON (Data Access MONitor) subsystem introduced in the previous release, which allows tracking data access in memory in relation to a selected process running in user space. For instance, the subsystem enables analysis of which specific memory areas the process accessed throughout its operation, as well as which memory areas remained unutilized.
    • DAMON_RECLAIM for identifying and evicting memory areas that have not been accessed. This mechanism can be used for proactive soft eviction of memory pages when nearing depletion of free memory.
    • DAMOS (Data Access Monitoring-based Operation Schemes) for applying specified madvise() operations, such as freeing additional free memory, to memory areas of a process that have a recorded access frequency. Parameter configuration for DAMOS is done through debugfs.
    • The ability to monitor physical address space in memory (previously, only virtual addresses could be tracked).
  • The zstd compression algorithm implementation has been updated to version 1.4.10, significantly enhancing the performance of various kernel subsystems that utilize compression (for example, kernel image unpacking is sped up by 35%, while the performance of unpacking compressed data in Btrfs and SquashFS increased by 15%, and in ZRAM by 30%). Previously, the kernel utilized a separate implementation of zstd based on version 1.3.1, released over three years ago, which did not include many important optimizations. In addition to upgrading to the current version, the added patch also simplifies synchronization with the upstream zstd branch, allowing for direct code generation for inclusion in the kernel from the main zstd repository. In the future, the zstd code in the kernel is planned to be updated as new versions of the zstd library are released.
  • A significant number of improvements have been made to the eBPF subsystem. It now allows calling kernel module functions from BPF programs. The function bpf_trace_vprintk() has been implemented, which unlike bpf_trace_printk() allows for printing more than three arguments at once. A new data storage structure (BPF map) bloom filter has been added, permitting use of the similarly named probabilistic data structure to determine the presence of an element in a set. A new attribute BTF_KIND_TAG has been added, which can be used in BPF programs to bind tags to function parameters, for example, to simplify error detection in user programs. In libbpf, the creation of custom sections .rodata.* / .data.* is allowed, support for uprobe and kprobe trace events has been implemented, and an API for copying all types of BTF from one object to another has been added. Support for AF_XDP has been moved from libbpf to a separate library, libxdp. A JIT compiler for the BPF virtual machine has been implemented for the MIPS architecture.
  • For the ARM64 architecture, support for the ARMv8.6 timer extensions has been implemented, including those that ensure self-synchronization of system register views without using ISB instructions.
  • For the PA-RISC architecture, the ability to use the KFENCE mechanism to detect memory errors has been implemented, as well as support for the KCSAN race condition detector.
  • The ability to configure tracefs access rights at the level of individual users and groups has been provided; for example, access to tracing tools can now be restricted to members of a specific group.
• Virtualization and Security
  • In the io_uring and device-mapper subsystems, support for generating audit events has been implemented. In io_uring, access control through LSM modules can now be managed. The auditing of the openat2() system call has also been added.
  • The kernel code has been completely rid of uninterruptible case statements in switch (without return or break after each case block). When building the kernel, the "-Wimplicit-fallthrough" mode can now be applied.
  • Changes have been included to tighten boundary checks when executing the memcpy() function.
  • In the io_uring asynchronous input/output interface, it is now possible to apply security policies to input/output operations, defined by SELinux and Smack modules.
  • In the IMA (Integrity Measurement Architecture) subsystem, which allows an external service to verify the state of core subsystems to ensure their authenticity, the ability to apply rules based on the group identifier (GID) to which the file belongs or in which the user accessing the file is included has been implemented.
  • Some advanced thread protection mechanisms, seccomp(), against Spectre-class attacks are disabled by default, as they were deemed excessive and did not significantly enhance security while negatively impacting performance. The use of Retpoline protection has been revised.
  • The cryptoloop mechanism has been removed, which was replaced in 2004 by dm-crypt and still supports the same algorithms if necessary.
  • By default, unprivileged access to the eBPF subsystem is prohibited. This change has been made to prevent the use of BPF programs to bypass protection against side-channel attacks. If necessary, an administrator can restore unprivileged user access to eBPF.
  • The ACRN hypervisor, designed for real-time tasks and use in critical systems, has added support for creating/deleting virtual devices and for passing through MMIO devices.
  • Support for KPP (Key-agreement Protocol Primitives) definitions has been added to the cryptographic engine, simplifying the development logic for cryptosystem drivers.
  • Isolation mode has been implemented for the Hyper-V hypervisor, of virtual machineswhich involves encrypting the contents of memory.
  • In the hypervisor KVM support for the RISC-V architecture has been added. The possibility of migrating virtual machines running under an AMD SEV and SEV-ES environment within the host environment has been implemented. An API for live migration of guest systems encrypted using AMD SEV (Secure Encrypted Virtualization) has been added.
  • For the PowerPC architecture, the STRICT_KERNEL_RWX mode is enabled by default, blocking the use of memory pages that are accessible for both writing and execution.
  • Support for memory hotplug has been discontinued on 32-bit x86 systems, which has been inoperable for over a year.
  • The liblockdep library has been removed from the kernel, and will now be maintained separately from the kernel.
• Network subsystem
  • A new option SO_RESERVE_MEM has been implemented for sockets, allowing a specific amount of memory to be reserved for the socket that will always remain available and not be reclaimed. Using this option can improve performance by reducing memory allocation and deallocation operations in the network stack, especially during conditions of memory shortage in the system.
  • Support for the Automatic Multicast Tunneling protocol (RFC 7450) has been added, enabling the delivery of multicast traffic from multicast-enabled networks to receivers in networks without multicast. The protocol operates by encapsulating the traffic in UDP packets.
  • Improvements have been made in the encapsulation of IOAM (In-situ Operations, Administration, and Maintenance) data within transit packets.
  • The ethtool netlink API now provides the ability to manage the power consumption modes of transceivers.
  • The netfilter subsystem has implemented packet classification at the egress level, which occurs when the driver receives a packet from the kernel's network stack. Support for the appropriate filters in nftables was introduced in version 1.0.1. Netfilter now allows for matching and modifying inner headers and data for UDP and TCP (inner header / payload) that follow the transport header.
  • New sysctl parameters arp_evict_nocarrier and ndisc_evict_nocarrier have been added, which will clear the ARP cache and the ndisc (neighbor discovery) table in case of link drop (NOCARRIER).
  • The fq_codel (Controlled Delay) queuing management mechanism has introduced Low Latency, Low Loss, and Scalable Throughput (L4S) modes.
• Hardware
  • Initial support for the DP 2.0 (DisplayPort 2.0) specification and tunneling DisplayPort over USB4 has been implemented in the amdgpu driver. Support for display controllers has been added for the APU Cyan Skillfish (equipped with Navi 1x GPU). Support for APU Yellow Carp (mobile Ryzen 6000 "Rembrandt" processors) has been expanded.
  • Support for Intel Alder Lake S chips has been stabilized in the i915 driver, along with the implementation of Intel PXP (Protected Xe Path) technology, which enables the operation of a hardware-protected graphics session on systems with Intel Xe chips.
  • The nouveau driver has undergone bug fixes and improvements in code styling.
  • Support for x86-compatible Vortex CPUs (Vortex86MX) has been added. Linux has previously worked on such processors, but explicit identification of these CPUs was needed to disable protections against Spectre/Meltdown attacks, which are not applicable to these chips.
  • Initial support for x86 platforms Surface Pro 8 and Surface Laptop Studio has been added.
  • A driver has been added for sound chips used in AMD Yellow Carp and Van Gogh APU, as well as support for sound systems and codecs Cirrus CS35L41, Maxim MAX98520/MAX98360A, Mediatek MT8195, Nuvoton NAU8821, NVIDIA Tegra210, NXP i.MX8ULP, Qualcomm AudioReach, Realtek ALC5682I-VS, RT5682S, RT9120, Rockchip RV1126, and RK3568.
  • The ishtp_eclite driver has been added for accessing Intel PSE (Programmable Service Engine) embedded controllers using the ISHTP (Integrated Sensor Hub Transport Protocol), for example, to obtain data about the battery, temperature, and information related to UCSI (USB Type-C Connector System Software Interface).
  • A driver for Nintendo Switch game controllers has been added, supporting Switch Pro and Joy-Con devices. Support for Wacom Intuos BT tablets (CTL-4100WL/CTL-6100WL) and the Apple 2021 Magic Keyboard has also been added. Improved support for Sony PlayStation DualSense controllers is included. Support for side buttons on Xiaomi Mi mouse has been added.
  • The RT89 driver has been added with support for Realtek 802.11ax wireless chips, as well as drivers for Asix AX88796C-SPI Ethernet adapters and Realtek RTL8365MB-VC switches.
  • For Apple M1 chips, drivers for PCI and PASemi i2c have been added.
  • Support for ARM SoCs, devices, and boards Raspberry Pi Compute Module 4, Fairphone 4, Snapdragon 690, LG G Watch R, Sony Xperia 10 III, Samsung Galaxy S4 Mini Value Edition, Xiaomi MSM8996 (Mi 5, Mi Note 2, Mi 5s, Mi Mix, Mi 5s Plus, and Xiaomi Mi 5), Sony Yoshino (Sony Xperia XZ1, and Sony Xperia XZ Premium), F(x)tec Pro1 QX1000, Microchip LAN966, CalAmp LMU5000, Exegin Q5xR5, sama7g5, Samsung ExynosAutov9, Rockchip RK3566, RK3399 ROCK Pi 4A+, RK3399 ROCK Pi 4B+, Firefly ROC-RK3328-PC, Firefly ROC-RK3399-PC-PLUS, ASUS Chromebook Tablet CT100, Pine64 Quartz64-A, Netgear GS110EMX, Globalscale MOCHAbin 7040, NXP S32G2, Renesas R8A779M*, Xilinx Kria, Radxa Zero, JetHub D1/H1, and Netronix E70K02 has been added.

Source: opennet.ru