Release of Linux Kernel 6.9

After two months of development, Linus Torvalds released the Linux kernel 6.9. Among the most notable changes are the dm-vdo module for deduplication and compression of block devices, a direct access mode for files in FUSE, support for creating pidfd for individual threads, the BPF token mechanism, Rust support on ARM64 systems, the classification of the Ext2 FS as deprecated, removal of the old NTFS driver, and support for the Intel FRED mechanism.

The new version includes 15,680 fixes from 2,106 developers, with a patch size of 54 MB (the changes affected 11,825 files, adding 687,954 lines of code and removing 225,344 lines). In the previous release, there were 15,641 fixes from 2,018 developers with a patch size of 44 MB. About 42% of all changes presented in 6.9 are related to device drivers, approximately 17% pertain to updates in architecture-specific code, 13% relate to the network stack, 7% to file systems, and 4% to internal kernel subsystems.

Key innovations in kernel 6.9:

• Disk subsystem, input/output, and file systems
  • A new dm-vdo (virtual data optimizer) handler has been added to the Device Mapper (DM), allowing for the realization of a virtual block device based on existing block devices with features such as deduplication of repeating data, data compression, exclusion of empty blocks, and increasing the size of the block device as needed (thin provisioning). These capabilities are implemented at the block device level and are independent of the file system used (for example, dm-vdo can automatically merge duplicate data and store information in a compressed format for any FS). Use of dm-vdo is supported for physical storage up to 256TB, and logical volumes can be created of up to 4PB. It is recommended to use lvm for managing vdo partitions. The VDO technology was developed by Permabit and was opened after its acquisition by Red Hat in 2017.
  • In the FUSE subsystem, applied for implementing file systems in user space, an initial implementation of the 'passthrough' mode has been added, allowing direct access to file data at the kernel level while bypassing the user-space process. This can significantly improve performance in certain situations. For example, FUSE FS implementations that operate in read-only mode and restrict access to files can deliver the contents of files from the underlying FS without passing them through the FUSE process.
  • The driver implementing the Ext2 file system has been marked as deprecated. The reason given is that the driver only supports 32-bit timestamps in inodes, which will overflow on January 19, 2038. It is recommended to use the ext4 driver instead, which supports Ext2 file systems and is fully compatible with them, but may use timestamps in Ext2 partitions that are not affected by the 2038 problem, provided the FS was created with inodes larger than 255 bytes (in the ext2 driver, 32-bit timestamps were used regardless of the inode size).
  • The old NTFS file system driver has been removed, and starting with version 5.15, it has been replaced by a new NTFS3 driver. Having two drivers implementing NTFS in the kernel has been deemed unnecessary, given that the old driver had not been updated for many years, is in a poor state, and can only operate in read-only mode.
  • Support for mapping user IDs of mounted file systems has been added to the zonefs and hugetlbfs file systems, which is used to map files of a certain user on a mounted foreign partition to another user in the current system.
  • In NFSv4, administrators have been given the ability to clear file open and locking states.
  • For the Ext4 file system, only bug fixes and updates to kunit tests are noted.
  • In Btrfs, the transition of functions to the use of page folios has continued.
  • In the XFS file system, work has continued on implementing the ability to use the fsck utility for checking and fixing identified problems in online mode, without unmounting the file system.
  • The system call pwritev2() has been updated to include the RWF_NOAPPEND flag, which allows specifying an offset for writing, even if the file was opened in append mode.
  • New ioctl commands have been added: FS_IOC_GETUUID - returns the UUID identifier of the specified filesystem, and FS_IOC_GETFSSYSFSPATH - determines the location in /sys/fs of the mounted filesystem.
  • The efs, qnx4, and coda filesystems have been transitioned to use the new partition mounting API.
  • The implementation of file operations in case-insensitive mode has been improved. Performance has increased by initially performing case-sensitive comparisons and reverting to case-insensitive searches. Issues have been resolved when mounting overlayfs on directories with case-insensitive settings.
• Memory and system services
  • Support for the Intel FRED (Flexible Return and Event Delivery) mechanism has been implemented, designed to enhance the efficiency and reliability of low-level event delivery compared to the currently used IDT (Interrupt Descriptor Table) mechanism. Performance improvements and reduced latency are achieved by returning events via the IRET processor instruction rather than passing them through the IDT table. Reliability is enhanced due to the separate handling of event arrival in kernel context and user context, protection against nested NMI execution, and saving all CPU registers related to exceptions in an extended stack frame.
  • The ability to optimize data access for individual CPU cores has been added through the use of named address spaces implemented in the GCC as an extension of GNU C.
  • The PIDFD_THREAD flag has been added to the pidfd_open() function, allowing the creation of pidfd for individual threads rather than only using pidfd in the context of the thread group leader. A pseudo-filesystem has also been proposed for access to pidfd through the virtual filesystem. Unlike process identification using pid, the pidfd identifier is linked to a specific process and does not change, while the PID may be reassigned to another process after the current process completes.
  • The BPF subsystem has introduced a BPF-token mechanism that allows selective delegation of access rights to privileged BPF operations for applications. For example, it is possible to grant an unprivileged application access to specific BPF subsystems without providing full CAP_BPF rights.
  • A new type of shared memory, bpf_arena, has been added to the BPF subsystem, defining an area accessible for sharing between BPF programs and user-space processes. The may_goto instruction has been added, enabling the organization of loop operations that can be interrupted by the verifier. It is now possible to generate arbitrary TCP SYN cookies from BPF programs and create BPF handlers to combat SYN flooding.
  • Continued the integration of changes from the Rust-for-Linux branch related to using the Rust language as a second language for developing drivers and kernel modules (Rust support is not enabled by default and does not make Rust a mandatory build dependency of the kernel). Rust support has been added for 64-bit ARM processors. A transition to Rust 1.76 has been made. The ‘container_of!’ macro has been added. Instead of the unstable ‘ptr_metadata’ functionality, the stable method ‘byte_sub’ is now employed. A ‘time’ module has been added with the time conversion function ‘msecs_to_jiffies()’.
  • The io_uring subsystem has gained the ability to truncate files (ftruncate_file).
  • A new type of workqueues, WQ_BH (workqueue Bottom Halves), has been added for asynchronous code execution in the context of software interrupts, aimed at replacing deprecated tasklets.
  • The timer subsystem has been significantly reworked, improving the logic for selecting the active CPU core to execute the triggered timer, so as not to wake up inactive cores from sleep.
  • The ability to update the core energy consumption model (EM, Energy Model) during operation has been implemented, which can be used, for example, to account for the impact of operating temperature on CPU energy efficiency. The performance of the em_cpu_energy() function has been significantly enhanced, now running 1.43 times faster in tests on stationary systems, and 1.69 times faster in tests on the RockPi 4B board.
  • Support has been added for running ARM64-based systems in LPA2 mode with a 52-bit virtual address space.
  • Continuous recording of PTE (Page Table Entry) has been implemented for ARM64 systems, increasing performance by enhancing the efficiency of TLB (Translation Lookaside Buffer) usage.
  • Patches have been accepted to enhance the performance of the memory management subsystem by reducing the occurrence of competing locks in vmalloc().
  • A live patching mechanism has been implemented for the LoongArch architecture, allowing for kernel fixes to be applied without rebooting.
  • Support for the membarrier() system call has been implemented for RISC-V systems, providing memory barriers for threads running in the system.
  • Requirements have been raised for the version of LLVM/Clang that can be used to build the kernel. Building now requires at least LLVM release 13.0.1 (previously supported builds in LLVM 11+).
  • The 'User trace events' mechanism, which allows creating trace events from user processes to track user space activity, has been enhanced with support for exporting event information in various formats (USER_EVENT_REG_MULTI_FORMAT).
  • The function call tracing mechanism has been enhanced to track the state of incoming function arguments when tracing function exits. The return operator values can now be matched with the arguments used during the function call.
  • The perf utility has added support for the 'cluster' output aggregation mode ('perf stat -a --per-cluster') for combining shared resource statistics. The ability to utilize the libcapstone library for disassembling processor instructions ('perf script -F disasm') has been implemented. Memory consumption optimizations have been made while executing the 'perf report' and 'perf annotate' commands.
• Virtualization and Security
  • Protection against the RFDS (Register File Data Sampling) vulnerability has been added in Intel Atom processors, which allows extraction of residual information from the processor's register files (RF, Register File) used for jointly storing the contents of registers across all tasks on the same CPU core. To block the vulnerability, a microcode update is required, and the VERW instruction should be used to clear the contents of microarchitectural buffers when returning from kernel to user space. To enable protection at kernel boot, the flag "reg_file_data_sampling=on" can be specified. Information about susceptibility to the vulnerability and the availability of necessary microcode for protection can be assessed in the file "\/sys\/devices\/system\/cpu\/vulnerabilities\/reg_file_data_sampling".
  • Basic support for guest system protection has been added through the AMD SEV-SNP (Secure Nested Paging) extension, which ensures secure operation with nested page tables and protects against "undeSErVed" and "SEVerity" attacks on AMD EPYC processors, which allow circumventing the AMD SEV (Secure Encrypted Virtualization) protection mechanism. KVM The necessary changes for using SNP are planned to be added in the 6.10 branch.
  • Modules implementing the IMA (Integrity Measurement Architecture) and EVM (Extended Verification Module) technologies have been transitioned to use the LSM (Linux Security Modules) framework, which significantly simplifies the code without loss of functionality, consolidating duplicated functionality and leveraging standard features available through LSM. The IMA module is intended for verifying the integrity of operating system components through digital signatures and hashes. The EVM module protects extended file attributes (xattrs) from attacks aimed at compromising their integrity (EVM will not allow an offline attack where a malicious actor modifies metadata, for example, by booting from their own storage device).
  • The system calls lsm_list_modules(), lsm_get_self_attr(), and lsm_set_self_attr() have been redesigned for greater compatibility with 32-bit environments. These calls are intended to output the list of loaded LSM modules (Linux Security Modules) and get/set attributes of an LSM module. The change breaks backward compatibility, but since the new system calls were added in the previous kernel release and are not yet used in applications, Linus Torvalds deemed the change acceptable.
  • An attempt has been made to resume the use of the UBSAN (Undefined Behavior Sanitizer) mechanism. The essence of the problem is that compilers handle integer overflow for signed and unsigned types differently. Signed overflows and pointer overflows fall under the category of undefined behavior, while unsigned overflows are truncated modulo 2n, retaining only the lower bits of the result ('wrap-around') and do not fall under undefined behavior. To eliminate situations that could lead to undefined behavior, the kernel is built with the option '-fno-strict-overflow', which leads to the use of 'wrap-around' for all integer overflows. GCC and Clang cannot accurately diagnose certain issues when using the '-fno-strict-overflow' flag, and enabling UBSAN is aimed at conducting collaborative work with compiler developers to eliminate false positives and identify integer overflows in places where explicit checks are absent.

    To check for potential overflows in the kernel, structures like 'var + offset PAGE_SHIFT) < pgoff){..}') are used, which are tied to building with the flag '-fno-strict-overflow' and do not cover all code where an overflow could potentially occur. The problem is that when using UBSAN, such checks led to a large number of false warnings, and due to this, UBSAN had to be disabled in 2021. The updated implementation proposes using special annotations __signed_wrap and __unsigned_wrap, as well as ready-made macros with checks add_would_overflow(a, b) and add_wrap(a, b), which allow separating the intended use of integer overflows by developers from the occurrence of accidental overflows that could lead to vulnerabilities. The proposal for a more extensive redesign of the kernel introducing additional type definitions was rejected by Linus Torvalds.

• Network subsystem
  • In the network subsystem, work has been done to reduce the occurrence of lock contention (the attempt to acquire a lock held by another thread). The use of RTNL locks has been minimized.
  • Added the ability to enable busy polling support in the context of individual epoll calls. The pool size and budget parameters can be set separately from the default system parameters.
  • Implemented the net_hotdata structure to enhance the caching efficiency of the most commonly used network configuration variables.
  • MPTCP has added support for the TCP_NOTSENT_LOWAT option for sockets, allowing limitation of the send buffer size. The MCTP socket API now includes support for network IDs, enabling the use of multiple non-overlapping MCTP networks on a single host.
  • IPSec has added support for redirecting ICMP error messages (RFC 4301).
  • The route scanning process has been accelerated for expired lifetimes.
  • XDP performance has been improved through stricter avoidance of allocating large memory blocks.
  • The ability to attach metadata to netconsole messages has been added.
  • In Netfilter, user-space-defined tables can now be linked to a controlling background process and will not be automatically removed after the user application finishes.
  • In nftables, adding elements to set collections with combined ranges has been accelerated.
• Hardware
  • The i915 driver continues the work on supporting Intel LunarLake (Xe 2) chips. New PCI identifiers have been added for devices based on Intel Arrow Lake and Alder Lake N chips. DisplayPort now supports tunneling (DP tunneling) and bandwidth allocation. Fastboot mode has been enabled for all platforms. Support for debug output tied to individual devices has also been added.
  • In the AMDGPU driver, preparations have been made to support AMD RDNA3.5 and RDNA4 GPUs. Support for ATHUB 4.1, LSDMA 7.0, JPEG DPG, IH 7.0, HDP 7.0, VCN 5.0, SMU 13.0.6, NBIO 7.11, SDMA 6.1, MMHUB 3.3, DCN 3.5.1, NBIF 6.3.1, VPE 6.1.1, and the RAS ACA framework has been added. A kernel module parameter freesync_video has been included to enable experimental support for optimizing video mode switching using FreeSync technology.
  • In the Nouveau driver, screen management code has been converted to use the kmemdup() function.
  • Work on the drm driver (Direct Rendering Manager) for Intel Xe architecture GPUs, used in Intel Arc graphics cards and integrated graphics starting from Tiger Lake processors, has continued.
  • A DRM driver has been added for Mediatek MT8188 VDOSYS1 chips.
  • Kernel settings related to video subsystems have been moved to the CONFIG_VIDEO section.
  • Support for ARM64 SoCs: Mediatek MT7981B (Filogic 820), MT7988A (Filogic 880), NXP i.MX8DXP, Renesas R8A779G2 (R-Car V4H ES2.0), R8A779H0 (R-Car V4M), TI J722S has been added.
  • Support for ARM boards and devices has been added: Android phones based on Tegra30, Chromebook models based on Mediatek MT8186, NAS, tablets, and gaming consoles based on Rockchips RK35xx, White Hawk boards based on Renesas SoCs, boards based on Qualcomm SM8550 (Snapdragon 8 Gen 2), Apalis Evaluation Board, Sielaff i.MX6 Solo Board, Samsung Galaxy Tab 4 10.1 LTE.
  • The code for the ALSA audio subsystem has been refactored. Support for Microchip SAM9x7, NXP i.MX95, and Qualcomm WCD939x audio systems has been added. The SoundWire driver now supports ASoC with AMD ACP 6.3 audio coprocessors, and for Intel systems, a DSPless mode has been implemented. Additional audio codecs Cirrus HD have been supported. The virtio driver has improved audio device management.
  • Support for Marvell Octeon PCI Endpoint NIC VF and Intel E825-C 100G Ethernet controllers has been added.

At the same time, the Latin American Free Software Foundation has released a completely free version of the kernel 6.9 — Linux-libre 6.9-gnu, stripped of firmware and driver elements containing non-free components or code snippets limited by the manufacturer. In the 6.9 release, the blob cleaning code has been updated for the amdgpu, ath12k, adreno, btusb, and r8169 drivers. Cleaning of the new ptp_fc3 driver has been conducted. Blob names in dts files (devicetree) for the Aarch64 architecture have been cleaned up. Issues with the cleaning of the i915 driver that caused hangs during initialization have been resolved. Changes regarding the handling of blobs supplied as hexadecimal dumps have been introduced.

Source: opennet.ru