Kis Cook from Google has called for modernization of the bug fixing process in the Linux kernel.

Kees Cook, former chief systems administrator of kernel.org and leader of the Ubuntu Security Team, now working at Google to enhance the security of Android and ChromeOS, expressed concern over the current process of addressing bugs in stable kernel branches. Approximately one hundred patches are included in stable branches weekly, and after the window for changes closes for the next release, this figure approaches a thousand (maintainers hold back patches until the window closes, then publish the accumulated fixes all at once after '-rc1' formation), which is a significant burden and demands considerable resources for maintaining products based on the Linux kernel.

According to Kees, the process of handling bugs in the kernel does not receive adequate attention, and the kernel lacks at least 100 additional developers to coordinate efforts in this area. Core kernel developers regularly fix bugs, but there are no guarantees that these fixes will be backported to kernel variants used by third-party vendors. Users of various Linux kernel-based products also do not have visibility into what bugs have been fixed and which kernel version their devices are using. Ultimately, manufacturers are responsible for the security of their products, but given the high frequency of patch publication in stable kernel branches, they are faced with a decision—whether to backport all patches, selectively port the most critical ones, or ignore all fixes.

The optimal solution would be to transfer only the most critical fixes and vulnerabilities, but the main problem lies in pinpointing such issues from the overall stream. The majority of emerging problems result from the use of the C language, which requires great care when working with memory and pointers. The situation is further complicated by the fact that many potential vulnerability fixes are not assigned CVE identifiers or receive such identifiers some time after the fix is published. Under these conditions, manufacturers find it very difficult to separate secondary fixes from critical issues that impact security. Statistics show that over 40% of vulnerabilities are resolved before receiving a CVE, and on average, there is a three-month delay between the release of a fix and the assignment of a CVE (i.e., initially the fix is perceived as a regular bug, but only months later does it become clear that it concerned a vulnerability).

As a result, without a separate branch for fixing vulnerabilities and lacking information about the security implications of specific issues, product manufacturers based on the Linux kernel are left with the continuous task of transferring all fixes from the latest stable branches. However, this work requires significant labor resources and faces resistance within companies due to the fear of introducing regressions that could disrupt normal product operation.

It is worth noting that, according to Linus Torvalds, all bugs are important, and vulnerabilities should not be separated from other types of bugs and prioritized into a separate category. This perspective is explained by the fact that for an ordinary developer who does not specialize in security issues, the connection between a fix and a potential vulnerability is not obvious (for many fixes, only a separate audit can reveal that they relate to security). According to Linus, it is the role of security specialists within communities responsible for maintaining kernel packages in Linux distributions to identify potential vulnerabilities from the general stream of fixes.

Kees Cook believes that the only solution to maintain kernel security while keeping long-term costs reasonable is for companies to transfer engineers, who are porting patches to local kernel builds, to a coordinated collaborative effort to maintain fixes and vulnerabilities in the main kernel (upstream). Currently, many manufacturers use outdated versions of the kernel in their products and backport patches on their own, which results in engineers at different companies duplicating each other's work to solve the same problem.

For instance, if 10 companies, each with one engineer backporting the same patches, redirect these engineers to fix bugs in upstream, instead of porting one patch, they could fix 10 different bugs for the common good or participate in reviewing proposed changes to prevent the inclusion of erroneous code in the kernel. Resources could also be directed towards creating new tools for testing and code analysis, which would enable the early detection of recurring types of errors.

Kees Cook also suggests making more active use of automated and fuzz testing directly during the kernel development process, applying continuous integration systems, and moving away from archaic email-based project management. Currently, effective testing is hindered by the fact that major testing processes are separated from development and occur only after releases are formed. Kees also recommended using languages that ensure better safety during development to reduce the number of errors. a high level of security, such as Rust.

Source: opennet.ru