Vyacheslav Dubeyko from IBM initiated a discussion in the kernel developers' mailing list regarding the use of machine learning models in the kernel, and also offered a set of patches for testing that included a library for integrating ML models into the kernel and an example of a symbolic driver utilizing the library.
Integrating ML models into the kernel can be useful for modifying the logic of subsystems based on processed data, optimizing performance, and changing configurations according to the internal state of systems. The use of machine learning, which can identify patterns and make predictions without manual algorithm implementation, will simplify the selection of the most effective kernel configuration considering the complexity and variability of modern workloads, as well as address issues such as predicting failures in storage systems.
In addition to the engine for executing models, the development of data collection tools for training ML models, the actual training of the model, and testing the results is being considered. Since ML model execution requires floating-point operations, and direct use of FPU is not allowed in the kernel, the proposed prototype serves as a layer for allowing various kernel subsystems to access ML models executed in user space, similar to how SPDK, DPDK, and ublk handlers are moved to user space.
Moving the execution and training of the model to user space simplifies maintenance and isolates the kernel from issues arising in the model execution code. During the training phase, data about the kernel's state and parameters can either be requested by the handler from user space or passed by the layer operating at the kernel level. Sysfs is used to manage the interaction between components in the kernel and user space. Adaptive training of the ML model may occur, where the kernel subsystem receives recommendations from the ML model, applies the suggested changes, and evaluates the effectiveness of the recommendation in modifying the state.
Source: opennet.ru
