Report on the development of the FreeBSD project from April to June 2019. Key changes include:
- General and System Issues
- The core team has decided to establish a working group to study the feasibility of migrating source texts from the centralized version control system Subversion to a decentralized system Git.
- Fuzz testing of the FreeBSD kernel has been conducted using the system and a number of bugs discovered during this process have been fixed. A layer for fuzz testing libraries was added for compatibility with 32-bit environments on 64-bit kernel systems. The ability to run syzkaller in bhyve-based virtual machines has been implemented. The next phase plans to expand testing coverage of system calls, employ LLVM sanitizer for kernel verification, use netdump for saving kernel dumps during crashes in fuzz testing, etc.
- Work has begun on updating the implementation of zlib at the kernel level. To ensure the kernel's access to the zlib code, the contrib/zlib directory has been renamed to sys/contrib/zlib, and the header file crc.h has also been renamed to avoid a conflict with zlib/crc.h. Cleanup of outdated code dependent on zlib and inflate has been conducted. Future plans include enabling the kernel to be built simultaneously with the old and new zlib for a gradual transition to the new version of compression functions;
- The infrastructure for Linux environment emulation (Linuxulator) has been updated. Support for Linux debugging tools, such as the strace utility, has been expanded. The linux-c7-strace package has been added to the ports, which can be used for tracing Linux executables instead of the standard truss and ktrace utilities that currently cannot decode some Linux-specific flags and structures. Additionally, the linux-ltp package with Linux Test Project executables has been added and compatibility issues with executables built with newer versions of glibc have been resolved;
- The implementation of deferred binding clean-up operations for physical memory pages (delayed invalidation) in the pmap mechanism has been transitioned to use a lock-free queue processing algorithm, which has resolved scalability issues during a high number of concurrent unmap operations.
- The vnode locking mechanism in the execution of system calls belonging to the execve() family has been changed, which improved efficiency when executing execve() for the same file simultaneously (for example, during build operations with parallel compiler invocation);
- Security
- The bhyve hypervisor continues to enhance support for Live migration of guest environments from one host to another and the Save/Restore functionality, which allows freezing the guest system while saving its state to a file, and then resuming execution.
- Support for disk images in QCOW2 format has been added to bhyve through the use of the libvdsk library. Installation of a
version of bhyve, which has been adapted to use file operation handlers based on libvdsk, is required. During the reporting period, efforts were also made in libvdsk to simplify the integration of support for new formats, improve read and write performance, and add Copy-On-Write support. Remaining tasks include integrating libvdsk into the main composition of bhyve; - A traffic information collection system has been added to the ports
, allowing for the creation of traps for malicious network requests (IP addresses and domains from blacklists are checked) and sending information about detected activity to a centralized server for subsequent blocking or analysis of attack attempts; - A platform for detecting attacks, analyzing logs, and monitoring file integrity has been added to the ports (a fork of Ossec with integration support for );
- Network subsystem
- The ena driver has been updated with support for second-generation ENAv2 (Elastic Network Adapter) network adapters, used in the Elastic Compute Cloud (EC2) infrastructure to organize communication between EC2 nodes at speeds of up to 25 Gb/s. Support for NETMAP has been added to the ena driver.
- A new MMC/SD stack has been accepted into FreeBSD HEAD, based on the CAM framework, allowing the connection of devices with the SDIO (Secure Digital I/O) interface. For example, SDIO is used in WiFi and Bluetooth modules for many boards, such as the Raspberry Pi 3. The new stack also enables the use of the CAM interface to send SD commands from user-space applications, allowing for the creation of user-level device drivers. Work has begun on creating drivers for Broadcom wireless chips operating in FullMAC mode (where the chip performs its own operating system implementation for its 802.11 wireless stack).
- Work is underway to implement NFSv4.2 (RFC-7862) for FreeBSD. The new version of NFS adds support for the posix_fadvise, posix_fallocate functions, SEEKHOLE/SEEKDATA modes in lseek, and local file copying operations on the server (without transferring to the client).
Currently, FreeBSD has basic support for the LayoutError, IOAdvise, Allocate, and Copy operations. The Seek operation, necessary for using lseek(SEEKHOLE/SEEKDATA) with NFS, remains to be implemented. NFSv4.2 support is planned to be included in FreeBSD 13.
- Storage systems and file systems
- The project aimed at reworking the FUSE (File system in USErspace) subsystem driver, which allows for the creation of file system implementations in user space, is nearing completion. The originally supplied driver was outdated and contained many bugs. As part of the driver modernization project, support for FUSE protocol 7.23 has been implemented (the previously supported version was 7.8, released 11 years ago), code for checking access rights on the kernel side has been added (‘-o default_permissions’), calls to VOP_MKNOD, VOP_BMAP, and VOP_ADVLOCK have been added, the possibility of interrupting FUSE operations has been ensured, support for unnamed pipes and Unix sockets in fusefs has been added, kqueue support for /dev/fuse has been introduced, mounting parameter updates via ‘mount -u’ have been allowed, support for exporting fusefs via NFS has been implemented, RLIMIT_FSIZE accounting has been realized, flags FOPEN_KEEP_CACHE and FUSE_ASYNC_READ have been added, significant performance optimizations have been made, and caching organization has improved.
- Support for the BIO_DELETE operation has been added to the memory page eviction code for the swap pager, which allows the use of the TRIM command when deleting blocks from SSDs to increase their lifespan.
- Hardware support
- Work has continued on the implementation of support for the ARM64 SoC Broadcom BCM5871X with ARMv8 Cortex-A57 processors, aimed at use in routers, gateways, and network storage devices. During the reporting period, support for internal and external iProc PCIe buses has been improved, support for BNXT Ethernet has been added, and work is underway to utilize the built-in crypto engine to accelerate IPsec. Integration of the code into the HEAD branch is expected in the second half of the year.
- Work has begun on supporting the 64-bit SoC NXP LS1046A based on the ARMv8 Cortex-A72 processor, featuring an integrated network packet processing acceleration engine, 10 Gb Ethernet, PCIe 3.0, SATA 3.0, and USB 3.0. Basic platform support (multi-user SMP) and SATA 3.0 have already been implemented. Support for USB 3.0, SD/MMC, and I2C is under development. Plans include support for Ethernet, GPIO, and QSPI. Completion of the work and inclusion in the HEAD branch is expected in the fourth quarter of 2019.
- Drivers mlx5en and mlx5ib have been updated for Mellanox ConnectX-4 [Lx], ConnectX-5 [Ex], and ConnectX-6 [Dx] Ethernet and InfiniBand adapters. Support has been added for Mellanox Socket Direct adapters (ConnectX-6), allowing for throughput of up to 200 Gb/s on the PCIe Gen 3.0 bus. Support for the RShim driver has been added for multi-core 'BlueField' chips. The mstflint package with a set of diagnostic utilities for Mellanox adapters has been added to the ports.
- Applications and ports system
- Components of the graphical stack have been updated. The drm.ko (Direct Rendering Manager) driver has been ported from the Linux 5.0 kernel. This driver is considered experimental and has been added to the ports tree as graphics/drm-devel-kmod. Since the driver uses the updated Linux KPI framework for compatibility with the DR API of the Linux kernel, FreeBSD CURRENT is required for operation. The drm driver vboxvideo.ko for the VirtualBox virtual GPU has also been ported from Linux. The Mesa package has been updated to release 18.3.2 and transitioned to using LLVM from port devel/llvm80 instead of devel/llvm60.
- The FreeBSD ports tree has surpassed the milestone of 37,000 ports, with the number of open PRs remaining at 2,146. During the reporting period, 7,837 changes were made by 172 developers. Three new participants received committer rights. Among significant version updates in the ports are MySQL 5.7, Python 3.6, Ruby 2.5, Samba 4.8, Julia 1.0, Firefox 68.0, Chromium 75.0.3770.100. All Go language ports have been updated to use the 'USES=go' flag. The 'USES=cabal' flag has been added for the Cabal package manager used for Haskell code. Strict stack protection mode has been enabled. Python version 3.6 has been proposed as the default, replacing 2.7.
- Release of the utility prepared , offering functionality similar to /sbin/sysctl, which uses for output and provides an extended set of options. Nsysctl can be used for visual monitoring of sysctl values and presenting information about objects in a structured format. Output is possible in XML, JSON, and HTML formats;
Source: opennet.ru
