Report on the development of FreeBSD for the third quarter of 2019

Published Report on the development of the FreeBSD project from July to September 2019. Notable changes include:

• General and System Issues
  • The core team generally approved the possibility of including code into the system that is supplied under the BSD license with an additional patent usage agreement (BSD+Patent), but each component under this license must be individually approved for inclusion in the system;
  • The first meeting of the working group created for the migration of source texts from the centralized Subversion source code management system to a decentralized Git system took place. The discussion on the feasibility of migration is still ongoing, and many questions still need to be resolved (for example, how to handle contrib/, whether to regenerate hashes in the current git repository, and how to best implement commit testing);
  • From NetBSD ported the KCSAN (Kernel Concurrency Sanitizer) tool, which helps identify the occurrences of race conditions between kernel threads running on different CPUs;
  • Work is underway to utilize the integrated assembler (IAS) in Clang instead of the assembler from GNU binutils;
  • The Linux environment emulation infrastructure (Linuxulator) has been adapted to work on the ARM64 architecture. The 'renameat2' system call has been implemented. The strace utility has been enhanced for diagnosing issues with Linux executables running in the Linuxulator. The problem with crashes when linking executables with the latest glibc has been resolved. Ports with Linux components for the Linuxulator have been updated to CentOS 7.7;
  • As part of the Google Summer of Code program, students successfully completed six projects: a unified (IPv4/IPv6) ping utility was prepared, tools for testing firewalls and identifying errors in the kernel (Kernel sanitizer) were developed, a mac_ipacl module was proposed, code for virtual memory compression was written, and work was done to separate the port building process from local installation;
  • The project for fuzz testing the FreeBSD kernel continues using the system The module adds a new programming interface to the kernel's. During the reporting period, syzkaller identified and fixed more than ten errors. A separate server has been allocated for running syzkaller on virtual machines based on bhyve, and using
    syzbot has established checks for various FreeBSD subsystems within Google infrastructure. Information about all crashes has been organized for transmission to the backtrace.io service for easier grouping and analysis;
  • Work is underway to update the zlib implementation at the kernel level.
    Compression-related code has been transitioned from zlib 1.0.4, released over 20 years ago, to the current zlib 1.2.11 codebase. Functions compress, compress2, and uncompress have been added to the kernel to standardize access to zlib. The code ensuring PPP protocol operation from the netgraph subsystem has been transitioned to utilize the system's zlib implementation instead of its own version of this library. The new zlib has also been adopted by subsystems kern_ctf.c, opencryptodeflate, geom_uzip, subr_compressor,
    if_mxge, bxe updated, and ng_deflate;
  • A new kernel interface sysctlinfo, allowing for the finding of elements in the sysctl parameters database, processed in the form of MIB (Management Information Base), and transmitting information about objects to the user space.
• Security
  • A kernel module has been developed mac_ipacl, based on the TrustedBSD MAC Framework and implementing an access control system for network stack settings in jail environments. For instance, using mac_ipacl, the host system administrator can prevent the root user in the jail environment from changing or setting IP addresses or subnet parameters for specific network interfaces. The proposed mandatory access control system , rather than taking focus. allows setting lists of IP addresses and subnets permissible for Jail, forbidding the assignment of specific IP addresses and subnets in Jail, or restricting parameter changes to only certain network interfaces;
  • Intel has handed over the porting project of the software stack TPM 2.0 (Trusted Platform Module) for interacting with a chip for secure computations, which is typically used for verified boot of firmware and OS bootloaders. The stack components are represented in the form of ports security/tpm2-tss, security/tpm2-tools, and security/tpm2-abrmd. The tpm2-tss port includes libraries for using the TPM2 API, tpm2-tools provides command-line utilities for performing operations with TPM, while tpm2-abrmd contains a background process implementing the components of the TPM Access Broker and Resource Manager, multiplexing requests from different TPM users to a single device. In addition to verified boot, in FreeBSD, TPM can be used to enhance the security of Strongswan IPsec, SSH, and TLS by performing cryptographic operations on a separate chip.
  • The kernel for the amd64 architecture has been adapted for booting using the W^X (write XOR execute) protection technique, which stipulates that memory pages cannot be simultaneously accessible for writing and execution (the kernel can now boot using executable memory pages that are write-prohibited). The new kernel protection method has been included in the HEAD branch and will be part of FreeBSD releases 13.0 and 12.2.
  • For the system calls mmap and mprotect is implemented the PROT_MAX() macro, allowing for the determination of an acceptable set of access control flags (PROT_READ, PROT_WRITE, PROT_EXEC) for further modification. Using PROT_MAX(), a developer can prevent a memory region from being categorized as executable, or request memory that does not allow execution but may later be converted to executable. For example, a memory area may be opened for writing only during dynamic linking or JIT code generation, but after writing is complete, it is restricted to reading and execution, so in case of compromise, an attacker cannot grant write permission for that block of memory. In addition to PROT_MAX(), the sysctl vm.imply_prot_max is also implemented, which determines the allowable flags based on the original parameters of the first mmap call when activated.
  • To enhance protection against vulnerability exploitation, in addition to address space randomization (ASLR) techniques, a mechanism for randomization of pointer offsets that address the initial stack frame and place structures with information about the environment, program launch parameters, and data for executable images in ELF format has been proposed;
  • Work has been carried out to remove the unsafe function gets from libc (this function has been removed from the C11 standard) and to adjust ports still using this function. The change is planned to be proposed in FreeBSD 13.0;
  • An experimental project has been launched to create orchestration tools for jail environments based on the framework pot for creating and exporting images, implemented similarly to Docker, and the driver nomad, providing an interface for dynamically launching applications in a jail environment. The proposed model allows for separating the processes of creating jail environments and deploying applications within them. One of the project's goals is to provide a means for manipulating jails as containers in a Docker-like style;
• Storage systems and file systems
  • From NetBSD to the 'makefs' utility migrated support for the FAT file system (msdosfs). The prepared changes allow creating FAT filesystem images without using the md driver and without root privileges;
  • The driver for the FUSE (File system in USErspace) subsystem has been redesigned, allowing the creation of file system implementations in user space. The originally supplied driver contained many bugs and was based on FUSE version 7.8, released 11 years ago. As part of the driver modernization project, support for the FUSE 7.23 protocol has been implemented, code for checking access rights on the kernel side ('-o default_permissions') has been added, VOP_MKNOD, VOP_BMAP, and VOP_ADVLOCK calls have been included, the ability to interrupt FUSE operations has been provided, support for unnamed pipes and Unix sockets in fusefs has been added, the use of kqueue for /dev/fuse has been enabled, mounting parameters can be updated through 'mount -u', support for exporting fusefs via NFS has been implemented, RLIMIT_FSIZE accounting has been introduced, FOPEN_KEEP_CACHE and FUSE_ASYNC_READ flags have been added, significant performance optimizations have been made, and caching organization has been improved. The new driver is included in the head and stable/12 branches (part of FreeBSD 12.1);
  • The work on implementing NFSv4.2 (RFC-7862) for FreeBSD is nearing completion. During the reporting period, the focus was on testing. Compatibility testing with the Linux implementation has been completed, but testing of the pNFS server with NFSv4.2 is still ongoing. Generally, the code is already considered ready for integration into the FreeBSD head/current branches. The new version of NFS has added support for the posix_fadvise, posix_fallocate functions, SEEKHOLE/SEEKDATA modes in lseek, and local copying of file parts on the server (without transferring to the client);
• Hardware support
  • A project has been initiated to improve FreeBSD performance on laptops. The first device for which hardware support was audited in FreeBSD is the Lenovo X1 Carbon seventh generation;
  • CheriBSD, a branch of FreeBSD for the research processor architecture CHERI (Capability Hardware Enhanced RISC Instructions), updated to support the upcoming ARM Morello processor, which will support the CHERI memory access control system based on the Capsicum project protection model. The Morello chip plan is set to be released in 2021. CheriBSD developers are also continuing to track the development of the CHERI reference prototype based on the MIPS architecture;
  • Support for RockChip RK3399 chips used in RockPro64 and NanoPC-T4 boards has been expanded. The most significant improvement has been the support for eMMC and the development of a new driver for the eMMC controller used on the board;
  • Work has continued on implementing support for the ARM64 SoC Broadcom BCM5871X with ARMv8 Cortex-A57 processors, aimed at use in routers, gateways, and network storage. During the reporting period
    support for iProc PCIe has been expanded and the ability to use hardware cryptographic operations to accelerate IPsec has been added.
    Code integration into the HEAD branch is expected in the fourth quarter;
  • Significant progress is being noted in the development of the FreeBSD port for the powerpc64 platform. The main focus is on ensuring good performance on systems with IBM POWER8 and POWER9 processors, but optional support for older Apple Power Macs, x500, and Amiga A1222 is also maintained. The powerpc*/12 branch continues to be provided with gcc 4.2.1, while the powerpc*/13 branch will soon be transitioned to llvm90. Out of 33306 ports, 30514 build successfully;
  • The porting of FreeBSD for the 64-bit NXP LS1046A SoC based on the ARMv8 Cortex-A72 processor, featuring an integrated network packet processing acceleration engine, 10 Gb Ethernet, PCIe 3.0, SATA 3.0, and USB 3.0 has continued. During the reporting period, support for USB 3.0, SD/MMC, I2C, DPAA network interface, and GPIO was implemented. Plans include support for QSPI and performance optimization of the network interface. Completion and inclusion in the HEAD branch are expected in Q4 2019.
  • The ena driver has been updated to support the second generation of ENAv2 (Elastic Network Adapter) network adapters used in the Elastic Compute Cloud (EC2) infrastructure, facilitating communication between EC2 nodes at speeds of up to 25 Gb/s. NETMAP support has been added and tested in the ena driver, alongside memory layout adaptation for enabling LLQ mode in Amazon EC2 A1 environments.
• Applications and ports system
  • The components of the graphics stack and ports related to xorg have been updated. Ports using USE_XORG and XORG_CAT have transitioned to the USES framework instead of calling bsd.xorg.mk through bsd.port.mk. These ports now include the flag 'USES=xorg' in their makefiles. The XORG_CAT functionality has been separated from bsd.xorg.mk and is now activated by the flag 'USES=xorg-cat'. Tools have been added for directly building xorg ports from the git repository.
    freedesktop.org, which allows for the creation of ports for unreleased versions, for example. In the future, there are plans to prepare tools for using the meson build system instead of autotools for building xorg ports.

    Work has been completed to clean up old xorg ports tied to unsupported components, such as the removal of the port x11/libXp, while the ports x11/Xxf86misc, x11-fonts/libXfontcache, and graphics/libGLw have been marked as obsolete.

  • Efforts have been made to improve Java 11 and newer release support in FreeBSD, as well as porting some changes to the Java 8 branch. After implementing support for new features in Java 11 such as Java Flight Recorder, HotSpot Serviceability Agent, HotSpot Debugger, DTrace, Javac Server, Java Sound, and SCTP, the focus shifted to ensuring compatibility test pass rates. The number of test failures has been reduced from 50 to 2.
  • KDE Plasma, KDE Frameworks, KDE Applications, and Qt have been kept up to date and upgraded to the latest releases.
  • Xfce desktop ports have been updated to the release. 4.14;
  • The FreeBSD ports tree has surpassed 38,000 ports, with the number of open PRs slightly exceeding 2,000, of which 400 PRs are still unaddressed. During the reporting period, 7,340 changes were made by 169 developers. Two new members, Santhosh Raju and Dmitri Goutnik, have been granted committer rights. A new version of the pkg package manager 1.12 has been released, featuring overlay support in the ports tree and cleaning up bsd.sites.mk. Notable version updates in the ports include: Lazarus 2.0.4, LLVM 9.0, Perl 5.30, PostgreSQL 11, Ruby 2.6, Firefox 69.0.1, Firefox-esr 68.1.0, Chromium 76.0;
  • Project development continues ClonOS, developing is a specialized distribution for deploying virtual server infrastructure. In terms of solved tasks, ClonOS resembles systems like Proxmox, Triton (Joyent), OpenStack, OpenNebula, and Amazon AWS, with the main difference being the use of FreeBSD and the ability to manage, deploy, and control FreeBSD Jail containers and virtual environments based on Bhyve and Xen hypervisors. Recent changes include support for
    cloud-init for Linux/BSD VMs and cloudbase-init for Windows VMs, the start of a transition to using custom images, the implementation of Jenkins CI for testing builds, and a new pkg repository for installation
    of ClonOS from packages.

Source: opennet.ru