FreeBSD Development Report Q2019 XNUMX

Published report on the development of the FreeBSD project from July to September 2019. Changes include:

• General and system questions
  • The Core team has generally approved the inclusion of BSD-licensed code with an additional patent agreement (BSD+Patent), but the decision to include each component under this license in the system must be approved separately;
  • The first meeting of the working group created to carry out the migration of source codes from the centralized source code management system Subversion to the decentralized system Git took place. The discussion on the appropriateness of migration is still ongoing, and many issues remain to be decided (for example, how to deal with contrib/, whether to regenerate hashes in the current git repository, and how best to implement commit testing);
  • From NetBSD ported the KCSAN (Kernel Concurrency Sanitizer) toolkit, which allows you to detect the occurrence of race conditions between kernel threads running on different CPUs;
  • Work is underway to use Clang's built-in assembler (IAS) instead of the assembler from GNU binutils;
  • The Linux environment emulation infrastructure (Linuxulator) has been adapted to work on the ARM64 architecture. The "renameat2" system call has been implemented. The strace utility has been improved to diagnose problems in Linux executables that run in Linuxulator. Fixed crash when linking executables with fresh glibc. Ports with Linux components for Linuxulator updated to CentOS 7.7;
  • As part of the Google Summer of Code program, students successfully completed six projects: an implementation of a unified (IPv4 / IPv6) ping utility was prepared, tools were developed for testing firewalls and detecting errors in the kernel (Kernel sanitizer), a mac_ipacl module was proposed, and code was written to compress virtual memory and work has been done to separate the process of building ports from local installation;
  • A project to fuzz test the FreeBSD kernel using the system syzkaller. During the reporting period, with the help of syzkaller, more than ten errors were identified and eliminated. To run syzkaller in virtual machines based on bhyve, a separate server is allocated, and using
    syzbot checks various FreeBSD subsystems in the Google infrastructure. Transfer of information about all crashes to the backtrace.io service has been organized to simplify their grouping and analysis;

  • Work is underway to update the implementation of zlib at the kernel level.
    The compression-related code has been moved from zlib 1.0.4, which was released over 20 years ago, to the current zlib 1.2.11 codebase. The compress, compress2, and uncompress functions have been added to the kernel to unify access to zlib. The code that provides the PPP protocol from the netgraph subsystem has been switched to use the system implementation of zlib, instead of the native edition of this library. The subsystems kern_ctf.c, opencryptodeflate, geom_uzip, subr_compressor,
    if_mxge, bxe updated and ng_deflate;

  • A new kernel interface is being developed sysctlinfo, which allows you to find elements in the sysctl parameter base, processed in the form of a MIB (Management Information Base), and transfer information about objects to user space.
• Security
  • Kernel module developed mac_ipacl, based on the TrustedBSD MAC Framework and implementing an access control system to network stack settings for jail environments. For example, using mac_ipacl, a host system administrator can prevent the root user in a jail environment from changing or setting IP addresses or subnet settings for certain network interfaces. Proposed Mandatory Access Control System Allows set lists of IP addresses and subnets allowed for Jail, prohibit the installation of certain IPs and subnets in Jail, or restrict parameter changes only for certain network interfaces;
  • Intel donated a software stack port to the project TPM 2.0 (Trusted Platform Module) for interacting with the secure computing chip, which is usually used for verified booting of firmware and OS bootloader. Stack components are provided in the form of securtity/tpm2-tss, security/tpm2-tools, and security/tpm2-abrmd ports. The tpm2-tss port includes libraries for using the TPM2 API, tpm2-tools provides command-line utilities for performing TPM operations, and tpm2-abrmd contains a background process with the implementation of the TPM Access Broker and Resource Manager components that multiplex requests from different TPM users to a single device . In addition to verified boot on FreeBSD, TPM can be used to enhance Strongswan IPsec, SSH, and TLS security by performing cryptographic operations on a separate chip;
  • The kernel for the amd64 architecture is adapted to be loaded using the W^X (write XOR execute) protection technique, which implies that memory pages cannot be simultaneously available for writing and execution (the kernel can now be loaded using executable memory pages for which writing is disabled). The new kernel protection method is included in the HEAD branch and will be part of the FreeBSD 13.0 and 12.2 releases;
  • For mmap and mprotect system calls implemented the PROT_MAX() macro, which allows to determine the set of access restriction flags (PROT_READ, PROT_WRITE, PROT_EXEC) that is valid for further changes. Using PROT_MAX(), a developer can prohibit the transfer of a memory area to the executable category or request memory that does not allow execution, but can later be transferred to executable. For example, a memory area can be open for writing only for the duration of dynamic linking or JIT code generation, but after the write is completed, it is limited only to reading and execution, and in the future, in the event of a compromise, the attacker will not be able to allow writing for this memory block. In addition to PROT_MAX(), sysctl vm.imply_prot_max is also implemented, which, when activated, determines the set of valid flags based on the initial parameters of the first mmap call;
  • To strengthen protection against exploitation of vulnerabilities, in addition to the address space randomization (ASLR) technique, a mechanism is proposed for randomizing offsets of pointers addressing the initial stack frame and structures placed on the stack with information about the environment, program launch parameters, and data for executable images in ELF format;
  • Work has been done to remove the unsafe gets function from libc (starting with the C11 standard, this function is excluded from the specification) and to correct the ports that still use this function. The change is planned to be offered in FreeBSD 13.0;
  • A pilot project was launched to create tools for orchestrating jail environments based on the framework can for creating and exporting images, implemented similarly to Docker, and a driver nomad, which provides an interface for dynamically launching applications in a jail environment. The proposed model allows separating the processes of creating jail environments and deploying applications in them. One of the goals of the project is to provide a means to manipulate jails like Docker-style containers;
• Storage and file systems
  • From NetBSD to the "makefs" utility carried over support for the FAT file system (msdosfs). Prepared changes allow you to create FS images with FAT without using the md driver and without root privileges;
  • The FUSE (File system in USErspace) subsystem driver has been reworked, which allows creating implementations of file systems in user space. The driver that was originally shipped contained many bugs and was based on FUSE 7.8 released 11 years ago. As part of the driver modernization project, the FUSE 7.23 protocol support was implemented, code was added to check permissions on the kernel side ("-o default_permissions"), VOP_MKNOD, VOP_BMAP and VOP_ADVLOCK calls were added, the ability to interrupt FUSE operations was added, support for unnamed pipes and unix sockets was added in fusefs, it became possible to use kqueue for /dev/fuse, updating mount parameters via "mount -u" was allowed, support for exporting fusefs via NFS was added, RLIMIT_FSIZE accounting was implemented, the FOPEN_KEEP_CACHE and FUSE_ASYNC_READ flags were added, significant performance optimizations were made and caching organization was improved. The new driver is included in the head and stable/12 branches (included with FreeBSD 12.1);
  • The NFSv4.2 implementation (RFC-7862) for FreeBSD is almost complete. During the reporting period, the focus was on testing. Compatibility tests with the Linux implementation have been completed, but testing of the pNFS server with NFSv4.2 is still ongoing. In general terms, the code is already considered ready for integration into FreeBSD's head/current branches. The new version of NFS adds support for the posix_fadvise, posix_fallocate functions, SEEKHOLE/SEEKDATA modes in lseek, local copying of file parts on the server (without transfer to the client);
• Hardware Support
  • Started a project to improve FreeBSD on laptops. The first device to be audited for hardware support in FreeBSD was the seventh generation Lenovo X1 Carbon laptop;
  • CheriBSD, a fork of FreeBSD for research processor architecture CHERI (Capability Hardware Enhanced RISC Instructions), updated to support the upcoming ARM Morello processor, which will support CHERI's memory access control system based on the Capsicum security model. Morello chip plan release in 2021. The CheriBSD developers also continue to monitor the development of the CHERI reference prototype based on the MIPS architecture;
  • Expanded support for RockChip RK3399 chips used in RockPro64 and NanoPC-T4 boards. The most significant improvement was eMMC support and the development of a new driver for the eMMC controller used in the board;
  • Work continued on implementing support for the ARM64 SoC Broadcom BCM5871X with ARMv8 Cortex-A57 processors aimed at use in routers, gateways, and NAS. During the reporting period
    expanded support for iProc PCIe and added the ability to use hardware cryptographic operations to speed up IPsec.
    Integration of code into the HEAD branch is expected in the fourth quarter;

  • Significant progress has been made in the development of the FreeBSD port for the powerpc64 platform. The focus is on providing quality performance on systems with IBM POWER8 and POWER9 processors, but is optionally supported on older Apple Power Macs, x500 and Amiga A1222. The powerpc*/12 branch continues to ship with gcc 4.2.1, and the powerpc*/13 branch will soon be migrated to llvm90. Out of 33306 ports, 30514 are successfully assembled;
  • Porting of FreeBSD to NXP LS64A 1046-bit SoC based on ARMv8 Cortex-A72 processor with integrated network packet processing acceleration engine, 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 continued. During the reporting period, support for USB 3.0, SD / MMC, I2C, DPAA network interface and GPIO was implemented. We plan to support QSPI and optimize network interface performance. Completion and inclusion in the HEAD branch is expected in Q4 2019;
  • The ena driver has been updated to support the second generation of ENAv2 (Elastic Network Adapter) network adapters used in the Elastic Compute Cloud (EC2) infrastructure to organize communication between EC2 nodes at speeds up to 25 Gb/s. Added and tested NETMAP support to the ena driver, and adjusted the memory layout to enable LLQ mode in Amazon EC2 A1 environments;
• Applications and ports system
  • Updated graphics stack components and ports related to xorg. Ports using USE_XORG and XORG_CAT have been migrated to the USES framework instead of calling bsd.xorg.mk via bsd.port.mk. Such ports now include the "USES=xorg" flag in their makefiles. The XORG_CAT functionality has been extracted from bsd.xorg.mk and is now enabled with the "USES=xorg-cat" flag. Added tools for directly generating xorg ports from a git repository
    freedesktop.org, which, for example, allows you to generate ports for versions that have not yet been released. In the future, it is planned to prepare a toolkit to use the meson build system instead of autotools to build xorg ports.

    Work has been done to clean up old xorg ports tied to components that are no longer supported, such as removing the x11/libXp port and deprecating the x11/Xxf86misc, x11-fonts/libXfontcache and graphics/libGLw ports;

  • Work has been done to improve support for Java 11 and newer releases in FreeBSD, and to port some changes to the Java 8 branch. Since FreeBSD has implemented support for new Java 11 features such as Java Flight Recorder, HotSpot Serviceability Agent, HotSpot Debugger , DTrace, Javac Server, Java Sound and SCTP, work has shifted to ensuring that all compatibility tests pass. The number of failures when passing tests has been reduced from 50 to 2;
  • The KDE Plasma desktop, KDE Frameworks, KDE Applications and Qt are kept up to date and updated to the latest releases;
  • Ports with Xfce desktop updated to release 4.14;
  • The FreeBSD ports tree has passed the milestone of 38000 ports, the number of open PRs is slightly over 2000, of which 400 PRs are still pending. During the reporting period, 7340 changes were made by 169 developers. Two new contributors (Santhosh Raju and Dmitri Goutnik) got committer rights. A new release of the pkg 1.12 package manager has been published with support for overlays in the ports tree and cleaning up bsd.sites.mk. Significant version updates in ports include: Lazarus 2.0.4, LLVM 9.0, Perl5.30, PostgreSQL 11, Ruby 2.6, Firefox 69.0.1, Firefox-esr 68.1.0, Chromium 76.0;
  • The development of the project continues ClonOS, developing a specialized distribution kit for deploying a virtual server infrastructure. In terms of tasks to be solved, ClonOS resembles such systems as Proxmox, Triton (Joyent), OpenStack, OpenNebula and Amazon AWS, the main difference from which is the use of FreeBSD and the ability to manage, deploy and manage FreeBSD Jail containers and virtual environments based on Bhyve and Xen hypervisors . Of the recent changes, there is support
    cloud-init for Linux/BSD VM and cloudbase-init for Windows VM, start of transition to native images, use of Jenkins CI for build testing and new pkg repository for installation
    ClonOS from packages.

Source: opennet.ru