After almost two years of development release , implementation of the ZFS filesystem presented as a module for the Linux kernel. The module has been tested with Linux kernels from 2.6.32 to 5.1. Ready installation packages will be available shortly. for the main Linux distributions, including Debian, Ubuntu, Fedora, RHEL/CentOS. The ZFS on Linux module is already included in the distributions of Debian, Ubuntu, Gentoo, Sabayon Linux, and ALT Linux.
Within the framework of ZFS on Linux, the implementation of ZFS components has been prepared, relating both to the functionality of the filesystem and to the operation of the volume manager. In particular, the following components have been implemented: SPA (Storage Pool Allocator), DMU (Data Management Unit), ZVOL (ZFS Emulated Volume), and ZPL (ZFS POSIX Layer). Additionally, the project enables the use of ZFS as a backend for the Lustre cluster filesystem. The developments of the project are based on the original ZFS code imported from the OpenSolaris project, augmented by enhancements and fixes from the Illumos community. The project is being developed with the involvement of staff from the Lawrence Livermore National Laboratory under a contract with the U.S. Department of Energy.
The code is distributed under the CDDL open-source license, which is incompatible with GPLv2, preventing integration of ZFS on Linux into the mainline Linux kernel, as mixing code under GPLv2 and CDDL licenses is not permissible. To circumvent this licensing incompatibility, it was decided to distribute the product entirely under the CDDL license as a separately loadable module, which is provided separately from the kernel. The stability of the ZFS on Linux codebase is regarded as comparable to other filesystems for Linux.
Key Changes:
- Built-in support for encrypting stored data at the filesystem and partition level has been added. By default, the aes-256-ccm algorithm is used for encryption. The command 'zfs load-key' is suggested for loading encryption keys;
- The ability to transfer encrypted data while executing the 'zfs send' and 'zfs receive' commands has been implemented. When specifying the '-w' option, already encrypted data in the pool is transmitted to another pool as is, without intermediate decryption. In such copying, data remains protected by the sender's key, allowing this mode to be used for backups to untrusted systems (in case of recipient compromise, without the key, an attacker will not be able to access the data);
- Support has been added for removing primary storage devices from the storage pool, both when attached individually and as part of a mirror. Removal is performed using the command "zpool remove". During the removal process, data from the excluded device is copied to the remaining primary devices in the pool;
- The command "zpool checkpoint" has been added to save the current state of the pool with the ability to rollback subsequent changes to the saved point in time (a snapshot of the entire pool is created). This feature may be useful during potentially risky complex administrative tasks that typically lead to irreversible changes (for example, activating flags for new ZFS functionality or clearing data);
- The command "zpool trim" has been added to inform the devices used in the pool about sectors that are no longer in use. Using the TRIM operation allows for improved efficiency of SSDs and prevents performance degradation. To enable a continuous background process for issuing TRIM commands, a new property "autotrim" has been proposed;
- The command "zpool initialize" has been added for initializing all unallocated disk space, which ensures its immediate readiness for use without performance degradation on the first access (for example, when placing virtualized storage such as VMware VMDK);
- Support for accounting and quotas at the project level has been added, complementing previously available user and group-level quotas. Essentially, projects are separate object spaces associated with a distinct identifier (project ID). The binding is defined through the operation āchattr -pā or through attribute inheritance. To manage projects, the commands "zfs project" and "zfs projectspace" have been introduced, allowing for project creation and limits on disk space to be set;
- The ability to create Lua scripts for automating various tasks with ZFS has been added. Scripts are executed in special isolated environments using the command "zpool program";
- A new library has been implemented , providing a stable API for managing ZFS from applications written in Python. The library is a wrapper around libzfs_core and offers an identical set of functionalities while applying more Python-friendly types;
- Compatibility for the utilities arcstat, arcsummary, and dbufstat with Python 3 is ensured. The utilities arcstat.py, arc_summary.py, and dbufstat.py have been renamed to versions without the ā.pyā extension;
- Support for the Linux kernel Direct IO (O_DIRECT) interface has been added, allowing access to data without buffering and bypassing the cache;
- Performance optimizations have been introduced:
- The performance of the āscrubā and āresilverā commands has been accelerated by splitting them into two phases (a dedicated phase for scanning metadata and determining the location of data blocks on the disk, allowing subsequent verification using sequential data reading);
- Support for allocation classes has been added,
which allow the inclusion of relatively small SSDs into the pool and use them for storing only specific types of frequently accessed blocks, such as metadata, DDT data, and small file blocks; - The performance of administrative commands such as
āzfs listā and āzfs getā has been enhanced through the caching of the necessary metadata for their operation; - Support for parallelizing block allocation operations has been added by launching separate āallocatorā processes for each metaslab group. On typical systems, performance improvement is noted at 5-10%, but on larger systems (8 128 GB SSD, 24 core NUMA, 256 GB RAM), the block allocation operations can see a boost of up to 25%;
- The ability to delay the execution of the āresilverā command (rebuilding the distribution of data considering changes in storage configuration) has been added ā if a new operation is launched before the previous one has completed, the new handler will start executing only after the previous one finishes;
- Optimizations have been made to the ZIL (ZFS Intent Log) that allow for creating and processing blocks while other blocks are still being processed by the storage;
- The time for registering zvols in the system has been reduced. When the pool contains a large number of partitions, they now become available immediately after executing āzpool importā;
- Hardware acceleration support for SHA256 hash computation and AES-GCM encryption operations has been added, utilizing chips that support Intel QAT (Quick Assist Technology). Support for hardware acceleration tools for the Intel C62x chipset and Atom C3000 CPU has been added.
Source: opennet.ru
