After nearly two years of development Release , an implementation of the ZFS file system packaged as a module for the Linux kernel. The module has been tested with Linux kernels from 2.6.32 to 5.1. Ready installation packages coming soon for major Linux distributions including Debian, Ubuntu, Fedora, RHEL/CentOS. The ZFS on Linux module is already included in Debian, Ubuntu, Gentoo, Sabayon Linux and ALT Linux distributions.
As part of ZFS on Linux, an implementation of ZFS components related to both the operation of the file system and the functioning of the volume manager has been prepared. In particular, the following components are implemented: SPA (Storage Pool Allocator), DMU (Data Management Unit), ZVOL (ZFS Emulated Volume) and ZPL (ZFS POSIX Layer). Additionally, the project provides the ability to use ZFS as a backend for the Luster cluster file system. The project's work is based on original ZFS code imported from the OpenSolaris project and enhanced with improvements and fixes from the Illumos community. The project is being developed with the participation of employees of the Livermore National Laboratory under a contract with the US Department of Energy.
The code is distributed under a free CDDL license, which is incompatible with GPLv2, which does not allow ZFS on Linux to be integrated into the main branch of the Linux kernel, since mixing code under GPLv2 and CDDL licenses is not allowed. To circumvent this license incompatibility, it was decided to distribute the entire product under the CDDL license as a separately loadable module, which is supplied separately from the core. The stability of the ZFS on Linux codebase is rated as comparable to other file systems for Linux.
Major changes:
- Added built-in support for encryption of stored data at the level of the file system and partitions. The default encryption algorithm is aes-256-ccm. The "zfs load-key" command has been proposed to load encryption keys;
- Implemented the ability to transfer encrypted data when executing the 'zfs send' and 'zfs receive' commands. When specifying the “-w” option, data already encrypted in the pool is transferred to another pool as is, without intermediate decryption. With such copying, the data remains protected by the sender's key, which allows using this mode for backup to untrustworthy systems (if the recipient is compromised, the attacker will not be able to access the data without the key);
- Added support for removing primary drives from a storage pool, connected both individually and as part of a mirror. Removal is carried out with the "zpool remove" command. The deletion process copies the data from the excluded drive to the remaining primary drives in the pool;
- Added the "zpool checkpoint" command to save the current state of the pool with the ability to roll back further changes to the saved point in time (a snapshot of the entire pool is created). This feature can be useful in the process of performing potentially dangerous complex administrative work, which under normal conditions leads to irreversible changes (for example, activating flags for new ZFS functionality or clearing data);
- The "zpool trim" command has been added to inform the drives used in the pool about sectors that are no longer in use. The use of the TRIM operation makes it possible to increase the efficiency of SSDs and prevent degradation of their performance. A new "autotrim" property has been proposed to enable the continuous background process of transmitting TRIM commands;
- Added "zpool initialize" command to initialize all unallocated disk space, which allows it to be instantly ready for use, without performance degradation on first access (for example, when hosting virtualized storages such as VMware VMDK);
- Added support for accounting and project-level quotas, in addition to previously available user and group-level quotas. In essence, projects are a separate space of objects associated with a separate identifier (project ID). The binding is defined via the 'chattr -p' operation or via attribute inheritance. For project management, the “zfs project” and “zfs projectspace” commands are presented, which allow you to manage the creation of projects and set disk space limits for them;
- Added the ability to create Lua scripts to automate various work with ZFS. Scripts are run in special isolated environments using the "zpool program" command;
- New library implemented , which provides a stable API for administering ZFS from Python applications. The library is a wrapper around libzfs_core and provides an identical set of functions, but uses closer Python types;
- The arcstat, arcsummary, and dbufstat utilities have been made compatible with Python 3. The arcstat.py, arc_summary.py, and dbufstat.py utilities have been renamed to versions without the ".py" extension;
- Added support for the Linux Direct IO (O_DIRECT) kernel interface, which allows accessing data without buffering and bypassing the cache;
- Performance optimizations introduced:
- The work of the “scrub” and “resilver” commands has been accelerated due to splitting into two phases (a separate phase has been allocated for scanning metadata and determining the location of blocks with data on the disk, which allows further verification using sequential data reading);
- Added support for Allocation classes,
allowing relatively small SSDs to be pooled and used to store only certain types of commonly used blocks, such as metadata, DDT data, and small file blocks; - Improved performance of admin commands such as
"zfs list" and "zfs get", by caching the metadata necessary for their operation; - Added support for parallelization of block allocation operations by running separate allocator processes for each metaslab group. On conventional systems, there is a performance increase of 5-10%, but on large ones (8 128 GB SSD, 24 core NUMA, 256 GB of RAM), the increase in block allocation operations can reach 25%;
- Added the possibility of delayed execution of the "resilver" command (rebuilding data distribution taking into account changes in the configuration of drives) - if the previous one has not yet completed when starting a new operation, then the new handler will start executing only after the previous one has finished;
- Optimizations have been added to the ZIL (ZFS Intent Log) to allow creating and processing blocks in the presence of blocks that are still being processed by the storage;
- Reduced registration time for partitions (zvol) in the system. When a pool contains a large number of partitions, they are now available immediately after doing "zpool import";
- Added support for hardware accelerated calculation of SHA256 hashes and AES-GSM encryption operations using Intel QAT (Quick Assist Technology) chips. Added support for hardware acceleration of the Intel C62x chipset and CPU Atom C3000.
Source: opennet.ru