For Linux kernel 6.2, improvements have been proposed for Btrfs to address the 'write hole' issue in RAID 5/6 implementation. The essence of the problem is that if a crash occurs during writing, it initially becomes unclear which block was correctly written on which RAID device and which block's write was incomplete. In attempts to recover RAID in such situations, destruction of blocks corresponding to unwritten blocks may occur, as the state of the RAID blocks becomes unsynchronized. This issue arises in any RAID1/5/6 arrays where no special measures have been taken to combat this effect.
In RAID implementations, such as RAID1 in btrfs, this problem is solved using checksums in both copies; in case of a mismatch, the data is simply restored from the second copy. This approach also works if some device starts to return incorrect data instead of completely failing.
However, in the case of RAID5/6, the filesystem does not store checksums for parity blocks: under normal circumstances, the correctness of the blocks is checked by the fact that they all have checksums, and the parity block can be reconstructed from the data. However, in the case of partial writing, this approach may not work in certain situations. In this case, when recovering the array, it is possible that the blocks affected by the incomplete write will be restored incorrectly.
In the case of btrfs, this problem is most relevant when the written data size is smaller than the stripe size. In this scenario, the file system must perform a read-modify-write (RMW) operation. If it encounters blocks with incomplete writes, the RMW operation can cause corruption that may go undetected despite checksums. Developers have made changes so that the RMW operation checks the checksums of blocks before carrying out this operation, and during data recovery, it performs checksum verification after the write. Unfortunately, in situations involving incomplete stripe writes (RMW), this leads to additional overhead for computing checksums, though it significantly increases reliability. For RAID6, this logic is not yet ready, but for such a failure in RAID6, a write must fail on two devices simultaneously, which is less likely.
Additionally, it's worth noting the recommendations for using RAID5/6 from the developers, which suggest that in Btrfs, the storage profile for metadata and data can differ. You can use a RAID1 profile (mirror) or even RAID1C3 (3 copies) for metadata, while using RAID5 or RAID6 for data. This ensures reliable protection of metadata and eliminates the 'write hole' on one hand, while also providing a more efficient use of space characteristic of RAID5/6 on the other. This helps avoid corruption in metadata, whereas data corruption can be corrected.
It is also noteworthy that for SSDs in Btrfs, starting from kernel 6.2, asynchronous execution of the 'discard' operation (marking released blocks that no longer need to be physically stored) will be activated by default. The advantage of this mode is high performance due to the effective batching of 'discard' operations in the queue, followed by processing the queue by a background handler, meaning that normal filesystem operations do not slow down, unlike with synchronous 'discard' as blocks are released, allowing SSDs to make more optimized decisions. On the other hand, there is no longer a need to use utilities like fstrim, as all available blocks will be cleared in the filesystem without the need for additional scanning and without slowing down operations.
Source: opennet.ru
