After a year-long lull in development on the new branch of the fault-tolerant distributed file system and as the second candidate for releases. Recently, with the change of ownership of the company developing LizardFS, new management has been adopted, and the developers have changed. Over the past two years, the project has distanced itself from the community and hasn't given it due attention, but the new team intends to revive the previous relationships with the community and establish close collaboration with it. The project's code is written in C and C++ and is under the GPLv3 license.
LizardFS is a distributed clustered file system that allows data to be spread across different servers while presenting access to them as a single large partition, which can be used similarly to traditional disk partitions. The mounted partition with LizardFS supports POSIX file attributes, ACL, locks, sockets, pipes, device files, symbolic and hard links. The system has no single point of failure, and all components are backed up. It supports parallel data operations (multiple clients can access files simultaneously).
To ensure fault tolerance, data is divided into replicas, which are distributed across different nodes with redundancy (multiple copies are placed on different nodes) — in case of node or storage failure, the system continues to operate without data loss and automatically redistributes data based on the remaining nodes. For storage expansion, it's enough to connect new nodes without shutting down for maintenance (the system itself replicates part of the data to the new servers and balances the storage regarding the new servers). Similarly, for reducing the cluster size, outdated equipment can simply be disconnected from the system.
Data and metadata are stored separately. It is recommended to install two metadata servers operating in a master-slave mode and at least two data storage servers (chunkservers). Additionally, log servers can be used for backing up metadata, storing information about metadata changes and enabling recovery in case all existing metadata servers fail. Each file is divided into chunks of up to 64 MB. The chunks are distributed across storage servers according to the chosen replication mode: standard (explicitly defining the number of copies to place on different nodes, including binding to specific directories — for important data, the number of copies can be increased, while for non-essential data it can be reduced), XOR (RAID5), and EC (RAID6).
The storage can scale to petabyte sizes. Use cases include archiving, storing virtual machine images, multimedia data, backups, serving as a DRC (Disaster Recovery Center), and as storage in high-performance computing clusters. LizardFS provides very high read speeds for files of any size and shows good performance when writing large and medium-sized files entirely, especially when there is no constant modification, intensive operations with open files, and one-off operations involving many small files.
Notable features of the file system include support for snapshots, which reflect the state of files at a specific time, and a built-in trash can (files are not deleted immediately and remain available for recovery for some time). Access to the section can be restricted by IP address or password (similar to NFS). There are mechanisms for quotas and quality of service management, allowing limitations on size and bandwidth for certain categories of users. It is possible to create geographically distributed storages, with segments placed in different data centers.
The LizardFS project was founded in 2013 as a fork of , and primarily differs by having a replication mode based on Reed-Solomon error correction codes (similar to raidzN), enhanced ACL support, a client for the Windows platform, additional optimizations (for example, when the client and storage server are combined, blocks are delivered from the current node where possible, and metadata is cached in memory), a more flexible configuration system, support for proactive data reading, directory quotas, and internal overhauls.
The release of LizardFS 3.13.0 is planned for the end of December. The main innovation of LizardFS 3.13 is the use of a consensus algorithm for ensuring fault tolerance (switching master servers in case of failure). (a custom implementation of uRaft is used, which was previously used in commercial products). The use of uRaft simplifies configuration and reduces recovery delays after a failure but requires at least three operational nodes, one of which is used for quorum.
Other changes include: a new client based on the FUSE3 subsystem, solutions for error correction issues, and the nfs-ganesha plugin has been rewritten in C. The update 3.13.0-rc2 fixes several critical bugs that rendered previous test releases of the 3.13 branch nearly unusable (fixes for the 3.12 branch have not yet been published, and upgrading from 3.12 to 3.13 still results in complete data loss).
In 2020, the focus will be on the development
, a completely rewritten core of LizardFS, which, according to the developers, will provide a threefold increase in performance compared to the 3.12 branch. In Agama, there will be a transition to an event-driven architecture, asynchronous I/O based on , and predominantly operation in user space (to reduce dependency on kernel caching mechanisms). Additionally, a new debugging subsystem and a network activity analyzer with performance auto-tuning support will be offered.
The LizardFS client will receive full support for versioning write operations, which will enhance fault recovery reliability, address issues that arise when different clients access the same data, and significantly improve performance. The client will be transitioned to its own network subsystem operating in user space. The first working prototype of LizardFS based on Agama is planned for the second quarter of 2020. At the same time, integration tools for LizardFS with the Kubernetes platform are promised to be implemented.
Source: opennet.ru
