LF 1.1.0, a decentralized, replicated key/value data store, is now available. The project is being developed by ZeroTier, which is developing a virtual Ethernet switch that allows you to combine hosts and virtual machines located at different providers in one virtual local network, the participants of which exchange data in P2P mode. The project code is written in C language. The new release is notable for its transition to the free MPL 2.0 license (Mozilla Public License).
Previously, the LF code was available under a BSL (Business Source License), which is not free due to discrimination against certain categories of users. The BSL license was proposed by the co-founders of MySQL as an alternative to the Open Core model. The essence of BSL is that the code of advanced functionality is initially available for modification, but for some time can be used free of charge only if additional conditions are met, which require the purchase of a commercial license to circumvent.
LF is a completely decentralized system and allows you to deploy a single data store in a key-value format on top of an arbitrary number of nodes. Data is kept synchronized across all nodes, and all changes are fully replicated across all nodes. All nodes in LF are equal to each other. The absence of separate nodes coordinating the operation of the storage allows you to get rid of a single point of failure, and the presence of a complete copy of the data on each node eliminates the loss of information when individual nodes fail or are disconnected.
To connect a new node to the network, you do not need to obtain separate permissions - anyone can start their own node. LF's data model is built around a directed acyclic graph (DAG), which simplifies synchronization and allows for a variety of conflict resolution and security strategies. Unlike distributed hash table (DHT) systems, the IF architecture is initially designed for use in unreliable networks where the constant availability of nodes is not guaranteed. Among the areas of application of LF, the creation of the most survivable storage systems is mentioned, in which relatively small volumes of critical data are stored that rarely change. For example, LF is suitable for key stores, certificates, identity parameters, configuration files, hashes and domain names.
To protect against overload and abuse, a limit on the intensity of write operations to the shared storage is applied, implemented on the basis of proof of work - in order to be able to save data, a participant in the storage network must complete a certain task, which is easily verified, but requires large computational resources (similar to organizing the expansion of systems based on blockchain and CRDT). The calculated values are also used as a sign when resolving conflicts.
As an alternative, a certificate authority can be launched on the network to issue cryptographic certificates to participants, giving the right to add records without confirmation of work and giving priority in resolving conflicts. By default, the storage is available without restrictions for connecting any participants, but optionally, based on a certificate system, fenced-off private storages can be created, in which only nodes certified by the owner of the network can become participants.
Main features of LF:
- Easy to deploy your own storage and connect to existing public storage networks.
- There is no single point of failure and the ability to involve everyone in maintaining the storage.
- High speed access to all data and the ability to access data remaining on its node, even after a disruption in network connectivity.
- A universal security model that allows you to combine various conflict resolution mechanisms (local heuristics, weight based on completed work, taking into account the trust level of other nodes, certificates).
- A flexible API for querying data that allows multiple nested keys or value ranges to be specified. Ability to bind multiple values to one key.
- All data is stored in encrypted form, including keys, and verified. The system can be used to organize the storage of confidential data on untrustworthy nodes. Records for which the keys are not known cannot be determined by brute force (without knowing the key, it is impossible to obtain the data associated with it).
Limitations include a focus on storing small, rarely changing data, the absence of locks and guaranteed data consistency, high requirements for CPU, memory, disk space and bandwidth, and a constant increase in storage size over time.
Source: opennet.ru