The release of the distributed DBMS rqlite 7.0 has taken place, which uses SQLite as a storage engine and allows you to organize the operation of a cluster from storages synchronized with each other. Of the features of rqlite, it is noted that it is easy to install, deploy and maintain a distributed fault-tolerant storage, somewhat similar to etcd and Consul, but using a relational data model instead of the key / value format. The project code is written in Go and distributed under the MIT license.
The Raft consensus algorithm is used to keep all nodes in a synchronized state. Rqlite uses the original SQLite library and the go-sqlite3 driver, on top of which a layer is run that processes client requests, performs replication to other nodes and monitors the consensus on the choice of the leader node.
Changes to the database can only be made by the node that is selected as the leader, but write connections can be directed to other cluster nodes, which will return the address of the leader to repeat the request (they promise to add automatic forwarding of the call to the leader in the next version). The main emphasis is on fault tolerance, so the DBMS scales only on reads, and writes are the bottleneck. It is possible to run an rqlite cluster from a single node and such a solution can be used to provide access to SQLite over HTTP without providing fault tolerance.
The SQLite data on each node is not stored in a file, but in memory. At the layer level with the implementation of the Raft protocol, a log is kept of all SQLite commands that lead to a database change. This log is used for replication (replay level replication on other nodes), when starting a new node, or for recovering from a loss of connectivity. To reduce the size of the log, automatic packaging is used, which starts after a specified number of changes and leads to the commit of a snapshot, relative to which a new log is started (the state of the database in memory is identical to the snapshot + the accumulated change log).
rqlite features:
- Ease of deployment of a cluster, without the need for a separate installation of SQLite.
- The ability to quickly get replicated SQL storage.
- Ready for use in working projects (production-grade).
- The presence of an HTTP(S) API that allows you to update data in batch mode and determine the leading node of the cluster. A command line interface and client libraries for various programming languages are also provided.
- The presence of a service for determining other nodes, allowing you to create clusters dynamically.
- Support for encryption of data exchange between nodes.
- Ability to adjust the level of checking the relevance and consistency of data when reading.
- The optional ability to connect nodes in read-only mode that do not take part in the determination of consensus and are used to increase the scalability of the cluster for read operations.
- Support for a native form of transactions based on combining commands in a single request (transactions based on BEGIN, COMMIT, ROLLBACK, SAVEPOINT and RELEASE are not supported).
- Support for creating hot backups.
In the new release:
- Added support for automatic rqlite clustering using a new node discovery service that can run on Consul and etcd distributed storage. The service allows rqlite nodes to automatically find each other - the administrator just needs to run several nodes on different servers by specifying the common address of the Consul or etcd cluster (for example, “example.com:8500”), and the nodes will automatically be clustered. The leading node periodically updates information about its address in the Consul or etcd storage, which makes it possible to change the leader in the future without the need to reconfigure the remaining nodes, as well as add new nodes even after the leader has changed. Support for the old Discovery mode service powered by AWS Lambda has been discontinued.
- The CLI interface allows specifying several hosts at once - if the first node is unavailable, the following hosts will be contacted.
- Redesigned code for parsing rqlited command line arguments.
- The deprecated protobuf package has been deprecated.
- The BoltDB storage used in the implementation of the Raft protocol has been replaced with bbolt, a fork from the etcd project.
Source: opennet.ru