The release of the distributed database management system rqlite 6.0 has been announced, which uses SQLite as its storage engine and allows for the organization of a cluster of synchronized storage systems. rqlite is noted for its simplicity in installation, deployment, and maintenance of a distributed fault-tolerant storage system, somewhat similar to etcd and Consul, but utilizing a relational model for data management instead of a key/value format. The project's code is written in Go and is distributed under the MIT license.
To maintain all nodes in a synchronized state, the Raft consensus algorithm is used. Rqlite employs the original SQLite library and the standard go-sqlite3 driver, on top of which a layer processes client requests, performs replication to other nodes, and monitors the consensus in selecting the leader node.
Changes to the database can only be made by the node that is selected as the leader, but connections for write operations can also be directed to other nodes in the cluster, which will return the leader's address for request retrying (automatic forwarding to the leader is promised in the next version). The main emphasis is on fault tolerance, which is why the DBMS scales only for read operations, with write operations being a bottleneck. It is possible to run an rqlite cluster from a single node, and this solution can be used to organize access to SQLite over HTTP without providing fault tolerance.
SQLite data on each node is stored not in a file but in memory. At the layer implementing the Raft protocol, a log of all SQLite commands that lead to changes in the database is maintained. This log is used during replication (replication at the level of executing requests on other nodes), starting a new node, or recovery after loss of connectivity. To reduce the log size, automatic packaging is applied, which is triggered after a specified number of changes and results in a snapshot being committed to disk, relative to which a new log begins (the in-memory state of the database is identical to the snapshot plus the accumulated log of changes).
Features of rqlite:
- Ease of deploying a cluster without the need for a separate SQLite installation.
- The ability to quickly obtain a replicated SQL storage.
- Ready for use in production projects.
- Availability of an HTTP(S) API that allows for batch updates of data and identifies the cluster's leading node. A command-line interface is also provided, along with the ability to use various client libraries created for SQLite.
- Availability of a service for identifying other nodes, allowing dynamic cluster creation.
- Support for encrypting data exchanges between nodes.
- Ability to configure the level of data freshness and consistency checks upon reading.
- Optional capability to connect nodes in read-only mode, which do not participate in consensus determination and are used to increase cluster scalability for read operations.
- Support for a custom form of transactions based on grouping commands in a single request (transactions based on BEGIN, COMMIT, ROLLBACK, SAVEPOINT, and RELEASE are not supported).
- Support for creating hot backups.
The new release includes significant architectural changes aimed at enhancing the reliability of the cluster by improving the request routing process for reading and writing to the appropriate cluster nodes. Rqlite nodes can now multiplex between multiple logical connections using TCP connections established between nodes via the Raft protocol. If a request requires the authority of the leader node but is sent to a secondary node, the secondary node can determine the leader's address and relay it to the client without executing a consensus calculation via the Raft protocol.
This change has also eliminated the need for a separate component for synchronizing metadata and ended the separate handling of Raft state and metadata. Secondary nodes now route requests to the leader node only when necessary to determine the address of the leader node. The API provides the ability to obtain information about the status of other nodes in the cluster. A new command '.sysdump' has been added to the command-line interface.
Source: opennet.ru
