The release of the distributed database rqlite 7.0 has taken place, which uses SQLite as its storage engine and allows for the organization of a cluster of synchronized storage solutions. Notable features of rqlite include the ease of installation, deployment, and maintenance of a distributed fault-tolerant storage system, similar to etcd and Consul, but utilizing a relational data model instead of a key/value format. The project's code is written in Go and is distributed under the MIT license.
To maintain all nodes in a synchronized state, the Raft consensus algorithm is used. Rqlite employs the original SQLite library and the go-sqlite3 driver, on top of which a layer runs that processes client requests, performs replication to other nodes, and monitors consensus achievement for leader election.
Changes to the database can only be made by the node elected as the leader, but connections for write operations can be directed to other cluster nodes, which will return the address of the leader for request retry (the next version promises to add automatic forwarding of requests to the leader). The primary focus is on fault tolerance, which is why the database scales only for read operations, and write operations become a bottleneck. It is possible to run an rqlite cluster from a single node, and this solution can be used to provide access to SQLite over HTTP without ensuring fault tolerance.
SQLite data on each node is stored not in a file but in memory. At the layer implementing the Raft protocol, a log of all SQLite commands that lead to database changes is maintained. This log is used for replication (replicating at the request playback level on other nodes), when starting a new node, or for recovery after a loss of connectivity. To reduce the size of the log, automatic packing is applied, which is triggered after a specified number of changes and results in a snapshot being taken, relative to which a new log begins (the in-memory state of the database is identical to the snapshot plus the accumulated log of changes).
Features of rqlite:
- Ease of deploying a cluster without the need for a separate SQLite installation.
- The ability to quickly obtain a replicated SQL storage.
- Ready for use in production projects.
- Availability of an HTTP(S) API that allows batch data updates and defines the leading node of the cluster. A command-line interface and client libraries for various programming languages are also provided.
- Availability of a service for identifying other nodes, allowing dynamic cluster creation.
- Support for encrypting data exchanges between nodes.
- Ability to configure the level of data freshness and consistency checks upon reading.
- Optional capability to connect nodes in read-only mode, which do not participate in consensus determination and are used to increase cluster scalability for read operations.
- Support for a custom form of transactions based on grouping commands in a single request (transactions based on BEGIN, COMMIT, ROLLBACK, SAVEPOINT, and RELEASE are not supported).
- Support for creating hot backups.
In the new release:
- Added support for automatic clustering of rqlite using a new service for node discovery that can work based on distributed storage solutions like Consul and etcd. This allows rqlite nodes to automatically find each other — the administrator just needs to start a few nodes on different servers, specifying a common address for the Consul or etcd cluster (e.g., "example.com:8500"), and the nodes will automatically be grouped into a cluster. The leading node periodically updates its address information in Consul or etcd, allowing the leader to change later without needing to reconfigure the other nodes, as well as adding new nodes even after changing the leader. Support for the old node discovery service (Discovery mode) based on AWS Lambda has been discontinued.
- In the CLI interface, it is allowed to specify multiple hosts at once — if the first node is unavailable, the subsequent hosts will be accessed.
- Refactored the code for parsing command-line arguments of rqlited.
- Discontinued the use of the deprecated protobuf package.
- The BoltDB storage used in the implementation of the Raft protocol has been replaced with bbolt, a fork of the etcd project.
Source: opennet.ru
