The developers of the SQLite project have started testing an experimental backend called HCtree, which supports row-level locking and provides a high level of parallelism in processing requests. The new backend aims to improve the efficiency of SQLite usage in client-server systems that handle a large number of simultaneous write requests to the database.
The b-tree structures originally used in SQLite for data storage are not designed for such load, which limits SQLite to writing in a single thread. As an experiment, the developers have begun to develop an alternative solution using the HC-tree structure for storage, which is better suited for parallelizing write operations.
To enable simultaneous execution of multiple operations in HCtree, a transaction partitioning mechanism is applied, utilizing page-level locking and resembling MVCC (multi-version concurrency control), but using transaction checks based on keys and key ranges instead of page sets. Read and write operations are tied to a database snapshot, with changes becoming visible in the main database only after the transaction is completed.
Clients can use three operations to open transactions:
- “BEGIN” — transactions do not consider data access from other clients. If write operations are performed within the transaction, the transaction can be committed only if no other write operations have occurred in the database during its execution.
- “BEGIN CONCURRENT” — transactions gather information about access from other clients. If write operations are performed within the transaction, the transaction can be committed if other transactions have been committed to the database since the snapshot was created.
- “BEGIN EXCLUSIVE” — after opening the transaction, it locks operations from other transactions until its completion.
HCtree supports master-slave replication, allowing transactions to be transferred to another database and keeping secondary databases synchronized with the primary one. HCtree also removes the limitation on database size—using 48-bit data page identifiers instead of 32-bit, which increases the maximum database size from 16 tebibytes to 1 exbibyte (a million tebibytes). The performance of SQLite with the HCtree backend is expected to be no less than that of the classic single-threaded backend. SQLite clients supporting HCtree will be able to access both the HC-tree structured database and legacy SQLite databases.
Source: opennet.ru
