The developers of the SQLite project have begun testing an experimental HCtree backend that supports row-level locking and provides a high level of parallelism in query processing. The new backend is aimed at improving the efficiency of using SQLite in client-server systems that have to process a large number of simultaneous write requests to the database.
The b-tree structures originally used in SQLite for storing data are not designed for this kind of load, which limits SQLite to only write to one stream. As an experiment, the developers began to develop an alternative solution that uses HC-tree structures for storage, more suitable for parallelizing write operations.
To handle multiple operations at the same time, HCtree records use a transaction separation mechanism that uses page-level locks, similar to MVCC (multi-version contention control), but uses key-range and key-range based transaction checks instead of page sets. Read and write operations are performed in relation to the database snapshot, changes to which become visible in the main database only after the transaction is completed.
Clients can use three transaction opening operations:
- "BEGIN" - transactions do not take into account access data from other clients. If write operations are performed inside a transaction, the transaction can be committed only if there were no other write operations in the database during the time of its execution.
- "BEGIN CONCURRENT" - transactions collect information about the access of other clients. If write operations are performed inside a transaction, the transaction can be committed if other transactions have been committed in the database since the snapshot was created.
- "BEGIN EXCLUSIVE" - after opening a transaction, it blocks operations from other transactions until it is completed.
HCtree supports master-slave replication, which allows you to transfer transactions to another database and keep secondary databases in sync with the main database. HCtree also removes the database size limit - instead of 32-bit data page identifiers, HCtree uses 48-bit ones, which increases the maximum database size from 16 tebibytes to 1 exbibyte (million tebibytes). It is expected that the performance of SQLite with the HCtree backend will be at least as good as the classic single-threaded backend. SQLite clients with HCtree support will be able to access both HC-tree databases and legacy SQLite databases.
Source: opennet.ru