The first stable release of the TidesDB project has been published, which develops a library for storing data in key/value format. The project can be used as a standalone engine or embedded into applications. Key features include support for ACID transactions and optimization for efficient data storage on Flash drives and in memory. The code is written in C and is distributed under the MPL 2.0 license. Bindings have been prepared for C++, Go, Java, Python, Rust, Lua, and JavaScript.
An LSM-tree (Log-Structured Merge-tree) is used for data storage, providing high performance not only during reading but also when performing write and update operations. Read operations do not block data writing and reading, while write operations only block the column family in which the write is being performed. To further speed up reading, indexing and caching based on bloom filters are supported (allowing membership checks with possible false positives but excluding misses on existing elements).
There is full support for ACID transactions, allowing atomic changes across column families and, if necessary, atomic rollbacks of operations. Data can be stored in compressed form with a choice of compression algorithms linked to the column family. Locking can be applied at the level of individual column families, and storage packaging can occur in parallel mode without blocking database operations.
Main Features:
- Support for column families (Column Families) — isolated sets of data in key/value format. Each column family uses its own memtable, SSTable, and WAL log.
- ACID transactions covering multiple operations with column families.
- Optimization for parallel access to data.
- Support for bi-directional iterators, allowing iteration over key/value pairs forward or backward relative to a selected position in the storage, using an initially defined sorting method.
- Support for adding custom key comparison functions. Built-in comparison functions include memcmp, string, and numeric.
- Operation log management (WAL, Write-Ahead Log) and automatic restoration of the memtable (in-memory storage) from this log at startup.
- Using Bloom filters to minimize storage access by checking for key existence before reading from SSTable (persistent storage).
- Support for Snappy, LZ4, and ZSTD algorithms for compressing records in SSTable and WAL logs.
- Ability to define the time-to-live (TTL) for records and automatically exclude expired entries.
- Three data synchronization modes in RAM with persistent storage: NONE (fastest), BACKGROUND (balanced), FULL (most reliable).
- Ability to configure memtable size, compression methods, bloom filters, synchronization mode, and packing parameters for each column family.
- Simple C API that returns 0 upon successful operation and -1 in case of an error.
- Utilization of SBHA (Sorted Binary Hash Array) for fast data retrieval in SSTable, allowing block offsets with keys to be determined without full scans.
- Compact portable binary format for database representation, supporting versioning.
- Built-in mechanisms for integrity checks, detecting data corruption, and recovery.
Source: opennet.ru
