for testing the second candidate for library releases with the implementation of a high-performance, compact embedded key-value database. The current version (0.5) is a technical release, marks the completion of any improvements and the transition to the phase of public final testing and stabilization, with the subsequent formation of the first full release of the library. libmdbx code licensed under the OpenLDAP Public License.
The MDBX library is a significantly revised fork from - transactional embedded DBMS of the "key-value" class based on without , which allows multi-threaded processes to work competitively and efficiently with a locally shared (not network) database. In turn, MDBX is faster and more reliable than LMDB, and at the same time retains all the key features of its ancestor, such as and non-blocking reads with linear scaling across CPU cores.
The most important differences between MDBX and LMDB are:
- Fundamentally, more attention is paid to code quality, testing and automatic checks.
- Significantly more control during operation, from parameter checking to internal audit of database structures.
- Auto-compactification and automatic database size management.
- A single database format for 32-bit and 64-bit assemblies.
- Estimation of the volume of samples by ranges (range query estimation).
- Support for keys twice the size of pancakes and user-selectable database page size.
The libmdbx release candidate is the result of a decision made in August 2019 to separate the MDBX and MithrilDB projects. At the same time, libmdbx decided to eliminate the (rational) maximum technical debt and stabilize the library. In fact, 2-3 times more has been done in the designated direction than was initially estimated and planned:
- Support for macOS and second-tier platforms has been implemented: FreeBSD, Solaris, DragonFly BSD, OpenBSD, NetBSD. AIX and HP-UX support can be added as needed.
- The code was sanitized using Undefined Behavior Sanitizer and Address Sanitizer, all warnings when building with “-Wpedantic”, all Coverity Static Analyzer warnings, etc. were eliminated.
- Updating .
- Merging (amalgamation) of source code for ease of embedding.
- CMake support.
- Support for nested transactions.
- Using the bootid to determine if the OS has rebooted (dirty shutdown of the database).
- End-to-end counting of updated/old pages and extended transaction information.
- MDBX_ACCEDE option to connect to an already open database in compatible mode.
- Using upon their availability.
- Hot backup in pipe.
- Specialized optimized internal sorting algorithm (up to 2-3 times faster than “qsort()” and up to 30% faster than “std::sort()”).
- Increased maximum key length.
- Automatic control of read ahead (in-memory database file caching strategy).
- More aggressive and faster auto-compactification.
- A more optimal strategy for merging B+ tree pages.
- Control of non-local file systems (NFS, Samba, etc.) to prevent database damage if used incorrectly.
- Expanded test suite.
Development of the "next" version of libmdbx will continue as a separate project , while the development vector of the "current" version of MDBX is aimed at freezing the feature set and stabilizing. This decision was made for three reasons:
- Total incompatibility: MithrilDB requires a different (incompatible) database file format and a different (incompatible) API to implement all the planned features.
- New source code: MithrilDB source code is license independent from LMDB, and the project itself is planned to be published under a different license (approved by Apache 2.0 license, not ).
- The separation avoids potential confusion, introduces more certainty and ensures an independent path for projects.
MithrilDB, like MDBX, is also based on and will also be characterized by extremely high performance, while eliminating a number of fundamental shortcomings of MDBX and LMDB. In particular, the problem of "long reads", which manifests itself as "swelling" of the database due to the fact that garbage processing is blocked by long reading transactions, will be eliminated. New MithrilDB features include:
- Support for placing the database on several heterogeneous media: HDD, SSD and non-volatile memory.
- Optimal strategies for “valuable” and “low-value”, for “hot”, “warm” and “cold” data.
- Using Merkle tree to monitor database integrity.
- Optional use of WAL and significantly improved performance in write-intensive scenarios with data integrity guarantees.
- Lazy catch-up committing data to disks.
Source: opennet.ru