The libmdbx 0.10.4 (MDBX) library was released with the implementation of a high-performance compact embedded key-value database, and the associated libfpta 0.3.9 (FPTA) library, which implements a tabular representation of data with secondary and composite indexes on top of MDBX. Both libraries are distributed under OSI approved licenses. All current operating systems and architectures are supported, as well as the Russian Elbrus 2000.
Historically, libmdbx is a deep reworking of the LMDB DBMS and is superior to its ancestor in reliability, feature set and performance. Compared to LMDB, libmdbx places a lot of emphasis on code quality, API stability, testing, and automated checks. A utility for checking the integrity of the database structure with some recovery capabilities is supplied.
Technology-wise, libmdbx offers ACID, strong change serialization, and non-blocking reads with linear scaling across CPU cores. Auto-compactification, automatic database size management, and range query estimation are supported. Since 2016, projects have been funded by Positive Technologies and since 2017 have been used in its products.
libmdbx offers a C++ API, as well as enthusiast-supported language bindings for Rust, Haskell, Python, NodeJS, Ruby, Go, and Nim. For libfpta, only the API description is publicly available in the form of a C/C++ header file.
Major innovations, improvements and corrections added since the previous news on May 9:
- Enables reproducible builds.
- Fixed a bug due to which, in very rare circumstances, a loop/freeze could occur during a transaction commit. The problem was identified by Positive Tecnologies specialists during internal testing of their own products.
- Tests have been improved and test scenarios have been expanded to check all reachable non-isomorphic states of the page tree and GC contents inside the database.
- In the C++ API, an extra “noexcept” has been fixed, additional overloads have been added for the “cursor::erase()” method, the implementation of buffers has been spared the use of “std::string” to ensure alignment (relevant for CLANG libstdc++).
- A regression in the dirty page spilling algorithm (selective ejection of changed database pages) that was manifested by a rare unexpected error MDBX_PROBLEM when changing data in huge transactions has been eliminated.
- A phasing test was carried out with the addition of a number of checks to ensure stability in the event of intentional damage to the database.
- Fixed minor warnings UndefinedBehaviorSanitizer and Coverity Scan issues.
- Fixed checking the outdated and no longer used internal flag “P_DIRTY” in nested pages inside database images created by older versions of the library.
- In CMake scripts, the search for compiler components required for LTO (link-time optimization) has been improved.
- The maximum number of simultaneous readers has been increased to 32767.
- Improved performance when using Valgrind and AddressSanitizer.
- On Windows, the recursive use of SRW-lock when working in MDBX_NOTLS mode (without using thread local storage) has been eliminated, bootid generation has been fixed if the system time has changed, WSL1 and WSL2 detection has been improved, and the ability to open a database on Plan 9 mounted via DrvFS has been added.
- In total, more than 160 changes were made to 57 files, ~5000 lines were added, ~2500 were deleted.
I would especially like to thank the Erigon project team (Ethereum ecosystem) for their assistance in testing in extreme use scenarios. It is significant that in five months since the release of libmdbx v0.10.0, with a database volume of 1-2 TB in each Erigon installation (used on 7% of Ethereum nodes), only three reports of database corruption were received, all of which occurred due to external reasons, and not software errors: in two cases the cause was RAM failures, in the third an error in resetting data in a specific configuration of the storage subsystem using BTRFS.
Source: opennet.ru