After six years of development, the release of DuckDB 1.0 has arrived, positioned as an SQLite variant for analytical queries. DuckDB combines features of SQLite such as compactness, embedding capabilities as an in-process library, single-file database storage, and a user-friendly CLI interface, with tools and optimizations for executing analytical queries that cover a significant portion of stored data, such as aggregating all table contents or merging several large tables. The project code is written in C++ and is distributed under the MIT license.
Version 1.0 is marked as the first stable release of the project, during which the main focus was on enhancing stability rather than increasing functionality. The new version also solidifies the data storage format, which has been backward compatible since the previous release. In future releases, developers intend to be more cautious about adding new features while maintaining compatibility between releases, as well as stabilizing the SQL dialect and C API. If changes to SQL semantics are necessary in the future, developers will provide advance warnings about upcoming changes and offer workarounds to maintain the functionality of existing code.
DuckDB supports an extended SQL dialect, including additional capabilities for handling very complex and long-running queries. For example, it allows the use of complex types (arrays, structures, unions), as well as executing arbitrary and nested correlated subqueries. Concurrent execution of multiple queries is supported, along with executing queries directly from files in CSV and Parquet formats. Import support from PostgreSQL databases is also available.
In addition to the SQLite shell code, the project utilizes a separate library parser from PostgreSQL, the Date Math component from MonetDB, its own implementation of window functions (based on the Segment Tree Aggregation algorithm), a regular expression handler based on the RE2 library, its own query optimizer, a multi-version concurrency control (MVCC) mechanism for handling concurrent task execution, and a vectorized query execution engine based on the Hyper-Pipelining Query Execution algorithm, allowing for large sets of values to be processed in a single operation.
Source: opennet.ru
