Citus Data, a subsidiary of Microsoft, has released the distributed database Citus 13.0, implemented as an extension to PostgreSQL 17. Citus enables horizontal scaling of PostgreSQL in a cluster based on standard hardware, allowing data to be spread across nodes using sharding with configuration for column-level partitioning and storage schemes. For applications, the Citus cluster appears as a single large PostgreSQL server, aggregating the resources of its constituent nodes. The code is written in C and is distributed under the AGPLv3 license.
Sharding allows for the organization of very large volumes of data, the total size of which significantly exceeds the local storage of each node in the cluster. Individual tables can be forcibly replicated across all nodes to speed up merge operations and work with foreign keys. To save disk space, data distributed across different nodes can be stored in a compressed format.
Distributed queries can be sent to any node in the cluster, but management and modification of the data schema must be done only through the node coordinating the cluster's operation. Incoming client queries are distributed to the necessary nodes and, if they span multiple nodes, their processing is parallelized. The cluster can be expanded as the size of the stored data grows by adding additional nodes and initiating rebalancing.
Typical examples of Citus usage include executing analytical queries and processing large data arrays in the form of time series (for instance, logs or sensor status polling). Citus is also suitable for modernizing existing infrastructure based on a single server PostgreSQL, the performance and storage of which are no longer sufficient due to increasing load or volume of incoming data. Using the Patroni toolkit, you can create fault-tolerant configurations with replicated standby nodes capable of taking over primary nodes in the event of a failure.
Changes in the Citus 13.0 release:
- Transition to using the PostgreSQL 17 branch.
- The ability to use the JSON_TABLE() function in distributed queries to convert data from JSON format into a relational representation that can be treated like regular PostgreSQL tables.
- Support for the syntax "MERGE … WHEN NOT MATCHED BY SOURCE."
- The ability to define an access method for distributed partitioned tables using the expression "CREATE TABLE … USING," as well as changing the access method through "ALTER TABLE … SET ACCESS METHOD."
- Support has been added for specifying automatically generated identification columns in distributed partitioned tables.
- The use of exclusion constraints in distributed partitioned tables is now allowed.
- Issues with role synchronization between nodes have been resolved.
- Improved data distribution for sharding when adding new nodes to the cluster.
- The "EXPLAIN" command now includes the options "MEMORY" and "SERIALIZE," which can be used to determine the memory size used and the time spent on data transformation for network transmission.
Source: opennet.ru
