The Apache Software Foundation has released a distributed DBMS Apache Cassandra 4.0, which belongs to the class of noSQL systems and is designed to create highly scalable and reliable storages of huge data arrays stored in the form of an associative array (hash). The release of Cassandra 4.0 is considered ready for production deployments and has already been tested in the infrastructures of Amazon, Apple, DataStax, Instaclustr, iland and Netflix with clusters of more than 1000 nodes. The project code is written in Java and distributed under the Apache 2.0 license.
Cassandra was originally developed by Facebook and was donated to the Apache Foundation in 2009. Cassandra-based industrial solutions have been deployed to power services from companies such as Apple, Adobe, CERN, Cisco, IBM, HP, Comcast, Disney, eBay, Huawei, Netflix, Sony, Rackspace, Reddit, and Twitter. For example, Apple's Apache Cassandra-based storage infrastructure has more than 160 clusters with 100 nodes and more than 300 petabytes of data. Huawei has more than 30 Apache Cassandra clusters with 100 nodes, and Netflix has more than 10 clusters with XNUMX nodes and more than a trillion requests per day.
Cassandra combines Dynamo's fully distributed hash system to provide near-linear scalability as data grows. Cassandra uses a data storage model based on a column family (ColumnFamily), which differs from systems like memcachedb, which store data only in a key / value pair, by the ability to store hashes with several levels of nesting. To simplify interaction with the database, the CQL (Cassandra Query Language) structured query language is supported, reminiscent of SQL, but truncated in functionality. Of the possibilities, one can note the support of namespaces and column families, the creation of indexes through the CREATE INDEX statement.
The DBMS allows you to create fault-tolerant storage: the data placed in the database is automatically replicated to several nodes of a distributed network, which can cover different data processing centers. When a node fails, its functions are picked up on the fly by other nodes. Adding new nodes to the cluster and updating the Cassandra version is done on the fly, without additional manual intervention and reconfiguration of other nodes. CQL-enabled drivers are prepared for Python, Java (JDBC/DBAPI2), Ruby, PHP, C++, and JavaScript (Node.js).
Main innovations:
- Improved performance and scalability. Improved efficiency of data exchange in SSTable (Sorted Strings Table) format between nodes. The protocol for messaging between nodes (Internode Messaging Protocol) has been optimized. The speed of transferring data streams between nodes has increased up to 5 times (mainly due to the use of the Zero Copy technique and the transfer of SSTables in its entirety), and the throughput for reading and writing operations up to 25%. Optimized the process of incremental recovery. Latencies due to garbage collector pausing have been reduced to a few milliseconds.
- Added support for an audit log that allows you to track user authentication operations and all executed CQL queries.
- Added the ability to maintain a complete binary request log, which allows you to save all request and response traffic. The commands βnodetool enablefullquerylog|disablefullquerylog|resetfullquerylogβ are proposed for management, and the fqltool utility is supplied for analyzing the log. Commands are provided for converting the log into a readable form (Dump), comparing activity slices (Compare), and re-executing (Replay) for analysis with reproduction of conditions inherent in a real load.
- Added support for virtual tables that reflect not data stored in SSTables, but information displayed via the API (performance metrics, settings information, cache contents, information about connected clients, etc.).
- The efficiency of compressed data storage has been improved to reduce disk space consumption and improve read performance.
- Data related to the system keyspace (system.*) is now placed in the first directory by default instead of being distributed across all directories with data, which allows the node to remain operational if one of the additional disks fails.
- Added experimental support for Transient Replication and Cheap Quorums. Temporary replicas do not store all data and use incremental restore to be consistent with full replicas. Lightweight quorums implement write optimisation, where no writes are made to temporary replicas until a sufficient set of full replicas is available.
- Added experimental support for Java 11.
- Added an experimental option to compare all Merkle Trees. For example, enabling the option on a 3-node cluster where two replicas are the same and one is stale will cause the stale replica to be upgraded using only one copy of the up-to-date replica.
- Added new functions currentTimestamp, currentDate, currentTime and currentTimeUUID.
- Added support for arithmetic operations in CQL queries.
- The ability to perform arithmetic operations between data with types "timestamp"/"date" and "duration" is provided.
- Added a mode for previewing the data streams required for restoration (nodetool repair --preview) and the ability to check the integrity of the restored data (nodetool repair --validate).
- SELECT queries now have the ability to process Map and Set elements.
- Added support for parallelization of the initial construction of materialized views (cassandra.yaml:concurrent_materialized_view_builders).
- Added support for sorting by certain metrics and limiting the number of output lines in the "nodetool cfstats" command.
- Provided settings to restrict user connection to only certain datacenters.
- Added the ability to limit the intensity (rate limit) of operations for creating and clearing snapshots.
- cqlsh and cqlshlib support Python 3 (Python 2.7 is still supported).
- Support for the Windows platform has been dropped. To run Cassandra on Windows, it is recommended to use Linux environments created based on the WSL2 (Windows Subsystem for Linux 2) subsystem or virtualization systems.
Source: opennet.ru