The Apache Software Foundation has released the distributed database Apache Cassandra 4.0, which belongs to the noSQL system class and is designed for creating highly scalable and reliable storage solutions for vast amounts of data stored in the form of an associative array (hash). The release of Cassandra 4.0 is considered ready for production deployments and has already been tested in the infrastructures of companies like Amazon, Apple, DataStax, Instaclustr, iland, and Netflix, with clusters consisting of over 1,000 nodes. The project's code is written in Java and is distributed under the Apache 2.0 license.
Initially, the Cassandra database was developed by Facebook and was handed over to the Apache Foundation in 2009. Industrial solutions based on Cassandra have been deployed to ensure the operation of services for companies such as Apple, Adobe, CERN, Cisco, IBM, HP, Comcast, Disney, eBay, Huawei, Netflix, Sony, Rackspace, Reddit, and Twitter. For example, Apple has deployed an infrastructure for storage based on Apache Cassandra that includes over a thousand clusters, totaling 160,000 nodes and storing more than 100 petabytes of data. Huawei uses more than 300 Apache Cassandra clusters, totaling 30,000 nodes, while Netflix has over 100 clusters covering 10,000 nodes and handling over a trillion requests per day.
Cassandra combines a fully distributed hash system called Dynamo, which provides near-linear scalability as data volume increases. Cassandra uses a data storage model based on a column family structure, which differs from systems like memcachedb that store data only in key/value pairs, allowing for the organization of hash storage with multiple levels of nesting. To simplify interaction with the database, the Cassandra Query Language (CQL), which resembles SQL but has reduced functionality, is supported. Key features include support for namespaces and column families, as well as index creation through the expression 'CREATE INDEX'.
The database allows for the creation of fault-tolerant storage: data placed in the database is automatically replicated across multiple nodes in a distributed network, which can span different data centersIn the event of a node failure, its functions are seamlessly taken over by other nodes. Adding new nodes to the cluster and updating the Cassandra version can be done on the fly, without additional manual intervention or reconfiguration of other nodes. Drivers that support CQL are available for Python, Java (JDBC/DBAPI2), Ruby, PHP, C++, and JavaScript (Node.js).
Key innovations:
- Improved performance and scalability. Data exchange efficiency in the SSTable (Sorted Strings Table) format between nodes has been enhanced. The protocol for message exchange between nodes (Internode Messaging Protocol) has been optimized. The data stream transfer rate between nodes has increased up to five times (primarily due to the implementation of Zero Copy techniques and the whole SSTables transmission), and the throughput for read and write operations has improved by 25%. The incremental recovery process has been optimized. Delays caused by garbage collector pauses are reduced to a few milliseconds.
- Support for an audit log has been added, allowing tracking of user authentication operations and all executed CQL queries.
- The capability to maintain a full binary query log has been added, enabling the preservation of all request and response traffic. Commands 'nodetool enablefullquerylog|disablefullquerylog|resetfullquerylog' are offered for management, and the fqltool utility is supplied for log analysis. Commands for converting the log to a readable format (Dump), comparing activity snapshots (Compare), and replaying (Replay) for analysis with real load conditions have been provided.
- Support for virtual tables has been added, which reflect not the data stored in SSTables, but information output through the API (performance metrics, settings information, cache content, details of connected clients, etc.).
- The efficiency of storing data in compressed form has been enhanced, allowing for reduced disk space consumption and improved read operation performance.
- Data related to the system keyspace (system.*) is now by default placed in the first directory instead of being distributed across all data directories, allowing the node to maintain functionality in the event of a failure of one of the additional disks.
- Experimental support for Transient Replication and Cheap Quorums has been added. Transient replicas do not store all data and use incremental recovery to synchronize with full replicas. Cheap Quorums optimize write operations, where writes to transient replicas are not performed until a sufficient set of full replicas is available.
- Experimental support for Java 11 has been added.
- An experimental option for comparing all Merkle Trees has been added. For example, enabling the option on a 3-node cluster where two replicas are identical and one is outdated will trigger an update of the outdated replica using only one copy operation from the current replica.
- New functions currentTimestamp, currentDate, currentTime, and currentTimeUUID have been added.
- Support for arithmetic operations in CQL queries has been added.
- The ability to perform arithmetic operations between data types 'timestamp'/'date' and 'duration' has been provided.
- A mode for previewing the necessary data streams for recovery (nodetool repair âpreview) and the ability to check the integrity of the recovered data (nodetool repair âvalidate) has been added.
- The SELECT queries now support the processing of Map and Set elements.
- Support for parallelizing the initial build phase of materialized views (cassandra.yaml:concurrent_materialized_view_builders) has been added.
- The 'nodetool cfstats' command now supports sorting by specific metrics and limiting the number of output lines.
- Settings have been provided to restrict user connections to specific data centers only.
- The ability to limit the rate of snapshot creation and cleanup operations has been added.
- Support for Python 3 has been implemented in cqlsh and cqlshlib (support for Python 2.7 is still retained).
- Support for the Windows platform has been discontinued. It is recommended to run Cassandra in Linux environments created based on the WSL2 (Windows Subsystem for Linux 2) or virtualization systems.
Source: opennet.ru
