The release of the open-source database Nebula Graph 3.2 has been published, designed for the efficient storage of large sets of interconnected data, forming a graph that can contain billions of nodes and trillions of connections. The project is written in C++ and is distributed under the Apache 2.0 license. Client libraries for accessing the database are available for Go, Python, and Java.
The database employs a distributed architecture without resource sharing (shared-nothing), implying the launch of independent and self-sufficient processes for query processing (graphd) and storage (storaged). A meta-service handles data movement orchestration and provides metadata about the graph. A protocol based on the RAFT algorithm is used to ensure data consistency.
Key features of Nebula Graph:
- Ensuring security by granting access only to authenticated users, whose permissions are defined through a role-based access control (RBAC) system.
- Capability to connect different types of storage engines. Support for extending the query language with new algorithms.
- Ensuring minimal latency during data reading or writing while maintaining high throughput. During testing in a cluster consisting of one graphd node and three storaged nodes with a 632 GB database, including a graph of 1.2 billion vertices and 8.4 billion edges, latency was at the level of several milliseconds, while throughput reached up to 140,000 queries per second.
- Linear scalability.
- SQL-like query language, powerful enough yet easy to comprehend. It supports operations such as GO (bidirectional traversal of graph nodes), GROUP BY, ORDER BY, LIMIT, UNION, UNION DISTINCT, INTERSECT, MINUS, and PIPE (using the result from the previous query). Indexes and user-defined variables are supported.
- Ensuring high availability and fault tolerance.
- Support for creating snapshots of the database state to facilitate backup creation.
- Ready for industrial application (already used in the infrastructure of companies JD, Meituan, and Xiaohongshu).
- The ability to change the storage schema and update data without stopping or affecting ongoing operations.
- TTL support for data lifespan management.
- Commands for managing settings and storage hosts.
- Tools for job management and scheduling (currently, only COMPACT and FLUSH jobs are supported).
- Operations for searching the full path and shortest path between specified vertices.
- OLAP interface for integration with third-party analytics platforms.
- Utilities for importing data from CSV files or from Spark.
- Export metrics for monitoring using Prometheus and Grafana.
- Web interface Nebula Graph Studio for visualizing graph operations, navigating the graph, designing storage schemas, and loading data.
In the new release:
- Support for the extract() function has been added to extract substrings matching a specified expression.
- Configuration file settings have been optimized.
- Optimization rules have been added for removing the useless AppendVertices operator and disabling the application of edge and vertex filters.
- Reduced the volume of data copied for JOIN operations, as well as for Traverse and AppendVertices operators.
- Performance of SHORTEST PATH and SUBGRAPH has been optimized.
- Improved memory allocation (Arena Allocator used).
Source: opennet.ru
