PayPal has opened the source code of its fault-tolerant database JunoDB, which manipulates data in a key-value format. The system is designed with a focus on high security, horizontal scalability, fault tolerance, and the ability to handle hundreds of thousands of simultaneous connections with predictable latency. Almost all services at PayPal, from user login to processing financial transactions, are tied to JunoDB. The project code is written in Go (with a client library in Java) and is distributed under the Apache 2.0 license. Future development will accept contributions, improvements, and changes from the community.
The architecture of JunoDB is based on a load balancer that receives requests from client applications and distributes them among proxy servers, which simultaneously communicate with a group servers of storage while the request is being processed. Each proxy server establishes connections with all servers storage nodes and redirects requests to the storage server group based on a partitioning index stored in a distributed configuration storage system called etcd.
Data is partitioned and bound to storage nodes using hashing, which reduces data movement when adding or removing nodes in the cluster. To ensure fault tolerance, each data shard is replicated across multiple storage nodes, allowing data retention in the event of server failures. Geographically distributed storage is supported, where groups of nodes are located in different data centers.
Data is stored on storage nodes in memory or in local storage based on the RocksDB library. For persistent storage, data is stored in encrypted form (encryption keys can be defined both by the client and set at the proxy level).
To access the database from applications, a client library is provided, offering an API for applications in Java, Go, and C++. The client side is simplified as much as possible, while complex logic and configurations are delegated to the DBMS. Communication between the client and the load balancer or proxy occurs over an encrypted communication channel. For management and sending requests, a command-line interface can be used, which replicates all capabilities of the client API.
The system is designed to handle requests with predictably low latencies, for example, a cluster of three storage nodes and one proxy, configured from n1-highmem-32 instances (32 Intel Xeon 2.30GHz CPUs, 214G RAM, and 450G SSD storage), managed to provide fixed latencies not exceeding 2.5 ms in 95% of cases and 16 ms in 99% while processing 200,000 simultaneous TLS connections at a throughput of 15,000 requests per second (at 3,000 simultaneous connections and a throughput of 80,000 requests per second, the latencies did not exceed 6 ms in 95% and 15 ms in 99%). In PayPal, services based on JunoDB handle approximately 350 billion requests per day.
Source: opennet.ru
