Cloudflare has announced that it is migrating its content delivery network to use the Pingora proxy, written in Rust. The new proxy has replaced the NGINX server-based configuration with Lua scripts, and handles over a trillion requests per day. It is noted that the transition to a specialized proxy allowed not only to implement new features and increase security due to safe work with memory, but also led to a significant increase in performance and resource savings - a solution based on Pingora does not require the use of Lua, therefore it consumes 70% less resources CPU and 67% less memory while handling the same amount of traffic.
For a long time, a system for proxying traffic between users and end servers based on NGINX and Lua scripts satisfied the needs of Cloudflare, but as the network grew and its complexity increased, a universal solution was not enough, both in terms of performance, and due to limitations in extensibility and implementation of new opportunities for clients. In particular, there were difficulties in adding functionality beyond a simple gateway and load balancer. For example, in the event that the server fails to process a request, it became necessary to redirect the request to another server, supplying it with a different set of HTTP headers.
Instead of an architecture with splitting requests into separate worker processes (workers), Pingora uses a multi-threaded model, which in Cloudflare usage scenarios (high concentration of traffic from different sites with a large statistical shift) showed a more efficient distribution of resources between CPU cores. In particular, binding unbalanced requests to processes in nginx led to an unbalanced load on the CPU cores, as a result of which resource-intensive requests and blocking I / O slowed down the processing of other requests. In addition, binding the connection pool to handler processes did not allow reuse of already established connections from other handler processes, which reduces efficiency when there are a large number of handler processes.
nginx:
Pingora:
The introduction of Pingora made it possible to reduce the number of new connection establishment operations by 160 times and increase the share of reused requests from 87.1% to 99.92%. In addition to reducing reconnects and more efficient use of CPU cores, the performance of the new proxy is mainly due to getting rid of the slow Lua handlers used with nginx.
The Rust language was chosen as allowing for high performance combined with the availability of memory-safe tools. It is mentioned that despite the high qualification of Cloudflare engineers and code reviews written in the C language, it was not possible to avoid errors that lead to memory problems (for example, a vulnerability in the HTML parser). As for the new code, we talked about cases of analysis of crashes in Pingora, which turned out to be caused not by problems in the application, but by a bug in the Linux kernel and hardware failures.
Additionally, it can be noted that Linus Torvalds commented on the inclusion of support for the Rust language in the Linux kernel, voiced at the Open-Source Summit Europe conference taking place these days. The 6.0 kernel did not include patches for developing device drivers in the Rust language, but according to Linus, they are likely to be adopted in the 6.1 kernel, he is not going to pull with integration. As a motive for adding support for Rust, in addition to a positive impact on security, Linus also calls the opportunity to increase interest in working on the core of new participants, which is important in the context of aging old-timers.
Source: opennet.ru