Cloudflare has announced the transition of its content delivery network to the use of the Pingora proxy, written in Rust. This new proxy has replaced the previous configuration based on NGINX with Lua scripts and handles over a trillion requests per day. It is noted that switching to a specialized proxy has not only enabled the implementation of new features and increased security through safe memory handling but has also led to significant performance improvements and resource savings — the Pingora-based solution consumes 70% less CPU resources and 67% less memory while processing the same traffic volume, as it does not require the use of Lua.
For a long time, the system for proxying traffic between users and endpoints servers based on NGINX and Lua scripts met Cloudflare's needs, but as the network grew and its complexity increased, the universal solution proved insufficient both in terms of performance and due to limitations in scalability and implementing new client features. In particular, challenges arose in adding functionalities that went beyond a simple gateway and load balancer. For example, it became necessary to redirect requests to another server in case of a request processing failure, proxy server supplying it with a different set of HTTP headers.
Instead of an architecture that separates requests among individual worker processes, Pingora employs a multithreaded model, which has demonstrated more efficient resource distribution between CPU cores in Cloudflare's usage scenarios (high traffic concentration from various sites with significant statistical shifts). In particular, binding unbalanced requests to processes in NGINX resulted in uneven load on CPU cores, causing resource-intensive requests and blocking I/O to slow down the processing of other requests. Moreover, binding the connection pool to worker processes did not allow already established connections from other worker processes to be reused, diminishing efficiency when there are many worker processes.
NGINX:
Pingora:
The implementation of Pingora has reduced the number of new connection setup operations by 160 times and increased the share of reusable requests from 87.1% to 99.92%. In addition to the reduction in repeated connections and more efficient use of CPU cores, the improved performance of the new proxy was primarily due to eliminating the slow handlers used with nginx that were written in Lua.
The Rust language was chosen for its ability to achieve high performance combined with tools that ensure safe memory operation. It is mentioned that despite the high qualifications of Cloudflare engineers and code reviews, mistakes leading to memory issues (such as an HTML parser vulnerability) could not be avoided in the code written in C. Regarding the new code, it was discussed that some crashes in Pingora were caused not by application issues but by a bug in the Linux kernel and hardware failures.
Additionally, it is worth noting the comment from Linus Torvalds about the inclusion of Rust support in the Linux kernel, made during the ongoing Open-Source Summit Europe conference. While the 6.0 kernel did not include patches for developing device drivers in Rust, Linus stated that they are likely to be accepted in the 6.1 kernel, as he does not intend to delay the integration. As motivation for adding Rust support, in addition to its positive impact on security, Linus mentioned the potential to increase interest in kernel work from new contributors, which is relevant in light of the aging of long-time contributors.
Source: opennet.ru
