Marc Brooker, an engineer at Amazon Web Services (AWS), addressed misconceptions related to enhancing the efficiency of small message transmission using Nagle's algorithm, which is the default in the TCP/IP stack. The recommendations center around disabling Nagle's algorithm by default, which can be done for specific applications by setting the TCP_NODELAY option for network sockets using the setsockopt call, a practice already established in projects like Node.js and curl.
Nagle's algorithm allows the aggregation of small messages to reduce traffic—it holds off sending new TCP segments until an acknowledgment of the previously sent data is received. For example, without aggregation, sending 1 byte results in an additional 40 bytes being sent with the TCP and IP headers. In modern conditions, the use of Nagle's algorithm leads to noticeable increases in latency, which are unacceptable for interactive and distributed applications.
There are three main arguments in favor of using the TCP_NODELAY option by default, which disables Nagle's algorithm:
- Incompatibility of Nagle's algorithm with the 'delayed ACK' optimization, where the ACK response is not sent immediately but after receiving response data. The problem is that in Nagle's algorithm, the arrival of the ACK packet serves as a signal to send aggregate data, and if the ACK packet is not received, sending occurs only after a timeout. This creates a feedback loop where the ACK packet does not serve as a signal because the other side does not receive data due to its accumulation on the sender's side, while the sender does not send it until the timeout occurs, as the ACK packet is not received.
- The RFC for Nagle's algorithm was adopted in 1984 and is not designed for the parameters of modern high-speed networks and servers data centers, which leads to responsiveness issues. The delay between sending a request and receiving a response (RTT) in modern networks is 0.5 ms + a few milliseconds when exchanging data between data centers in the same region + up to a hundred milliseconds when sending worldwide. During these milliseconds, a modern server can perform a massive amount of work.
- Modern distributed applications no longer send individual bytes of data, as aggregation of smaller data usually takes place at the application level. Even when the size of useful data is just a few bytes, the actual size of the information sent typically increases substantially after serialization, using API wrappers in JSON, and transmission with TLS encryption. Saving 40 bytes becomes less relevant.
Source: opennet.ru
