Google has announced the discovery of the specifications and reference implementation of the PSP (PSP Security Protocol), used to encrypt traffic between data centers. The protocol uses an architecture similar to IPsec's ESP (Encapsulating Security Payloads) to encapsulate traffic over IP, providing encryption, cryptographic integrity, and source authentication. The PSP implementation code is written in C and distributed under the Apache 2.0 license.
A feature of PSP is the optimization of the protocol to speed up calculations and reduce the load on the central processor through the transfer of encryption and decryption operations to the side of network cards (offload). To use hardware acceleration, you need special network cards that are compatible with the PSP. For systems with network cards that do not support PSP, a software implementation of SoftPSP is proposed.
The UDP protocol is used as a transport for data transfer. A PSP packet starts with an IP header followed by a UDP header and then a custom PSP header with encryption and authentication information. Next, the content of the original TCP/UDP packet is attached, which ends with a final PSP block with a checksum to confirm the integrity. The PSP header and the header and data of the encapsulated packet are always authenticated to prove the authenticity of the packet. The data of the encapsulated packet may be encrypted, allowing the option to selectively apply encryption while leaving part of the TCP header in the clear (while maintaining authentication control), for example, to enable packet inspection on transit network equipment.
PSP is not tied to any specific key exchange protocol, offers several packet format options and supports the use of different cryptographic algorithms. For example, support is provided for the AES-GCM algorithm for encryption and authentication (authentication) and AES-GMAC for authentication without encryption of direct data, for example, when the data is of no value, but you need to ensure that it has not been tampered with during transmission and exactly those that were originally sent.
Unlike typical VPN protocols, PSP uses encryption at the level of individual network connections, and not the entire communication channel, i.e. The PSP uses separate encryption keys for different tunneled UDP and TCP connections. This approach makes it possible to achieve stricter isolation of traffic from different applications and handlers, which is important when running applications and services of different users on the same server.
Google uses the PSP protocol both to protect its own internal communications and to protect the traffic of Google Cloud customers. The protocol was originally designed to work effectively in infrastructures like Google and should provide hardware-accelerated encryption in the face of millions of active network connections and hundreds of thousands of new connections per second.
Two modes of operation are supported - "stateful" and "stateless". In stateless mode, the keys for encryption are transmitted to the network card in the packet descriptor, and for decryption, they are extracted from the SPI (Security Parameter Index) field present in the packet using a master key (256-bit AES, stored in the memory of the network card and replaced every 24 hours), which saves network card memory and minimizes information about the status of encrypted connections stored on the equipment side. In stateful mode, the keys for each connection are stored on the network card in a special table, similar to how hardware acceleration is implemented in IPsec.
PSP provides a unique combination of TLS and IPsec/VPN protocol capabilities. TLS suited Google in terms of per-connection security, but didn't suit Google due to lack of flexibility for hardware acceleration and lack of UDP support. IPsec was protocol-independent and well-supported for hardware acceleration, but did not support per-connection keys, was designed for only a small number of tunnels, and had issues with hardware acceleration scaling due to full encryption state being stored in NIC memory tables (for example, 10 GB of memory is required to process 5 million connections).
In the case of the PSP, information about the encryption state (keys, initialization vectors, sequence numbers, etc.) can be transmitted in the TX packet descriptor or in the form of a pointer to the host system memory without occupying the memory of the network card. According to Google, earlier, about 0.7% of computing power and a large amount of memory were spent on encrypting RPC traffic in the company's infrastructure. The introduction of the PSP through the use of hardware acceleration has reduced this figure to 0.2%.
Source: opennet.ru