For more than 20 years we have been viewing web pages using the HTTP protocol. Most users do not even think about what it is and how it works. Others know that somewhere under HTTP there is TLS, and under it is TCP, under which is IP, and so on. And still others - heretics - believe that TCP is the last century, they want something faster, more reliable and more secure. But in their attempts to invent a new ideal protocol, they have returned to the technologies of the 80s and are trying to build their brave new world on them.
A bit of history: HTTP/1.1
In 1997, HTTP Version 1.1 received its RFC. At that time, the protocol had been used by browsers for several years, and the new standard lasted another fifteen. The protocol worked only on the principle of request-response and was intended mainly for the transmission of textual information.
HTTP was designed to work on top of the TCP protocol, which guarantees reliable delivery of packets to the destination. The operation of TCP is based on establishing and maintaining a reliable connection between endpoints and dividing traffic into segments. Segments have their own serial number and checksum. If suddenly one of the segments does not arrive or arrives with an incorrect checksum, then the transmission will stop until the lost segment is restored.
In HTTP/1.0, the TCP connection was closed after each request. It was extremely wasteful, because. establishing a TCP connection (3-way-handshake) is not a fast process. HTTP/1.1 introduced a keep-alive mechanism that allows a single connection to be reused for multiple requests. However, because it can easily become a bottleneck, different implementations of HTTP/1.1 allow multiple TCP connections to be opened to the same host. For example, Chrome and recent versions of Firefox allow up to six connections.
Encryption was also supposed to be left to other protocols, and for this, the TLS protocol began to be used over TCP, which reliably protected data, but even more increased the time required to establish a connection. As a result, the handshake process began to look like this:
Cloudflare illustration
Thus, HTTP/1.1 had a number of problems:
- Slow connection establishment.
- The data is transmitted in text form, which means that the transfer of pictures, videos and other non-text information is inefficient.
- One TCP connection is used for one request, which means that the remaining requests must either find another connection for themselves, or wait until the current request releases it.
- Only pull model is supported. There is nothing in the standard about server-push.
- Headings are transmitted in text.
If server-push is somehow implemented using the WebSocket protocol, then the rest of the problems had to be dealt with more radically.
A bit of modernity: HTTP/2
In 2012, Google began work on the SPDY protocol (pronounced βspeedyβ). The protocol was intended to solve the main problems of HTTP / 1.1 and at the same time had to maintain backward compatibility. In 2015, the IETF Working Group submitted the HTTP/2 specification based on the SPDY protocol. Here are the differences in HTTP/2:
- binary serialization.
- Multiplexing multiple HTTP requests into one TCP connection.
- Server-push out of the box (no WebSocket).
The protocol was a big step forward. He is strong and does not require the creation of multiple TCP connections: all requests to one host are multiplexed into one. That is, in one connection there are several so-called streams, each of which has its own ID. The bonus is a boxed server-push.
However, multiplexing leads to another cornerstone problem. Imagine that we are asynchronously executing 5 requests to the same server. When using HTTP/2, all these requests will be executed within the same TCP connection, which means that if one of the segments of any request is lost or incorrect, the transmission of all requests and responses will stop until the lost segment is restored. Obviously, the worse the quality of the connection, the slower HTTP/2 works. , in conditions where lost packets make up 2% of all packets, HTTP / 1.1 in the browser performs better than HTTP / 2 due to the fact that it opens 6 connections, not one.
This problem is called "head-of-line blocking" and, unfortunately, it is not possible to solve it using TCP.
Illustration by Daniel Steinberg
As a result, the developers of the HTTP/2 standard did a great job and did almost everything that could be done at the application layer of the OSI model. It's time to go down to the transport layer and invent a new transport protocol.
We need a new protocol: UDP vs TCP
It quickly became clear that introducing a completely new transport layer protocol is an unsolvable task in today's realities. The fact is that hardware or middle-boxes (routers, firewalls, NAT servers ...) know about the transport layer, and teaching them something new is an extremely difficult task. In addition, support for transport protocols is hardwired into the kernel of operating systems, and kernels also change not very willingly.
And here one could give up and say βWe, of course, will invent a new HTTP / 3 with preference and courtesans, but it will take 10-15 years to be implemented (after about this time most of the pieces of iron will be replaced)β, but there is one more not the most The obvious option is to use the UDP protocol. Yes, yes, the same protocol by which we threw files on the local network in the late nineties and early zero. Almost all today's pieces of iron are able to work with it.
What are the advantages of UDP over TCP? First of all, we don't have a transport layer session that the hardware knows about. This allows us to define the session at the endpoints ourselves and resolve conflicts there. That is, we are not limited to one or several sessions (as in TCP), but we can create as many of them as we need. Secondly, data transmission over UDP is faster than over TCP. Thus, in theory, we can break today's speed ceiling achieved in HTTP/2.
However, UDP does not guarantee the reliability of data transmission. In fact, we simply send packets, hoping that they will be received at the other end. Have not received? Well, no luck ... This was enough to transmit video for adults, but for more serious things you need reliability, which means you have to wind something else over UDP.
As in the case of HTTP / 2, work on the creation of a new protocol began at Google in 2012, that is, approximately at the same time as the start of work on SPDY. In 2013, Jim Roskind presented to the general public , and already in 2015, the Internet Draft was submitted for standardization by the IETF. Already at that time, the protocol developed by Roskind at Google was very different from the standard, so the Google version was called gQUIC.
What is QUIC
First, as already mentioned, this is a wrapper over UDP. A QUIC connection rises over UDP, in which, by analogy with HTTP / 2, several streams can exist. These streams exist only on endpoints and are served independently. If a packet loss occurs in one stream, the others will not be affected in any way.
Illustration by Daniel Steinberg
Secondly, encryption is no longer implemented as a separate layer, but is included in the protocol. This allows you to establish a connection and exchange public keys in one handshake, and also allows you to use the tricky 0-RTT handshake mechanism and generally avoid handshake delays. In addition, it is now possible to encrypt individual data packets. This allows you not to wait for the completion of receiving data from the stream, but to decrypt the received packets independently. This mode of operation was generally not possible in TCP, since TLS and TCP worked independently of each other, and TLS could not know what pieces TCP would cut data into. And consequently, he could not prepare his segments so that they fit into TCP segments one to one and could be decrypted independently. All these improvements allow QUIC to reduce latency compared to TCP.
Thirdly, the concept of light streams allows you to decouple the connection from the client's IP address. This is important, for example, when a client switches from one Wi-Fi access point to another by changing its IP. In this case, when using TCP, there is a long process during which existing TCP connections time out and new connections are created from a new IP. In the case of QUIC, the client simply continues to send packets to the server from the new IP with the old stream ID. Because The stream ID is now unique and not reused, the server understands that the client has changed IP, forwards the lost packets and continues communication to the new address.
Fourth, QUIC is implemented at the application level, not the operating system. On the one hand, this allows you to make changes to the protocol faster, because to get an update, it's enough just to update the library, and not wait for a new version of the OS. On the other hand, this leads to a strong increase in processor consumption.
And finally, headlines. Header compression just refers to the moments that differ in QUIC and gQUIC. I see no reason to devote much time to this, I can only say that in the version submitted for standardization, header compression was made as similar as possible to header compression in HTTP / 2. Read more .
How much faster is it?
It's a difficult question. The fact is that until we have a standard, there is nothing special to measure. Perhaps the only statistics we have are those of Google, which has been using gQUIC since 2013 and into 2016. that about 90% of the traffic going to their servers from the Chrome browser now uses QUIC. In the same presentation, they report that pages load about 5% faster through gQUIC, and 30% less freezes in streaming video compared to TCP.
In 2017, a group of researchers led by Arash Molavi Kakhki published to study the performance of gQUIC compared to TCP.
The study revealed several weaknesses of gQUIC, such as instability to network packet mixing, greed (unfairness) to the bandwidth of the channel and slower transmission of small (up to 10 kb) objects. The latter, however, can be compensated by using 0-RTT. In all other cases studied, gQUIC showed an increase in speed compared to TCP. It's hard to talk about specific numbers here. Better read or .
Here it must be said that this data is about gQUIC, and they are irrelevant for the developed standard. What will happen for QUIC: so far the mystery is sealed, but there is hope that the weaknesses identified in gQUIC will be taken into account and corrected.
A bit of the future: what about HTTP/3?
And here everything is crystal clear: the API will not change in any way. Everything will remain exactly the same as it was in HTTP / 2. Well, if the API remains the same, the transition to HTTP / 3 will have to be decided by using the latest version of the library on the backend that supports QUIC transport. True, for quite a long time you will still have to keep fallback to older versions of HTTP, tk. The Internet is not yet ready for a full transition to UDP.
Who is already supporting
Here existing implementations of QUIC. Despite the lack of a standard, the list is not bad.
No browser currently supports QUIC in a pro release. Recently there was information that HTTP / 3 support was enabled in Chrome, but so far only in Canary.
Of the HTTP/3 backends, it only supports ΠΈ but still experimental. NGINX at the end of spring 2019 that started work on HTTP/3 support but haven't finished yet.
What are the problems
You and I live in the real world, where no big technology can go to the masses without meeting resistance, and QUIC is no exception.
Most importantly, you need to somehow explain to the browser that βhttps://β is no longer a fact that leads to the 443rd TCP port. There may not be TCP at all. For this, the Alt-Svc header is used. It allows you to tell the browser that this website is also available on such and such a protocol at such and such an address. In theory, this should work like clockwork, but in practice we will come across the fact that UDP can be, for example, banned on the firewall to avoid DDoS attacks.
But even if UDP is not prohibited, the client may be behind a NAT router that is configured to hold a TCP session by IP address, and since we are using UDP which has no hardware session, NAT will not hold the connection, and QUIC session .
All these problems are due to the fact that UDP was not previously used to transmit Internet content, and hardware manufacturers could not foresee that this would ever happen. In the same way, admins are not yet very clear on how to properly configure their networks for QUIC to work. This situation will gradually change, and, in any case, such changes will take less time than the introduction of a new transport layer protocol.
Also, as already described, QUIC greatly increases CPU usage. Daniel Stenberg growth on the processor up to three times.
When is HTTP/3 Coming?
Standard by May 2020, but given that the documents scheduled for July 2019 remain unfinished at the moment, we can say that the date is likely to be pushed back.
Well, Google has been using its implementation of gQUIC since 2013. If you look at the HTTP request that is sent to the Google search engine, you can see this:
Conclusions
QUIC now looks like a rather crude, but very promising technology. Considering that over the past 20 years, all the optimizations of transport layer protocols have been mainly related to TCP, QUIC, which in most cases wins in terms of performance, already looks very good now.
However, there are still unresolved problems that will have to be dealt with in the next few years. The process can be delayed due to the fact that hardware is involved that no one likes to update, but nevertheless, all problems seem to be completely solvable, and sooner or later we will all have HTTP / 3.
The future is not far off!
Source: habr.com