For the past two weeks, I've been working on a networking engine for my game. Before this, I knew nothing about networking technologies in games, so I read numerous articles and conducted many experiments to understand all the concepts and be able to write my own networking engine.
In this guide, I would like to share various concepts you need to study before writing your own game engine, as well as the best resources and articles for learning them.
In general, there are two main types of networking architectures: peer-to-peer and client-server. In a peer-to-peer (p2p) architecture, data is transmitted between any pairs of connected players, whereas in a client-server architecture, data is transmitted only between players and the server.
Although peer-to-peer architecture is still used in some games, the standard is client-server: it is easier to implement, requires less bandwidth, and makes it easier to protect against cheating. Therefore, in this guide, we will focus on client-server architecture.
Specifically, we are most interested in authoritative servers: in such systems, the server is always right. For example, if a player thinks they are at coordinates (10, 5), and the server tells them they are at (5, 3), the client must replace its position with the one the server provides, not the other way around. Using authoritative servers simplifies cheat detection.
In game networking systems, there are three main components:
- Transport Protocol: how data is transmitted between clients and the server.
- Application Protocol: what is transmitted from clients to the server and from the server to clients, and in what format.
- Application Logic: how the transmitted data is used to update the state of clients and the server.
It is very important to understand the role of each part and the difficulties associated with them.
Transport Protocol
The first step is to choose a protocol for transporting data between the server and clients. There are two Internet protocols for this: and . However, you can also create your own transport protocol based on one of them or use a library that utilizes them.
Comparison of TCP and UDP
Both TCP and UDP are based on IP allows packets to be sent from the source to the recipient, but it does not guarantee that the sent packet will eventually reach the recipient, that it will get to them at least once, or that the sequence of packets will arrive in the correct order. Furthermore, a packet can only contain a limited amount of data, defined by its size. .
UDP is merely a thin layer on top of IP. Consequently, it shares the same limitations. In contrast, TCP has many features. It provides a reliable, ordered connection between two nodes with error checking. As a result, TCP is very convenient and is used in a multitude of other protocols, such as , and . But all these features come at a cost: .
To understand why these features may cause latency, one must grasp how TCP works. When the sender node transmits a packet to the recipient node, it expects to receive an acknowledgment (ACK). If it does not receive one after a certain period (either because the packet or acknowledgment was lost, or for some other reason), it resends the packet. Furthermore, TCP guarantees the receipt of packets in the correct order, so until the lost packet is received, all subsequent packets cannot be processed, even if they have already been received by the recipient node.
But as you might understand, latency is very important in multiplayer games, especially in active genres like FPS. That’s why many games use UDP with their own protocol.
A custom protocol based on UDP can be more effective than TCP for various reasons. For instance, it can mark certain packets as reliable and others as unreliable. Therefore, it does not concern itself with whether the unreliable packet reached the recipient. It can also handle multiple data streams so that a lost packet in one stream does not slow down the others. For example, there could be one stream for player input and another for chat messages. If a chat message, which is not urgent data, is lost, it will not delay the processing of input, which is critical. Alternatively, a custom protocol can implement reliability differently than TCP to be more efficient in gaming conditions.
So, if TCP is that bad, should we create our own transport protocol based on UDP?
It's a bit more complicated. While TCP is almost suboptimal for game networking systems, it can work quite well for your specific game and save you valuable time. For instance, latency might not be an issue for a turn-based game or a game that can only be played over LAN, where latency and packet loss are much lower than on the Internet.
Many successful games, including World of Warcraft, Minecraft, and Terraria, use TCP. However, most FPS games employ their own protocols based on UDP, so we'll discuss them in more detail below.
If you decide to use TCP, make sure to disable , as it buffers packets before sending, thereby increasing latency.
To learn more about the differences between UDP and TCP in the context of multiplayer games, you can read Glenn Fiedler's article .
Custom Protocol
So, you want to create your own transport protocol but don’t know where to start? You're in luck because Glenn Fiedler has written two fantastic articles about it. In them, you'll find plenty of insightful ideas.
The first article, from 2008 is simpler than the second one, from 2016. I recommend starting with the older one.
Keep in mind that Glenn Fiedler is a strong proponent of using a custom protocol based on UDP. After reading his articles, you will likely adopt his view that TCP has serious drawbacks in video games and want to implement your own protocol.
But if you're new to networking, do yourself a favor and use TCP or a library. Successfully implementing your own transport protocol requires learning a lot beforehand.
Networking Libraries
If you need something more efficient than TCP but don't want to bother with implementing your own protocol and getting into too many details, you can use a networking library. There are many available:
- by Glenn Fiedler
- , which is no longer maintained, but its fork seems to still be active.
- is a library designed for multiplayer FPS
- from Valve
I haven't tried them all, but I prefer ENet because it is easy to use and reliable. Additionally, it has clear documentation and tutorials for beginners.
Transport Protocol: Conclusion
In summary, there are two main transport protocols: TCP and UDP. TCP has many useful features: reliability, ordered packet delivery, error detection. UDP lacks all of this, but TCP inherently has increased latency, which is unacceptable for some games. Thus, to ensure low latency, one can create a custom protocol based on UDP or use a library that implements a transport protocol on UDP and is adapted for multiplayer video games.
The choice between TCP, UDP, and a library depends on several factors. Firstly, the needs of the game: does it require low latency? Secondly, the application protocol requirements: does it need a reliable protocol? As we will see in the next part, it is possible to create an application protocol for which an unreliable protocol may be suitable. Finally, one must also consider the experience level of the network engine developer.
I have two pieces of advice:
- Maximally abstract the transport protocol from the rest of the application to make it easy to replace without rewriting all the code.
- Do not engage in premature optimization. If you are not a networking specialist and are unsure whether you need your own transport protocol based on UDP, you can start with TCP or libraries that provide reliability, and then test and measure the performance. If problems arise and you are confident that the cause lies in the transport protocol, then it might be time to create your own transport protocol.
In conclusion of this part, I recommend reading by Brian Hook, which covers many of the topics discussed here.
Application Protocol
Now that we can exchange data between clients and the server, we need to determine what data to transmit and in what format.
The classic scheme is that clients send input or actions to the server, and the server sends the current game state back to the clients.
The server sends a filtered state containing only the entities that are near the player. It does this for three reasons. First, the full state can be too large to transmit at high frequency. Second, clients are mostly interested in visual and audio data, as most of the game logic is simulated on the game server. Third, in some games, players should not know certain data, such as the position of an opponent on the other side of the map, as otherwise they could sniff packets and know exactly where to move to avoid being killed.
Serialization
The first step will be converting the data we want to send (input or game state) into a suitable format for transmission. This process is called .
One might immediately think of using a human-readable format like JSON or XML. However, this would be completely inefficient and would waste a large part of the bandwidth.
Instead, it is recommended to use a binary format, which is much more compact. This means that packets will contain only a few bytes. One must consider the problem of , which may differ on different computers.
You can use libraries for data serialization, such as:
- from Google
- from Sandstorm
- by Sean Grant and Randolph Voorhies
Just make sure that the library creates portable archives and takes care of byte order.
An alternative solution could be a custom implementation, which is not particularly difficult, especially if you use a data-oriented approach in your code. Moreover, it will allow you to perform optimizations that are not always possible when using a library.
Glenn Fidler wrote two articles about serialization: and .
Compression
The amount of data transmitted between clients and the server is limited by the bandwidth of the channel. Data compression will allow more data to be transmitted in each snapshot, increase the refresh rate, or simply reduce the bandwidth requirements.
Bit packing
The first technique is bit packing. It involves using exactly the number of bits necessary to describe the required value. For example, if you have an enumeration that can have 16 different values, you can use only 4 bits instead of a full byte (8 bits).
Glenn Fidler explains how to implement this in the second part of the article. .
Bit packing works particularly well with quantization, which will be the topic of the next section.
Quantization
is a lossy compression technique that uses only a subset of possible values to encode a magnitude. The simplest way to implement quantization is by rounding floating-point numbers.
Glenn Fidler (again!) shows how to apply quantization in practice in his article. .
Compression Algorithms
The next technique will be lossless compression algorithms.
Here are, in my opinion, three of the most interesting algorithms to know:
- with a precomputed code, which is extremely fast and can yield good results. It has been used for compressing packets in the Quake3 networking engine.
- is a general-purpose compression algorithm that never increases data size. As you can see, it has been used in many applications. For state updates, it may be redundant, but it can be useful when you need to send asset files, long texts, or textures from the server to clients.
- is probably the simplest compression algorithm, but it is very effective for certain types of data and can be used as a preprocessing step before zlib. It is particularly suited for compressing textures made up of tiles or voxels, where many neighboring elements are repeated.
Delta Compression
The last compression technique is delta compression. This involves transmitting only the differences between the current game state and the last state received by the client.
It was first applied in the Quake3 networking engine. Here are two articles explaining how to use it:
- by Brian Hook
- by Fabien Sanglar [ articles on Habr, see the 'Networking Model' section]
Glenn Fidler also used it in the second part of his article .
Encryption
Additionally, you may need to encrypt the transmission of information between clients and servers. There are several reasons for this:
- privacy: messages can only be read by the recipient, and no other party performing network sniffing will be able to read them.
- authentication: a person wishing to perform the role of a player must know their key.
- cheating prevention: malicious players will find it much harder to create their own packets for cheating, as they will have to reproduce the encryption scheme and find the key (which changes with each connection).
I strongly recommend using a library for this. I suggest using , because it is particularly simple and has excellent tutorials. The tutorial on , which allows generating new keys with each new connection, is especially interesting.
Application Protocol: Conclusion
With this, we will conclude the application protocol. I believe that compression is completely optional, and the decision to use it depends only on the game and the required bandwidth. Encryption, in my opinion, is essential, but it can be omitted in the first prototype.
Application Logic
We are now able to update the state on the client, but we may encounter latency issues. After a player makes an input, they must wait for the game state update from the server to see what impact they had on the world.
Moreover, between two state updates, the world remains completely static. If the state update frequency is low, movements will be very jerky.
There are several techniques to reduce the impact of this issue, and I will discuss them in the next section.
Latency Smoothing Techniques
All the techniques described in this section are covered in detail in the series by Gabriel Gambetta. I highly recommend reading this excellent series of articles. It also includes an interactive demo that allows you to see how these techniques work in practice.
The first technique involves applying the input result directly, without waiting for a response from the server. This is called client-side predictionHowever, when a client receives an update from the server, they must ensure that their prediction was correct. If not, they simply need to change their state according to what they received from the server, because the server is authoritative. This technique was first used in Quake. More about it can be read in the article by Fabien Sanglar [ on Habr].
The second set of techniques is used to smooth the movement of other entities between two state updates. There are two ways to solve this problem: interpolation and extrapolation. In the case of interpolation, the last two states are taken and the transition from one to the other is shown. Its drawback is that it causes a slight delay because the client always sees what happened in the past. Extrapolation involves predicting where entities should currently be based on the last state received by the client. Its drawback is that if an entity completely changes its direction of movement, there will be a significant discrepancy between the prediction and the actual position.
The last and most advanced technique, useful only in FPS games, is lag compensation. When using lag compensation, the server takes into account client delays when they shoot at a target. For example, if a player scores a headshot on their screen, but in reality, their target was in a different position due to lag, it would be unfair to deny the player a kill due to this delay. Therefore, the server rewinds time to the moment when the player shot to simulate what the player saw on their screen and check the collision between their shot and the target.
Glenn Fidler (as always!) wrote an article in 2004 , in which he laid the foundation for synchronizing the physics simulation between server and client. In 2014, he wrote a new series of articles , in which he described other techniques for synchronizing physics simulation.
There are also two articles on the Valve company wiki, and , which discuss lag compensation.
Preventing Cheating
There are two main techniques for preventing cheating.
First: complicating the sending of malicious packets by cheaters. As mentioned above, a good way to implement this is through encryption.
Second: an authoritative server should only receive commands/input/actions. The client should not have the ability to change the state on the server, except by sending input. Then the server must check the input for validity every time it receives it before applying it.
Application Logic: Conclusion
I recommend implementing a way to simulate high delays and low update frequencies in order to test your game's behavior under poor conditions, even when the client and server are running on the same computer. This will greatly simplify the implementation of delay smoothing techniques.
Other Useful Resources
If you want to explore other resources dedicated to network models, you can find them here:
- — it’s worth reading his entire blog; there are many great articles. all articles on network technologies are compiled.
- by M. Fatih MAR — this is a detailed list of articles and videos about networking engines in video games.
- In also has many useful links.
Source: habr.com
