Introduction
Our company provides solutions for porting traditional desktop applications to the web. Our C++ compiler generates a combination of WebAssembly and JavaScript, which ensures both , and high performance.
As an example of its application, we decided to port a multiplayer game to the web and chose . Teeworlds is a multiplayer 2D retro game with a small but active player community (including myself!). It is small in terms of downloadable resources as well as CPU and GPU requirements — an ideal candidate.
Running in the browser, Teeworlds
We decided to use this project to experiment with general solutions for porting network code to the web. This is usually accomplished in the following ways:
- XMLHttpRequest/fetch, if the network part consists only of HTTP requests, or
- WebSockets.
Both solutions require hosting a server component on the server side, and neither allows using as a transport protocol. This is important for real-time applications, such as video conferencing software and games, because the delivery and order guarantees of the protocol can hinder low latency.
There is also a third way — using the browser network: .
supports both reliable and unreliable transmission (in the latter case, it attempts to use UDP as the transport protocol whenever possible), and can be used both with a remote server and between browsers. This means we can port the entire application to the browser, including the server component!
However, this comes with an additional difficulty: before two WebRTC peers can exchange data, they need to go through a relatively complex 'handshake' procedure for connection, which requires several third-party entities (a signaling server and one or more /).
In an ideal scenario, we would like to create a network API that internally uses WebRTC, but as close as possible to the UDP Sockets interface, which does not require a connection to be established.
This will allow us to leverage the advantages of WebRTC without needing to disclose the complex details of the application code (which we wanted to change as little as possible in our project).
Minimum WebRTC
WebRTC is a set of APIs available in browsers that enables peer-to-peer transmission of audio, video, and arbitrary data.
The connection between peers is established (even in the presence of NAT on one or both sides) through STUN and/or TURN servers via a mechanism called ICE. Peers exchange ICE information and channel parameters through the offer and answer of the SDP protocol.
Wow! That's a lot of acronyms at once. Let's briefly explain what these terms mean:
- — a protocol for bypassing NAT and obtaining a pair (IP, port) for directly exchanging data with the host. If it succeeds in its task, peers can exchange data with each other.
- also used to bypass NAT, but it does this by relaying data through a proxy visible to both peers. It adds latency and is more resource-intensive than STUN (because it is used throughout the communication session), but sometimes it is the only possible option.
- is used to select the best possible method for connecting two peers based on information obtained from direct peer connections as well as data received from any number of STUN and TURN servers.
- is a format for describing the parameters of a connection channel, such as ICE candidates, media codecs (for audio/video channels), etc. One peer sends an SDP Offer, while the other responds with an SDP Answer. After this, a channel is established.
To create such a connection, peers need to gather the information they received from STUN and TURN servers and exchange it with each other.
The problem is that they currently cannot exchange data directly, so there must be an out-of-band mechanism: a signaling server.
The signaling server can be very simple, as its only task is to redirect data between peers during the 'handshake' phase (as shown in the diagram below).
Simplified sequence of the WebRTC 'handshake'
Overview of the Teeworlds network model
The network architecture of Teeworlds is quite simple:
- Client and server components are two different programs.
- Clients join the game by connecting to one of several servers, each of which hosts only one game at a time.
- All data transfer in the game is done through the server.
- A special master server is used to collect a list of all public servers that are displayed in the game client.
By using WebRTC for data exchange, we can move the game's server component into the browser where the client is located. This provides us with a great opportunity…
To eliminate servers
The lack of server logic has a nice advantage: we can deploy the entire application as static content on Github Pages or on our own equipment behind Cloudflare, thereby ensuring fast load times and high uptime for free. Essentially, we might forget about them, and if we are lucky and the game becomes popular, we won't have to upgrade the infrastructure.
However, for the system to work, we still need to use external architecture:
- One or more STUN servers: we have a choice of several free options.
- At least one TURN server: there are no free options here, so we can either set up our own or pay for a service. Fortunately, most of the time, connections can be established through STUN servers (ensuring true P2P), but TURN is necessary as a backup.
- Signaling server: unlike the other two aspects, signaling is not standardized. What the signaling server will actually be responsible for depends somewhat on the application. In our case, a small amount of data needs to be exchanged before establishing a connection.
- Teeworlds master server: it is used by other servers to announce their existence and by clients to find public servers. Although it is not mandatory (clients can always connect to a known server manually), it would be good to have it so that players can join games with random people.
We decided to use Google's free STUN servers, and we deployed one TURN server ourselves.
For the last two points, we used :
- The master server for Teeworlds is implemented very simply: as a list of objects containing information (name, IP, map, mode, etc.) for each active server. Servers publish and update their own object, while clients retrieve the entire list and display it to the player. We also display the list on the homepage as HTML so players can simply click on the server to join the game.
- Signal handling is closely tied to our implementation of sockets, which is described in the next section.
The list of servers within the game and on the homepage
Socket Implementation
We want to create an API that is as close to Posix UDP Sockets as possible to minimize the necessary changes.
We also want to implement the bare minimum required for the simplest data exchange over the network.
For example, we do not need true routing: all peers exist in a single 'virtual LAN' connected to a specific instance of the Firebase database.
Therefore, we do not need unique IP addresses: to uniquely identify peers, it's sufficient to use unique Firebase key values (similar to domain names), and each peer locally assigns 'fake' IP addresses to each key that needs to be transformed. This completely eliminates the need for global IP address assignments, which is a non-trivial task.
Here is the minimal API that we need to implement:
// Create and destroy a socket
int socket();
int close(int fd);
// Bind a socket to a port, and publish it on Firebase
int bind(int fd, AddrInfo* addr);
// Send a packet. This lazily create a WebRTC connection to the
// peer when necessary
int sendto(int fd, uint8_t* buf, int len, const AddrInfo* addr);
// Receive the packets destined to this socket
int recvfrom(int fd, uint8_t* buf, int len, AddrInfo* addr);
// Be notified when new packets arrived
int recvCallback(Callback cb);
// Obtain a local ip address for this peer key
uint32_t resolve(client::String* key);
// Get the peer key for this ip
String* reverseResolve(uint32_t addr);
// Get the local peer key
String* local_key();
// Initialize the library with the given Firebase database and
// WebRTc connection options
void init(client::FirebaseConfig* fb, client::RTCConfiguration* ice);The API is simple and resembles the Posix Sockets API, but has several important differences: callback registration, local IP assignment, and 'lazy' connection.
Callback Registration
Even if the original program uses non-blocking I/O, the code needs to be refactored to run in a web browser.
The reason for this is that the event loop in the browser is hidden from the program (whether it’s JavaScript or WebAssembly).
In a native environment, we can write code like this
while(running) {
select(...); // wait for I/O events
while(true) {
int r = readfrom(...); // try to read
if (r < 0 && errno == EWOULDBLOCK) // no more data available
break;
...
}
...
}If the event loop is hidden from us, we need to turn it into something like this:
auto cb = []() { // this will be called when new data is available
while(true) {
int r = readfrom(...); // try to read
if (r < 0 && errno == EWOULDBLOCK) // no more data available
break;
...
}
...
};
recvCallback(cb); // register the callbackPurpose of local IPs
Node identifiers in our 'network' are not IP addresses, but Firebase keys (these are strings that look like: -LmEC50PYZLCiCP-vqde ).
This is convenient because we don’t need a mechanism to assign IPs and check their uniqueness (as well as their recycling after a client disconnects), but it is often necessary to identify peers by numerical value.
This is exactly what the functions resolve and reverseResolve: the application somehow obtains the string value of the key (through user input or through a master server), and can convert it into an IP address for internal use. The rest of the API also receives this value instead of the string for simplicity.
This is similar to a DNS lookup, except it is performed locally by the client.
That is, IP addresses cannot be shared among different clients, and if a global identifier is needed, it must be generated in another way.
Lazy connection
UDP does not require a connection, but as we have seen, before starting to transfer data between two peers, WebRTC requires a lengthy connection process.
If we want to provide the same level of abstraction, (sendto/recvfrom with arbitrary peers without prior connection), we must perform 'lazy' (deferred) connection within the API.
Here’s what happens during a standard data exchange between a 'server' and a 'client' when using UDP, and what our library needs to perform:
- The server calls
bind(), to inform the operating system that it wants to receive packets on a specified port.
Instead, we will publish the open port in Firebase under the server key and listen for events in its subtree.
- The server calls
recvfrom(), receiving packets sent from any host to this port.
In our case, we need to check the incoming packet queue sent to this port.
Each port has its own queue, and we add the source and destination ports to the beginning of WebRTC datagrams so we know which queue to redirect a new packet to upon arrival.
The call is non-blocking, so if there are no packets, we simply return -1 and set errno=EWOULDBLOCK.
- The client obtains the server's IP and port by some external means and calls
sendto(). An internal call is also made at this point, so the subsequentbind()request proceeds.recvfrom()will receive a response without explicit binding execution.
In our case, the client externally obtains a string key and uses the function resolve() to obtain the IP address.
At this stage, we initiate the WebRTC handshake if the two peers are not yet connected to each other. Connections to different ports of a single peer use the same WebRTC DataChannel.
We also perform an indirect bind(), so that the server can restore the connection in the next sendto() in case it was closed for any reason.
The server is notified of the client's connection when the client records its SDP offer under the server port information in Firebase, and the server responds there with its own response.
The diagram below shows an example of message flow for the socket scheme and the transmission of the first message from the client to the server:
The complete connection stage scheme between the client and the server.
Conclusion
If you have read this far, you are probably interested in seeing the theory in action. You can play the game on , give it a try!
A friendly match between colleagues.
The code for the networking library is freely available at . Join the conversation on our channel at !
Source: habr.com
