cloud gaming one of the top technologies to keep an eye on right now. In 6 years, this market should grow 10 times - from $45 million in 2018 to $450 million in 2024. Tech giants have already rushed to explore the niche: Google and Nvidia have launched beta versions of their cloud gaming services, Microsoft, EA, Ubisoft, Amazon and Verizon are preparing to enter the scene.
For gamers, this means that very soon they will be able to completely stop spending money on hardware upgrades and run powerful games on weak computers. Is it beneficial to other participants in the ecosystem? We tell why cloud gaming will increase their earnings and how we created a technology that makes it easy to enter a promising market.
Publishers, developers, TV manufacturers and telecom operators: why they all need cloud gaming
Game publishers and developers are interested in getting their product to the largest number of players as quickly as possible. Now, according to our data, 70% of potential buyers do not reach the game - they do not wait for the client to download and the installation file weighing tens of gigabytes. However, 60% of users , in principle, cannot run powerful games (AAA-level) on their computers in acceptable quality. Cloud gaming can solve this problem - not only will it not reduce the earnings of publishers and developers, but it will help them grow a paying audience.
Manufacturers of TVs and set-top boxes are now also looking towards cloud gaming. In the era of smart homes and voice assistants, they have to compete more and more for the attention of the user, and gaming functionality is the main way to attract this attention. With built-in cloud gaming, their client will be able to run modern games directly on the TV, paying for the service to the manufacturer.
Another potentially active participant in the ecosystem is telecom operators. Their way to increase revenue is to provide additional services. Gaming is just one of these services that operators are already actively implementing. Rostelecom launched the Gaming tariff, Akado sells access to our Playkey service. It's not just about broadband internet operators. Mobile operators, due to the active spread of 5G, will also be able to make cloud gaming their additional source of income.
Despite bright prospects, it is not so easy to enter the market. All existing services, including the products of technology giants, have not yet been able to fully overcome the problem of the "last mile". This means that due to the imperfection of the network directly in the house or apartment, the Internet speed of the user is not enough for the correct operation of cloud gaming.
Take a look at how the WiFi signal attenuates, spreading from the router around the apartment
Players that have been on the market for a long time and have powerful resources are gradually moving towards solving this problem. But launching your cloud gaming from scratch in 2019 is spending a lot of money, time, and possibly never building an effective solution. To help all participants in the ecosystem to develop in a rapidly growing market, we have developed a technology that allows you to quickly and cost-effectively launch your cloud gaming service.
How we made a technology that will make it easy to launch your own cloud gaming service
Playkey started developing its cloud gaming technology back in 2012. In 2014, a commercial launch took place, and by 2016, 2,5 million players used the service at least once. Throughout development, we have seen interest not only from gamers, but also from set-top box manufacturers and carriers. With NetByNet and ER-Telecom, we even launched several pilot projects. In 2018, we decided that our product might have a B2B future.
It is problematic to develop for each company its own version of cloud gaming integration, as we did in pilot projects. Each such implementation took from three months to six months. Why? Everyone has different equipment and operating systems: someone needs cloud gaming on an Android set-top box, and someone needs it like an iFrame in the web interface of their personal account for streaming to computers. In addition, everyone has a different design, billing (a separate wonderful world!) And other features. It became clear that it was necessary either to increase the development team tenfold, or to make the most universal boxed B2B solution.
In March 2019 we launched . This is software that companies can install on their servers and get a working cloud gaming service. How will it look to the user? He will see a button on the familiar site that allows you to launch the game in the cloud. When pressed, the game will start on the company's server, and the user will see the stream and be able to play remotely. Here's what it might look like in popular digital game distribution services.
Active struggle for quality. And passive too.
How Remote Click copes with numerous technical barriers, we will now tell. Cloud gaming of the first wave (for example, OnLive) ruined the poor quality of the Internet among users. Then, in 2010, the average Internet connection speed in the United States only 4,7 Mbps. By 2017, it has already grown to 18,7 Mbps, and soon 5G will appear everywhere and a new era will come. However, despite the fact that in general the infrastructure is ready for cloud gaming, the already mentioned βlast mileβ problem remains.
One side of it, which we call objective: the user really has problems with the network. For example, the operator does not allocate the declared maximum speed. Or using WiFi 2,4 GHz, noisy microwave and wireless mouse.
The other side, which we call subjective: the user does not even suspect that he has problems with the network (does not know that he does not know)! At best, he is sure that since the operator sells him a 100 Mbps tariff, he has 100 Mbps Internet. At worst, he has no idea what a router is, and the Internet is divided into blue and color. A real case from kasdev.
Blue and color internet.
But both parts of the last mile problem are solvable. In Remote Click, we use active and passive mechanisms for this. Below is a detailed account of how they deal with obstacles.
Active mechanisms
1. Effective noise-immune coding of transmitted data aka redundancy (FEC - Forward Error Correction)
When transmitting video data from the server to the client, error-correcting coding is used. With its help, we restore the original data in case of partial loss due to network problems. What makes our solution effective?
- Speed. Encoding and decoding are very fast. Even on "weak" computers, the operation takes no more than 1 ms for 0,5 MB of data. Thus, encoding and decoding add almost no delay when playing through the cloud. The importance is difficult to overestimate.
- Maximum data recovery potential. Namely, the ratio of the excess volume of data and the volume that is potentially recoverable. In our case, the ratio = 1. Let's say you need to transfer 1 MB of video. If we add 300 KB of additional data during encoding (this is called redundancy), then during the decoding process, to restore 1 original megabyte, we need only any 1 MB of the total 1,3 MB that the server sent. In other words, we can lose 300 KB and still recover the original data. As you can see, 300 / 300 = 1. This is the maximum possible efficiency.
- Flexibility in setting up an additional amount of data when encoding. We can set up a separate redundancy level for each video frame that needs to be sent over the network. For example, when we notice network problems, we can increase or decrease the level of redundancy.
We play Doom on Core i3, 4 GB RAM, MSI GeForce GTX 750 via Playkey.
2. Sending data
An alternative way to deal with losses is to request data again. For example, if the server and user are located in Moscow, then the transmission delay will not exceed 5 ms. With this value, the client application will have time to request and receive the lost part of the data from the server without the user noticing. Our system decides for itself when to apply redundancy and when to resend.
3. Individual setting of data transfer
To choose the best way to deal with losses, our algorithm analyzes the user's network connection and configures the data transmission system individually for each case.
He looks:
- connection type (Ethernet, WiFi, 3G, etc.);
- the used WiFi frequency band is 2,4 GHz or 5 GHz;
- WiFi signal strength.
If you rank connections by loss and delay, then the most reliable of all, of course, is the wire. Over Ethernet, losses are rare, and delays in the "last mile" are extremely unlikely. Then comes WiFi 5 GHz and only then WiFi 2,4 GHz. Mobile connections are generally rubbish, we are waiting for 5G.
When using WiFi, the system automatically configures the user's adapter, putting it in the most appropriate mode for use in the cloud (for example, turning off power saving).
4. Individual encoding setting
Video streaming exists thanks to codecs - programs for compressing and recovering video data. In uncompressed form, one second of video can easily exceed a hundred megabytes, and the codec reduces this value by an order of magnitude. We are armed with H264 and H265 codecs.
H264 is the most popular. Hardware work with it is supported by all the major manufacturers of video cards for over a decade. H265 is a cheeky young successor. They started supporting it in hardware five years ago. Encoding and decoding in H265 requires more resources, but the quality of the compressed frame is noticeably higher than in H264. And no volume increase!
Which codec to choose and what encoding parameters to set for a particular user, based on his hardware? A non-trivial task that we solve automatically. A smart system analyzes the capabilities of the equipment, sets the optimal encoder parameters and selects a decoder on the client side.
5. Loss compensation
We didn't want to admit it, but even we're not perfect. Some data lost in the bowels of the network cannot be recovered and we do not have time to send them. But in this case, there is a way out.
For example, adjusting the bitrate. Our algorithm constantly controls the amount of data sent from the server to the client. It fixes every shortage and even predicts possible future losses. Its task is to notice in time, and ideally predict when the losses reach a critical value and begin to create interference on the screen that is noticeable to the user. And adjust at this point the amount of data being sent (bitrate).
We also use the invalidation of unassembled frames and the mechanism of reference frames in the video stream. Both tools reduce the number of noticeable artifacts. That is, even with serious violations in data transmission, the image on the screen remains acceptable, and the game is playable.
6. Distributed sending
Time-distributed data sending also improves the quality of streaming. How exactly to distribute depends on specific indicators in the network, for example, the presence of losses, ping and other factors. Our algorithm analyzes them and chooses the best option. Sometimes the distribution in the interval of several milliseconds reduces the losses by several times.
7. Latency Reduction
One of the key characteristics when playing through the cloud is latency. The smaller it is, the more comfortable it is to play. The delay can be conditionally divided into two parts:
- network or data transfer delay;
- system delay (removal of control on the client side, image capture on the server, image encoding, the above mechanisms for adapting data to send, data collection on the client, image decoding and its rendering).
The network depends on the infrastructure and it is problematic to deal with it. If mice have gnawed the wire, dancing with a tambourine will not help. But the system delay can be reduced significantly and the quality of cloud gaming for the player will change dramatically. In addition to the already mentioned error-correcting coding and personalized settings, we use two more mechanisms.
- Fast data acquisition from control devices (keyboard, mouse) on the client side. Even on weak computers, 1-2 ms is enough for this.
- Drawing the system cursor on the client. The mouse pointer is processed not on a remote server, but in the Playkey client on the user's computer, that is, without the slightest delay. Yes, this does not affect the actual control in the game, but the main thing here is human perception.
Cursor drawing without delay in Playkey using Apex Legends as an example
Using our technology, with a network delay of 0 ms and a video stream of 60 FPS, the delay of the entire system does not exceed 35 ms.
Passive mechanisms
In our experience, many users have little idea how their devices connect to the internet. In interviews with the players, it turned out that some do not know what a router is. And that's okay! You do not need to know how an internal combustion engine works in order to drive a car. You should not require the knowledge of a system administrator from the user so that he can play.
However, it is still important to convey some technical points so that the player can independently remove the barriers on his side. And we help him.
1. 5GHz WiFi support indication
We wrote above that we see the Wi-Fi standard - 5 GHz or 2,4 GHz. And we also know if the network adapter of the user device supports the very possibility of operating at 5 GHz. And if yes, then we recommend using this range. We canβt change the frequency on our own yet, because we donβt see the characteristics of the router.
2. WiFi signal strength indication
For some users, the WiFi signal may be weak, even if the Internet is working well and seems to be at an acceptable speed. The problem will be revealed precisely with cloud gaming, which puts the network to real tests.
Signal strength is affected by obstacles such as walls and interference from other devices. The same microwaves emit a lot. As a result, there are losses that are imperceptible when working on the Internet, but critical for playing through the cloud. In such cases, we warn the user about interference, suggest moving closer to the router and turning off βnoisyβ devices.
3. Indication of traffic consumers
Even if the network is fine, other applications may consume too much traffic. For example, if a video on Youtube is launched in parallel with the game in the cloud or torrents are downloaded. Our application detects thieves and warns the player about them.
Fear from the past - debunking myths about cloudgaming
Cloud gaming as a fundamentally new way of consuming game content has been trying to break into the market for almost a decade. And as with any innovation, their history is a series of small victories and resounding defeats. It is not surprising that over the years, cloud gaming has become overgrown with myths and prejudices. At the dawn of the development of technology, they were justified, but today they are completely groundless.
Myth 1. The picture in the cloud is worse than in the original - as if you are playing on YouTube
Today, in a technically advanced cloud solution, the pictures of the original and the clouds are almost identical - you canβt find any difference with the naked eye. Individual setting of the encoder for the player's equipment and a set of mechanisms for dealing with losses close this issue. On a high-quality network, there is no blurring of frames or graphic artifacts. We even take permission into account. There is no point in streaming in 1080p if the player is using 720p.
Below are two Apex Legends videos from our channel. In one case, this is a gameplay recording when playing on a PC, in the other, through the Playkey.
Apex Legends on PC
Apex Legends on Playkey
Myth 2. Unstable quality
The network state is indeed inconsistent, but this issue has been resolved. We dynamically change the encoder settings according to the quality of the user's network. And we maintain a constantly acceptable FPS level with special image capture techniques.
How it works? The game has a 3D engine that builds a 3D world. But the user is shown a flat image. For him to see it, a memory picture is created for each frame - a kind of photograph of how this 3D world is seen from a certain point. This encoded picture is stored in the video memory buffer. We capture it from video memory and transfer it to the encoder, which already decrypts it. And so with each frame, one after another.
Our technology allows you to capture and decode an image in one stream, which increases FPS. And if these processes are carried out in parallel (a rather popular solution in the cloud gaming market), then the encoder will constantly access capture, pick up new frames with a delay and, accordingly, transmit them with a delay.
The video at the top of the screen was captured and decoded in one stream.
Myth 3. Because of the lags in the management, I will be a βcancerβ in multiplayer
The delay in control is normally a few milliseconds. And usually it is invisible to the end user. But sometimes you can see a tiny discrepancy between the movement of the mouse and the movement of the cursor. It does not affect anything, but creates a negative impression. The rendering of the cursor directly on the user's device described above eliminates this shortcoming. Otherwise, the total system delay of 30-35 ms is so low that neither the player nor his match rivals notice anything. The outcome of the battle is decided only by skills. The proof is below.
Streamer bends over Playkey
What's next
Cloud gaming is already a reality. Playkey, PlayStation Now, Shadow are working services with their own audience and market place. And like many young markets, cloud gaming will skyrocket in the coming years.
One of the scenarios that seems most likely to us is the emergence of their own services from game publishers and telecom operators. Someone will develop their own, someone will use ready-made boxed solutions, like RemoteClick.net. The more players in the market, the faster the cloud-based way of consuming game content will become mainstream.
Source: habr.com