It is very convenient and effective to organize various broadcasts for a large audience because segments can be loaded once onto the CDN.<\/li>\n<\/ul>\n\nDespite the simplicity of the engine, things are not as smooth as they seem. The main problem is that third-party player developers have deviated from Apple's recommendations, for example, regarding supported audio formats. In particular, many developers have started adding support for popular audio streams: mpeg2 video, mpeg2 audio, etc. As a result, it was necessary to create different playlist formats for different players.<\/p>\n
However, one of the biggest problems with the HLS engine is the high latency in data transmission.<\/p>\n
The roots of the \"lag\"<\/h2>\n \nThe main reason for high latency in HLS lies in the fact that programmers created the engine to achieve the highest quality image. Therefore, the parameters of the frame interval and the size of the playback buffer are simply not suitable for live video broadcasts. As a result, a fairly high latency in video transmission occurs, which can be 5-7 seconds.<\/p>\n
On the one hand, this is a small amount, for example, for those who watch a movie from a video hosting server. But for surveillance systems, the latency in video transmission can be very significant.<\/p>\n
If you are watching an office where employees take breaks from their monitors every hour, then a 5-second delay is of no concern. But people have started to complain that, for instance, during a football match broadcast, the chat has already said GOAL, but it's not yet visible in the video :). We already have several user cases where Ivideon must practically replace Skype.<\/p>\n
Can latency in HLS be overcome? The answer to this question sounds like an experienced rat exterminator's speech at a lecture for beginner pest controllers: 'You can't exterminate rats, but you can keep their population to a reasonable minimum.' So it is with latency in HLS; it won't be possible to eliminate it completely, but there are solutions on the market that can significantly reduce latency.<\/p>\n
Small segments<\/h2>\n \nAnother drawback of the engine is the use of small-sized files for data transmission. One might wonder, what's wrong with that?<\/p>\n
Anyone who has tried to copy a large number of small files from one storage device to another has surely noticed that the write speed for such a batch is much lower than that of a single large file of the same volume. Additionally, the intensity of requests to the hard drive increases significantly, which negatively affects the overall performance of the computer. Therefore, transmitting video data in small 10-second fragments also contributes to increased latency in the engine.<\/p>\n
Let\u2019s briefly summarize all the pros and cons of HLS technology.<\/p>\n
Advantages of HLS:<\/b><\/p>\n\nCompatibility with any devices. You can watch videos on any modern device, whether it's a smartphone, tablet, laptop, or desktop PC. The main requirement is that the web browser must be a modern version and compatible with HTML5 and Media Source Extensions.<\/li>\n Excellent image quality. The adaptive bitrate streaming function allows for dynamically changing the quality of the video stream based on the bandwidth of the internet connection, while the algorithm aims to maintain the highest quality.<\/li>\n No need for complex user equipment setup.<\/li>\n<\/ol>\n\nDisadvantages:<\/b><\/p>\n\nLimited support for running the engine on some devices.<\/li>\n High image transmission latency.<\/li>\n Significant increase in overhead and complexity in optimization due to the use of small-sized files. Due to the peculiarities of the container, we can never achieve a latency less than the size of the segment.<\/li>\n<\/ol>\n\nThe disadvantages of HLS outweighed its advantages for us and forced us to seek alternative options.<\/p>\n
What is WebRTC<\/h2>\n \nc<\/noindex>)<\/p>\nThe WebRTC platform was developed by Google in 2011 to transmit streaming video and audio data between browsers and mobile applications with minimal latency. It uses the standard UDP protocol and special flow control algorithms. Today, it is an open-source project that is actively supported and developed by Google.<\/p>\n
WebRTC is a set of technologies for peer-to-peer video and audio transmission. This means, for example, that users' browsers can send data directly to each other using WebRTC, without relying on remote servers for data storage and processing. All information is also processed by the browsers and mobile applications of the end users.<\/p>\n
The convenience and extensive capabilities of this technology have been rightly appreciated by developers of all popular browsers. Today, WebRTC support is implemented in Mozilla Firefox, Opera, Google Chrome (and all Chromium-based browsers), as well as in mobile applications on Android and iOS.<\/p>\n
Despite its undeniable advantages, WebRTC has several significant drawbacks. <\/p>\n
Challenges of Selection<\/h2>\n \nThe WebRTC technology is much more complex in terms of network interactions due to its P2P nature. It is difficult to debug and test, and it can behave unpredictably. Additionally, we need to overcome NAT and firewalls, and ensure functionality in networks where UDP is blocked.<\/p>\n
Google's WebRTC implementation is very challenging to use. There is even an entire company that provides services for assembling SDKs. Plus, integrating Google's implementation with our system has been very difficult without re-encoding all video.<\/p>\n
However, we have long wanted to provide users with the ability to work with a full-fledged 'live' video stream and minimize the latency of the image on the screen compared to the actual events. Additionally, we wanted to make the use of PTZ cameras more comfortable, where delays are critically important.<\/p>\n
Given that other implementations for combating lag currently have limited functionality and perform noticeably worse, we decided to use WebRTC. <\/p>\n
What We Did<\/h2>\n \n
For WebRTC to function correctly, it is essential first to conduct a technological modernization of the stack for web video. That\u2019s exactly what we did.<\/p>\n
First, we implemented a signaling protocol server for WebRTC over WebSocket and deployed a WebRTC peer server in the cloud based on the SDK from webrtc.org. Its purpose is to distribute video streams to client WebRTC peers in H.264 + Opus\/G.711 format without video transcoding.<\/p>\n
We chose WebSocket as the signaling protocol because it has robust support across all popular web browsers. This significantly reduces not only development overhead but also eliminates the time and resources spent on repeated TCP and TLS handshakes compared to AJAX.<\/p>\n
The issue is that by default, WebRTC does not provide a signaling protocol necessary for correctly establishing, maintaining, and terminating real-time video connections between the source and client applications.<\/p>\n
To implement the signaling technology on our own, we needed to develop our signaling server supporting multiple web protocols (WebSocket, WebRTC) along with secure session management and real-time notifications, video management, and many other parameters.<\/p>\n
We overcame P2P limitations by reducing latency not through P2P, but through UDP and flow control aimed at minimizing delays. This is also embedded in WebRTC, as the primary use case is P2P conversations through the browser.<\/p>\n
In the mobile client, we implemented a player using the SDK from webrtc.org, as it correctly handles stream management, includes all known Forward Error Correction (FEC) schemes, and has an appropriate packet retransmission mechanism for all browsers. Additionally, it is important that the SDK from webrtc.org is actively developed by Google.<\/p>\n
What results did we achieve from implementing WebRTC?<\/h2>\n \n
![\"WebRTC](\"\/wp-content\/uploads\/2019\/04\/d185b546f5fc4c15021481c0cc2fedbd.jpeg\")