From the first days of working on a cloud video surveillance system, we faced a problem, without solving which it was possible to put an end to Ivideon - it was our Everest, climbing which took a lot of strength, but now we have finally stuck an ice ax into the top of the cross-platform rebus.
The system for transmitting audio and video over the Internet should not depend on the equipment, Web clients and the standards they support, and also work correctly in the presence of Network Address Translators and firewalls. A cloud video surveillance user wants to access the service, even if he uses analog cameras, and prefers to watch live video broadcast on the most modern device.
It is very significant that the user wants to watch video with minimal delay. Pretty much the only way to show low latency video in a browser is to use WebRTC (web real-time communications). WebRTC is a set of technologies for peer-to-peer transmission of video and audio in browsers, originally designed for the transmission and playback of a video stream with low latency. For this, among other things, the UDP protocol is used.
Before telling you what the new engine gives the user, we will remind you why and why we support HLS technologies, and why we decided to move on.
HLS engine: pros and cons
()
HLS (HTTP Live Streaming) technology was developed by Apple, so it's no surprise that it was first supported on Apple devices. Today, HLS video is also supported by virtually all set-top boxes and many devices running the OS. Android.
The HLS engine uses the well-known H264 video codec in combination with AAC or MP3 audio streams to stream video data. The entire stream of audio and video data is packed into an MPEG-TS transport container. For transmission over the HTTP protocol, the information contained in the stream is divided into fragments described in m3u8 playlists. And only then these fragments, together with playlists, are transmitted via HTTP. Division into fragments automatically means a delay in seconds. Such a feature of the MPEG-TS container.
The HLS engine also supports multibitrate streams, Live/VOD.
The main advantages of HLS:
- built-in support in all major browsers;
- ease of implementation (when compared with WebRTC);
- it is very convenient and efficient to organize all sorts of broadcasts for a large audience due to the fact that segments can be uploaded to a CDN once.
Despite the simplicity of the engine, not everything is as smooth as it seems. The main problem is that the developers of third-party players have deviated from Apple's recommendations, for example, in terms of supported audio formats. In particular, many developers began to add the ability to work with popular audio streams: mpeg2 video, mpeg2 audio, etc. As a result, we had to create different playlist formats for different players.
But one of the biggest problems with the HLS engine is the high latency in data transfer.
The origins of the "brakes"
The main reason for the high latency of HLS lies in the fact that the programmers created the engine to get the highest quality picture. Therefore, the parameters of the frame interval used and the size of the playback buffer are simply not suitable for live video broadcasts. Because of this, there is a rather high delay in the transmission of the video sequence, which can be 5-7 seconds.
On the one hand, this is not much, for example, for those who watch a movie from a video hosting server. But for video surveillance systems, the delay in the transmission of the video sequence can be very important.
If you are watching an office where employees take their eyes off the monitors once an hour, then a delay of 5 seconds does not matter. But people began to complain that, for example, during the broadcast of a football match, they already wrote GOOOOL in the chat, but this is not on the video yet :). We already have a number of user cases where Ivideon should practically replace skype.
Is it possible to beat latency in HLS? The answer to this question sounds like a speech by an experienced rat exterminator at a lecture to novice exterminators: “Rats cannot be exterminated, but their population can be reduced to a reasonable minimum.” So with the delay in HLS, it will not work to remove it to zero, but there are solutions on the market that can significantly reduce the delay.
Fine cutting
Another disadvantage of the engine is the use of small files for data transfer. It would seem that this is bad?
Anyone who has tried to copy a large number of small files from one medium to another must have noticed that the write speed of such a set is much lower than one large file of the same size. Yes, and the intensity of access to the hard disk increases significantly, which generally negatively affects the performance of the entire computer. Therefore, the transmission of video data in the form of small 10 second fragments also contributes to the increased delay of the engine.
Let's briefly summarize all the pros and cons of HLS technology.
Advantages of HLS:
- Ability to work with any device. You can watch videos on any modern device, be it a smartphone, tablet, laptop or desktop PC. The main thing is that the web browser is up-to-date and compatible with HTML5 and Media Source Extensions.
- Excellent image quality. The adaptive data transmission function used allows you to dynamically change the quality of the transmitted video sequence depending on the bandwidth of the Internet connection, while the algorithm strives to keep the quality as high as possible.
- There is no need for complex user equipment setup.
Disadvantages:
- Limited support for working with the engine on some devices.
- High latency in image transmission.
- Strong increase in overhead and complexity of optimization due to the use of small files. Due to the nature of the container, we can never get a delay less than the segment size.
The disadvantages of HLS outweighed its advantages for us and forced us to look for alternatives.
What is WebRTC
()
The WebRTC platform was developed by Google in 2011 to transfer streaming video and audio data between browsers and mobile applications with minimal delay. For this, the standard UDP protocol and special flow control algorithms are used. Today it is an open source project, actively supported by Google and developed.
WebRTC is a set of technologies for peer-to-peer video and audio transmission. That is, for example, users' browsers using WebRTC can transfer data directly to each other, without using remote servers for storing and processing data. All information is also processed by end users' browsers and mobile applications.
The convenience and extensive capabilities of this technology have been appreciated by developers of all popular browsers. WebRTC support is currently available in Mozilla Firefox, Opera, Google Chrome (and all Chromium-based browsers), as well as in mobile apps running Android and iOS.
With all its undeniable advantages, WebRTC has several significant disadvantages.
Difficulties in choosing
WebRTC technology is much more complex in terms of network interactions due to the fact that it is about P2P. It is difficult to debug, test, it can behave unpredictably. At the same time, we need to overcome NAT and firewall, we need to ensure work in networks where UDP is blocked.
Google's WebRTC implementation is very difficult to use. There is even an entire company that provides SDK building services. Plus, Google's implementation was very difficult to integrate with our system without re-encoding the entire video.
However, we have long wanted to give users the opportunity to work with a full-fledged "live" video sequence and minimize the lag of the image on the screen from the events themselves. Plus, we had a desire to make the use of PTZ cameras more comfortable, where delays are critical.
Considering that other anti-lag implementations still have limited functionality and work noticeably worse, we decided to use WebRTC.
What have we done
Properly implementing the WebRTC platform is not an easy task. Any miscalculation or inaccuracy can lead to the fact that the delays in the transmission of the video sequence will not only not decrease in comparison with other platforms, but will even increase.
For WebRTC to work correctly, first of all, it is necessary to carry out a technological upgrade of the stack for working with web video. Which is what we did.
First, we implemented a WebRTC signaling protocol server over Websocket, and also deployed a WebRTC peer server in the cloud based on the webrtc.org SDK. Its task is to distribute video streams to client WebRTC peers in H.264 + Opus/G.711 format without video transcoding.
We chose Websocket as the signaling protocol because it already has good support in all popular web browsers. Due to this, you can significantly reduce not only development overhead, but also not waste time and resources on repeated TCP and TLS handshake compared to AJAX.
The point is that, by default, WebRTC does not provide the signaling protocol required to properly set up, maintain, and terminate real-time video communications between source and client applications.
And in order to independently implement the signaling technology, we needed to develop our own signaling server with support for several web protocols (Websocet, WebRTC). And with the ability to securely manage sessions and real-time notifications, manage video, and more.
We overcame the limitations of P2P by reducing latency not through P2P, but through UDP and flow control to reduce latency. This is also built into WebRTC, since the main use-case is p2p conversations through the browser.
In the mobile client, we implemented the player using the webrtc.org SDK, since it is the only one that correctly implements flow control, has all known Forward Error Correction (FEC) schemes, and correctly implements the mechanism for resending packets for all browsers. It is also important that the webrtc.org SDK is actively developed by Google.
What is the result of implementing WebRTC?
To view live video from cameras, we have added a new optimized player based on WebRTC to your personal account. It provides high speed video loading and completely eliminates the problem of latency accumulation as the viewing time increases.
After implementing WebRTC support in the Ivideon cloud service, we can say with full confidence that now our customers can watch full-fledged live video. Now the delay in broadcasting the video sequence does not exceed one second! For comparison, the previous HLS engine provided video delivery with a delay of 5-7 seconds. The difference in the speed of video demonstration is very significant, and the user will notice it immediately after starting to work with our video service.
As we expected, the implementation of the new player made it possible to increase the responsiveness of PTZ and voice communication with the camera.
There is only one subtle point to which we want to draw attention. The new WebRTC player is still working in test mode. And that's why we don't enable it for all our clients by default. But you can activate it yourself by enabling the corresponding item in the camera settings (to do this, go to ).
Features of the implementation of WebRTC in the Ivideon service
WebRTC is still an experimental technology at the moment. Its support is not yet correctly implemented in all browsers and user devices, and not in all cameras.
This is precisely what explains the fact that we have not yet made the WebRTC player the main default for all users.
For now, we recommend using WebRTC only in Google Chrome browsers. The latest versions of Firefox and Safari also support this technology, but unfortunately it is not yet stable.
We have not implemented WebRTC support for browsers on mobile devices yet. Now, if you log in from a mobile device and activate WebRTC, this mode will not work. However, WebRTC is available in our mobile applications for и .
And completing the story about the features of the implementation of WebRTC in our service, we note two more subtle points.
Firstly, the technology is focused on broadcasting live video in real time. Therefore, if your bandwidth is not enough to transmit the video sequence, you will notice frame drops (with HLS you will notice video fading and an increase in latency, while no frames will be dropped), but the video will still be broadcast in real time.
Secondly, since the technology is designed to work with live video in real time, we do not use it to work with archived video data.
Other service changes
At the moment, Flash no longer participates in the automatic engine selection mechanism. You can still use such a player, but for this you need to select it manually in the account or camera settings. This is not a tribute to fashion, just according to the statistics of our service, there are practically no users working with Flash. And trying to determine if the user's browser supports it, we lose about 2 seconds of precious time.
Here is a brief summary of the changes that await you in our cloud-based video surveillance system and personal account. Stay with us and follow the news!
Source: habr.com
