The modern world is hard to imagine without a plethora of media content, including audio and video data. Just recently, having a collection of MP3 files seemed like a dream. Today, 4K video files are considered quite ordinary. All this media content needs to be created, stored somewhere, and made accessible. Modern storage systems (including ) are among the best tools for working with content.
Of course, the primary consumers of capacity and bandwidth are video data. The constant increase in video resolution raises the requirements for hardware. As a result, equipment that was relevant yesterday quickly becomes obsolete. A typical transition to the next generation of resolution leads to a fourfold increase in the number of pixels in a frame. Consequently, just one minute of uncompressed 8K video occupies more than 100GB.
Today, professional work with high-resolution video content is no longer the prerogative of large studios. The growing popularity of series, streaming, and high-definition television is attracting more players into this business. All these studios continually generate massive amounts of 'raw' material that require further processing.
It has become customary that the majority of professionals in the content production industry are creative individuals. Among them, the primary approach to resolving technical issues related to disk capacity has been to purchase new external storage devices. Typically, these have been desktop NAS models with 2-5 disks. The choice is determined by the simple and straightforward procedures for their operation among non-technical specialists. The speed of operation is quite acceptable for individual use as DAS (especially with interfaces like Thunderbolt or USB 3.0). When joint access to data is needed, such a NAS (also functioning as DAS) can simply be connected to another workstation.
As the volume of the original material increases and the number of workers involved in its processing grows, this approach (let's call it 'traditional') is proving inadequate. Not only does the number of 'boxes' sharply increase (along with expenses for their acquisition), but accessibility to data also significantly declines. Moreover, during collaborative work, issues arise abundantly: conflicts over data access, insufficient speed, and more. Therefore, the 'traditional' approach is increasingly being replaced by more modern solutions based on centralized storage (or several storages) and the organization of collaborative access to content.
Of course, simply acquiring does not conclude the transition to a new content management concept. It will also be necessary to organize collaborative data access and ensure high-speed exchange between the storage and content processing nodes. There are several examples of constructing a content processing infrastructure. The main ones are as follows:
The simplest case for small studios. To organize data access, file protocols are utilized, supported by .
Medium-sized studios, which are simultaneously working on multiple projects. Here, a sensible choice would be to organize data access through a server pool. In this case, it is possible to implement fault-tolerant access to content in a 24/7 mode by duplicating all key components: servers, communication channels, switches, and NAS controllers. Constant access to data is extremely important during extended video processing, as no one wants to lose a significant amount of time, for instance, due to a failure in the rendering process. Additionally, with a server pool, it is possible to ensure load balancing for workstations to enhance overall performance.
Large studios, including those focused on widespread broadcastingIn such projects, fault tolerance through component duplication is already a must-have. Additionally, to speed things up, all resource-intensive processes for rendering and post-processing are offloaded from workstations to specialized servers that have the fastest possible access to the storage system with the content. Moreover, tiered data storage is often employed. This means using slow but large HDDs for storing original materials and archives, as well as fast SSDs for active work and/or caching. Within the unified storage system, several pools of different types of media are created, and automated tools such as and are utilized. In real large-scale projects, tiered storage is achieved through the use of multiple storage systems, each storing .
. As an example of how a media studio operates, we would like to present the organization of content processing at one of the broadcasting stations in Taiwan. A reasonably sufficient system architecture, as described in point 2, is applied here.
All media content is stored on the storage system and the JBOD expansion shelf . Each chassis and shelf is equipped with 24 NL-SAS disks, each with a capacity of 14TB. Disk space configuration:
- Storage system — 24x RAID60 pool
- Expansion shelf – 22x RAID60 pool. 2x hot spare
A server pool to provide data access – a cluster of 4 servers based on Windows Server. Access to the content is organized via the CIFS protocol. Physically, all 4 servers are connected to the storage system via Fibre Channel 16G without the use of switches, Client access to the server pool is provided via a 10GbE network. The clients use Edius v9 software in a Windows environment. Types of loads include:
- Working with 4K video on 7 streams – 2 clients
- Working with 2K video on 13 streams – 10 clients
As a result, under these loads, the system provides stable total performance of 1500 MB/s, which is comfortable for the current operation of the broadcasting station. If there is a need to increase disk space, the client simply needs to add additional shelves and expand the existing array with new disks. Of course, all these operations can be performed online without interrupting workflow.
Media has always played a crucial role in society. Today, this is more evident than ever due to the development of streaming and the entertainment industry. "Heavy" content requires a serious approach to building solutions for its processing. One of the key elements in such a solution is the disk subsystem. The storage system fits perfectly into this role, providing reliable and high-speed access as well as ease of expansion and performance scaling.
Source: habr.com
