Hi Habr! In this article, we will tell you whether it is worth organizing RAID arrays based on SATA SSD and NVMe SSD solid-state solutions, and will there be a serious profit from this? We decided to look into this issue by considering the types and types of controllers that allow this, as well as the scope of such configurations.
One way or another, each of us at least once in our lives heard such definitions as “RAID”, “RAID-array”, “RAID-controller”, but hardly attached serious importance to this, because all this is unlikely for an ordinary PC boyar Interesting. But everyone wants high speeds from internal drives and the reliability of their work. After all, no matter how powerful the filling of the computer is, the speed of the drive becomes a bottleneck when it comes to the combined speed of the PC and the server.
This was exactly until the moment when traditional HDDs were replaced by modern NVMe SSDs with a comparable capacity of 1 TB or more. And if earlier in PCs there were more often bundles of SATA SSD + a couple of capacious HDDs, today they are beginning to be replaced by another solution - NVMe SSD + a couple of capacious SATA SSDs. When it comes to corporate servers and the cloud, many have already successfully moved to SATA SSDs, simply because they are faster than regular tin cans and can handle more I/O at the same time.
However, the fault tolerance of the system is still at a fairly low level: we cannot, as in the “Battle of Psychics”, predict with an accuracy of even up to a week when one or another solid-state drive will die for a long time. And if HDDs “die” gradually, allowing you to catch the symptoms and take action, then SSDs “die” immediately and without warning. And now it's time to figure out why all this is needed at all? Is it worth it to organize RAID arrays based on SATA SSD and NVMe SSD solutions, and will there be a serious profit from this?
Why do you need a RAID array?
The word “array” itself already implies that several drives (HDD and SSD) are used to create it, which are combined using a RAID controller and recognized by the OS as a single data storage. The global task that RAID arrays can solve is minimizing data access time, increasing read / write speed and reliability, which is achieved due to the ability to quickly recover in case of failure. By the way, it is not necessary to use RAID for home backups. But if you have your own home server, to which you need constant access 24/7, then this is another matter.
There are more than a dozen levels of RAID arrays, each of which differs in the number of drives used in it and has its pros and cons: for example, RAID 0 allows you to get high performance without fault tolerance, RAID 1 allows you to set up automatic data mirroring without increasing speed, and RAID 10 combines the possibilities of the above. RAID 0 and 1 are the simplest (because they do not require software calculations) and, as a result, the most popular. Ultimately, the choice in favor of a particular RAID level depends on the tasks assigned to the disk array and the capabilities of the RAID controller.
Home vs. Enterprise RAID: What's the Difference?
The basis of any modern business is large amounts of data that must be securely stored on company servers. And yet, as we noted above, they must be provided with constant access 24/7. It is clear that, along with hardware, the software part is also important, but in this case we are talking about equipment that ensures reliable storage and processing of information. No software will save the company from ruin if the “iron” equipment does not correspond to the tasks assigned to it.
For these tasks, any hardware manufacturer offers so-called corporate devices. Kingston has powerful solid-state solutions in the face of SATA models и , as well as DC1000M U.2 NVMe, DCU1000 U.2 NVMe and DCP-1000 PCI-e NVMe models designed for use in data centers (data processing centers) and supercomputers. Arrays of such drives are usually used in conjunction with hardware controllers.
For the consumer market (that is, for home PCs and NAS servers), such drives are available as NVMe PCIe, but in this case it is not necessary to buy a hardware controller. You can limit yourself to a PC or NAS server built into the motherboard, unless of course you plan to assemble a home server for atypical tasks yourself (start a small home hosting for friends, for example). In addition, home RAID arrays, as a rule, do not imply the presence of hundreds and thousands of drives, being limited to two, four and eight devices (usually SATA).
Types and types of RAID controllers
There are three types of RAID controllers based on the principles of implementing RAID arrays:
1. Software, in which the control of the array falls on the CPU and DRAM (that is, the execution of the program code occurs on the processor).
2. Integrated, that is, built into the motherboards of a PC or NAS server.
3. Hardware (modular), which are discrete expansion boards for PCI/PCIe motherboard slots.
What is their fundamental difference from each other? Software RAID controllers are inferior to integrated and hardware RAID controllers in terms of performance and fault tolerance, but they do not require special hardware for operation. However, it is important to ensure that the host system's processor is powerful enough to run the RAID software without negatively impacting the performance of applications that are also running on the host. Integrated controllers, as a rule, are equipped with their own cache memory and use a certain amount of CPU resources.
But the hardware ones have both their own cache memory and an integrated processor for executing software algorithms. Usually they allow you to implement all kinds of RAID levels and support several types of drives at once. For example, SATA, SAS, and NVMe devices can be connected to modern Broadcom hardware controllers at the same time, which allows you not to change the controller when upgrading servers: in particular, when moving from SATA SSD to NVMe SSD, controllers do not have to be changed.
Actually, on this note, we approached the typology of the controllers themselves. If there are three modes, there must be some others? In this case, the answer to this question will be in the affirmative. Depending on the functions and capabilities, RAID controllers can be divided into several types:
1. Ordinary controllers with RAID function
In the entire hierarchy, this is the simplest controller that allows you to combine HDD and SSD into RAID arrays of levels “0”, “1” or “0 + 1”. Programmatically, this is implemented at the firmware level. However, such devices can hardly be recommended for use in the corporate segment, because they lack a cache and do not support arrays of levels “5”, “3”, etc. But for an entry-level home server, they are quite suitable.
2. Controllers paired with other RAID controllers
This type of controller can be paired with integrated motherboard controllers. This is implemented according to the following principle: a discrete RAID controller takes over the solution of “logical” tasks, and the built-in one takes over the functions of data exchange between drives. But there is a caveat: parallel operation of such controllers is possible only on compatible motherboards, which means that their scope is seriously narrowed.
3. Independent RAID controllers
These discrete solutions contain on board all the necessary chips to work with enterprise-class servers, with their own BIOS'om, cache memory and processor for fast error correction and checksum calculation. In addition, they meet high standards of reliability in terms of manufacturing and have high-quality memory modules.
4. External RAID controllers
It is not hard to guess that all of the controllers listed above are internal and are powered through the motherboard's PCIe connector. What does it say? And that the failure of the motherboard can lead to errors in the operation of the RAID array and data loss. External controllers are free from this misunderstanding, as they are placed in a separate case with an independent power supply. In terms of reliability, these controllers provide the highest level of data storage.
, Microsemi Adaptec, Intel, IBM, Dell, and Cisco are just some of the companies that currently offer hardware RAID controllers.
Operating modes of RAID controllers SAS/SATA/NVMe
The primary purpose of tri-mode HBA and RAID controllers (or controllers with Tri-Mode functionality) is to create NVMe-based hardware RAID. Broadcom's 9400 series controllers can do this: for example, . It belongs to an independent type of RAID controllers, is equipped with four SFF-8643 connectors and, thanks to Tri-Mode support, allows you to connect SATA/SAS and NVMe drives simultaneously. In addition, it is also one of the most energy efficient controllers on the market (consuming only 17 watts of power, while less than 1,1 watts for each of the 16 ports).
The connection interface is PCI Express x8 version 3.1, which allows for a bandwidth of 64 Gb / s (controllers for PCI Express 2020 are expected in 4.0). The 16-port controller is based on a 2-core chip and 72-bit DDR4-2133 SDRAM (4 GB), as well as the ability to connect up to 240 SATA / SAS drives, or up to 24 NVMe devices. In terms of organizing RAID arrays, levels “0”, “1”, “5” and “6”, as well as “10”, “50” and “60” are supported. By the way, the cache memory and other controllers in the 9400 series is protected from power failures by the optional CacheVault CVPM05 module.
The tri-mode technology is based on the SerDes data conversion function: converting serial data in SAS / SATA interfaces to parallel form in PCIe NVMe and vice versa. That is, the controller negotiates speeds and protocols to work seamlessly with any of the three types of storage devices. This provides a seamless way to scale data center infrastructures: users can use NVMe without major changes to other system configurations.
However, when planning configurations with NVMe drives, it is worth considering that NVMe solutions use 4 PCIe lanes to connect, which means that each drive uses all the SFF-8643 port lines. It turns out that only four NVMe drives can be connected directly to the MegaRAID 9460-16i controller. Or limit yourself to two NVMe solutions while connecting eight SAS drives at the same time (see the connection diagram below).
The figure shows the use of connector "0" (C0 / Connector 0) and connector "1" for NVMe connections, as well as connectors "2" and "3" for SAS connections. This arrangement can be reversed, but each x4 NVMe drive must be connected using neighbor lines. The controller operating modes are set through the StorCLI or Human Interface Infrastructure (HII) configuration utilities, which runs in a UEFI environment.
The default mode is "PD64" profile (only support SAS/SATA). As we said above, there are three profiles in total: “SAS / SATA only mode” (PD240 / PD64 / PD 16), “NVMe only mode” (PCIe4) and mixed mode, in which all types of drives can work: “PD64 -PCIe4" (support for 64 physical and virtual disks with 4 NVMe drives). In mixed mode, the value of the specified profile should be the following - "ProfileID=13". By the way, the selected profile is saved as a leader and is not reset even when you roll back to the factory settings via the Set Factory Defaults command. It can only be changed manually.
Should I create a RAID array on an SSD?
So, we have already understood that RAID arrays are the key to high performance. But is SSD RAID worth it for home and corporate use? Many skeptics say that the increase in speed is not so significant as to go broke on NVMe drives. But is it really so? Hardly. The biggest limitation to using an SSD in RAID (both at home and at the corporate level) may be only the price. Like it or not, the cost of a gigabyte of HDD space is much cheaper.
Connecting multiple solid state "drives" to a RAID controller to create an array of SSDs in certain configurations can have a huge impact on performance. However, do not forget that maximum performance is limited by the throughput of the RAID controller itself. The RAID level that offers the best performance is RAID 0.
A typical RAID 0 setup with two SSDs, which uses a fixed blocking and striping method between SSDs, will double the performance (compared to the speeds of a single SSD). At the same time, a RAID 0 array with four SSDs will already be four times faster than the slowest SSD in the array (depending on the bandwidth limit at the SSD RAID controller level).
Based on simple arithmetic, SATA SSD is about 3 times faster than a traditional SATA HDD. NVMe solutions are even more efficient - 10 times or more. Assuming two HDDs in a RAID 50 level doubles the performance by 6%, two SATA SSDs are 20x faster and two NVMe SSDs are 2000x faster. In particular, a single Kingston KC3200 NVMe PCIe drive can reach sequential read and write speeds of up to 0 MB/s, which in RAID 6 format will reach an impressive 4 GB/s. And the read / write speed of random blocks of 350 KB in size will turn from 000 IOPS to 700 IOPS. But... at the same time, "zero" RAID does not provide us with redundancy.
It can be said that at home, storage redundancy is usually not required, so RAID 0 indeed becomes the most appropriate RAID configuration for SSDs. This is a reliable way to get a significant performance boost as an alternative to using technologies such as Intel Optane-based SSDs. And here's how SSD solutions behave in the most popular RAID types (“1”, “5”, “10”, “50”) - we'll talk in our next article.
This article was prepared with the support of our colleagues at Broadcom, who provide their controllers to Kingston engineers for testing with enterprise-class SATA/SAS/NVMe drives. Thanks to this friendly symbiosis, customers do not have to doubt the reliability and stability of Kingston drives with HBA and RAID controllers from production. .
More information about Kingston products can be found company.
Source: habr.com