SSD-based performance technologies widely used in storage systems have long been invented. First of all, it is the use of SSD as storage space, which is 100% efficient, but expensive. Therefore, technologies of tiring and caching are used, where SSDs are used only for the most demanded (“hot”) data. Tearing is good for long-term (days-weeks) hot data usage scenarios. And caching, on the contrary, is for short-term (minutes-hours) use. Both of these options are implemented in the storage system. . In this article, we will consider the implementation of the second algorithm − .
The essence of SSD caching technology is the use of SSD as an intermediate cache between hard drives and controller RAM. SSD performance, of course, is lower than the performance of the controller's own cache, but the volume is an order of magnitude higher. Therefore, we get a compromise between speed and volume.
Indications for using SSD cache for reading:
- The predominance of read operations over write operations (most often typical for databases and web applications);
- The presence of a bottleneck in the form of a performance array of hard drives;
- The amount of data requested is less than the size of the SSD cache.
The indications for using an SSD cache for reading + writing are the same reasons, except for the nature of the operations - a mixed type (for example, a file server).
Most storage vendors use a read-only cache in their SSD products. The fundamental difference from them is the ability to use the cache also for writing. To activate the SSD caching functionality in the QSAN storage system, you need to purchase a separate license (supplied electronically).
SSD cache in XCubeSAN is physically implemented as separate SSD cache pools. There can be up to four of them in the system. Each pool, of course, uses its own set of SSDs. And already in the properties of the virtual disk, we determine whether it will use the cache pool and which one. Enabling and disabling cache usage for volumes can be done online without stopping I/O. Also on the "hot" you can add SSD to the pool and remove them from there. When creating an SSD pool cache, you must choose in which mode it will work: read-only or read+write. Its physical organization depends on it. Since there can be several cache pools, their functionality may be different (that is, the system can have both read and read + write cache pools at the same time).
In the case of using a read-only cache pool, it can consist of 1-8 SSDs. Disks do not have to be of the same size and of the same vendor, as they are combined into the NRAID + structure. All SSDs in the pool are shared. The system independently tries to parallelize incoming requests between all SSDs to achieve maximum performance. If one of the SSDs fails, nothing terrible will happen: after all, the cache contains only a copy of the data stored on the array of hard drives. It's just that the amount of available SSD cache will decrease (or become zero if you use the original SSD cache from a single drive).
If the cache is used for read + write operations, then the number of SSDs in the pool must be a multiple of two, since the content is mirrored on pairs of drives (using the NRAID 1+ structure). Cache duplication is necessary due to the fact that it may contain data that has not yet been written to hard drives. And in this case, the failure of the SSD from the cache pool would lead to the loss of information. In the case of NRAID 1+, an SSD failure will simply cause the cache to go into a read-only state, flushing unwritten data to the array of hard drives. After replacing a failed SSD, the cache will return to its original mode of operation. By the way, for greater security, a read + write cache can be assigned dedicated hot spares.
When using the SSD caching feature in XCubeSAN, there are a number of storage controller memory requirements: the more system memory, the more cache pool will be available.
Unlike, again, most storage manufacturers, which only offer an on / off option as an SSD cache setting, QSAN provides more options. In particular, you can choose the cache operation mode depending on the nature of the load. There are three preset templates that are closest in their work to the corresponding services: database, file system, web service. In addition, the administrator can create his own profile by setting the required parameter values:
- Block size (Cache Block Size) - 1/2/4 MB
- The number of requests to read the block so that it is copied to the cache (Populate-on-Read Threshold) - 1..4
- The number of requests to write a block so that it is copied to the cache (Populate-on-Write Threshold) - 0..4
Profiles can be changed "on the fly", but, of course, with resetting the contents of the cache and its new "warming up".
Considering the principle of operation of the SSD cache, we can distinguish the main operations when working with it:
Reading data when it's not in the cache
- The request from the host goes to the controller;
- Since the requested ones are not in the SSD cache, they are read from the hard drives;
- The read data is sent to the host. At the same time, it checks whether these blocks are "hot";
- If so, they are copied to the SSD cache for later use.
Reading data when it is present in the cache
- The request from the host goes to the controller;
- Since the requested data is in the SSD cache, it is read from there;
- The read data is sent to the host.
Writing data when using read cache
- The write request from the host goes to the controller;
- Data is written to hard drives;
- A successful write response is returned to the host;
- At the same time, it is checked whether the block is "hot" (the Populate-on-Write Threshold parameter is compared). If yes, then it is copied to the SSD cache for later use.
Writing data when using a read+write cache
- The write request from the host goes to the controller;
- Data is written to the SSD cache;
- A successful write response is returned to the host;
- Data from the SSD cache is written to hard drives in the background;
Check in the case
Test stand
2 servers (CPU: 2 x Xeon E5-2620v3 2.4Hz / RAM: 32GB) are connected by two ports via Fiber Channel 16G directly to the XCubeSAN XS5224D storage system (16GB RAM / controller).
Used 16 x Seagate Constellation ES, ST500NM0001, 500GB, SAS 6Gb/s, combined in RAID5 (15+1), for data array and 8 x HGST Ultrastar SSD800MH.B, HUSMH8010BSS200, 100GB, SAS 12Gb/s as cache
2 volumes were created: one for each server.
Test 1. SSD cache read-only with 1-8 SSDs
SSD Cache
- I/O Type: Customization
- Cache Block Size: 4MB
- Populate-on-read Threshold: 1
- Populate-on-write Threshold: 0
I/O pattern
- Tool: IOmeter V1.1.0
- Workers: 1
- Outstanding (Queue Depth): 128
- Access Specifications: 4KB, 100% Read, 100% Random
In theory, the more SSDs in the cache pool, the better the performance. In practice, this has been confirmed. The only significant increase in the number of SSDs with a small number of volumes does not lead to an explosive effect.
Test 2. SSD cache in read + write mode with 2-8 SSDs
SSD Cache
- I/O Type: Customization
- Cache Block Size: 4MB
- Populate-on-read Threshold: 1
- Populate-on-write Threshold: 1
I/O pattern
- Tool: IOmeter V1.1.0
- Workers: 1
- Outstanding (Queue Depth): 128
- Access Specifications: 4KB, 100% Write, 100% Random
Same result: explosive performance and scaling with more SSDs.
In both tests, the amount of production data was less than the total amount of cache. Therefore, over time, all blocks were copied to the cache. And the work has, in fact, been carried out with the SSD, almost without affecting the hard drives. The purpose of these tests was to demonstrate the effectiveness of warming up the cache and scaling its performance depending on the number of SSDs.
Now let's return from heaven to earth and check a more realistic situation when the amount of data is larger than the size of the cache. In order for the test to pass in a reasonable time (the cache “warm-up” period greatly increases with an increase in the volume size), we will limit ourselves to a volume size of 120GB.
Test 3. Database emulation
SSD Cache
- I/O Type: Database
- Cache Block Size: 1MB
- Populate-on-read Threshold: 2
- Populate-on-write Threshold: 1
I/O pattern
- Tool: IOmeter V1.1.0
- Workers: 1
- Outstanding (Queue Depth): 128
- Access Specifications: 8KB, 67% Read, 100% Random
Verdict
As an obvious conclusion, of course, the good efficiency of using SSD cache to improve the performance of any storage system suggests itself. Applied to this statement fully applies: the SSD caching function is excellently implemented. This concerns support for read and read + write modes, flexible work settings for any usage scenarios, as well as the final performance of the system as a whole. Therefore, for a very reasonable cost (the price of a license is commensurate with the cost of 1-2 SSDs), you can significantly increase overall performance.
Source: habr.com