In this article, we will tell you what infrastructure solutions our clients implement using Dell R730xd server platforms and why the price of renting this platform is with excellent communication channels to Ukraine and Russia, as well as , already with placement and connectivity at a price became a reality. We will share possible solutions based on these platforms using private vlan, 10G local area network and hardware Firewall from Cisco, which are available to our customers upon request. And also, in the best traditions, we will offer a bonus in the form of a free period of using Dell R730xd servers for Habrahabr readers.
Recently, we have been receiving more and more requests to build various corporate infrastructures, and this, unfortunately, is due not so much to the fact that we provide very high quality solutions, but to the price of these solutions and the level of security and the rule of law, which is provided in the Netherlands and the USA, but, alas, is often not available in Ukraine and Russia. Where, unfortunately, such solutions cost simply "space" money, since the concept of "long money" is alien to the post-Soviet countries in principle, against the background of other risks or in the absence of the necessary infrastructure and level of certification.
A good example here would be the difference in the price of iron. Eg, platform Dell R730xd, purchased for our clients, in the basic configuration 2 x Intel Dodeca-Core Xeon E5-2650v4 128GB DDR4 6x480GB SSD costs about 9000 euros. It goes without saying that there can be no question of any low price for renting this server in Ukraine or Russia, since the cost of rent is formed based on the payback of the cost of the platform for a period of 12-18 months. And this means that the minimum possible rental price, excluding the cost of accommodation, electricity and communication channels, will be in the range of $500-800 / month, depending on the degree of risk of the provider and the business plan. And you also need to find a good data center with the right level of certification and communication channels. Well, do not forget about taking into account the risks of illegal seizure of equipment in the course of some illegal investigative actions that may arise due to competition and the specifics of doing business in a particular region.
Therefore, it is not surprising that our proposal aroused great interest, especially since the data centers in which we provide the proposed servers have many certificates that are necessary and important for the corporate sector - , , , ΠΈ .
Now let's look at a few cases that may be of interest, both for beginners and small projects, and for fairly large system integrators who build solutions in their offices in containment areas or in data centers.
Advantages of Dell R730xd platforms when building data warehouses using Red Hat Ceph
It is no secret that the requirements for data storage and processing are constantly growing, and at an accelerated pace. If a few years ago it was enough to have 1TB of storage, which provided several hundred IOPS, now the needs have increased to tens of thousands of IOPS and a petabyte of space. These capacity and performance requirements are fueled in part by an increase in the volume of unstructured data, including music, images, videos, database backups, log files and other archives, financial and medical data, that is, data known as "Big Data". Not to mention the growing demands for data storage as a result of the spread and expansion of the Internet and new Internet resources. So with all these power requirements, customer expectations for high reliability and high performance are greater than ever.
While IT companies are trying to solve the problem of managing petabytes and even exabytes of data, the cloud storage model is becoming more common in modern data centers. More and more new software systems are being written that allow you to configure the interaction of the cloud environment with the hardware in an optimal way, one of these developments is Ceph.
Ceph is an open source distributed storage system designed to provide high performance, reliability, and scalability. Ceph implements object storage on a distributed computer cluster and provides interfaces for object, block, and file storage. Ceph provides fully distributed storage with no single point of failure and scalability to the petabyte level. Ceph replicates data and thus provides fault tolerance. The system is designed in such a way as to provide not only self-restoration, but also management, which helps to avoid unnecessary maintenance costs. Because Ceph uses generic software-controlled hardware and its management functions are available through application programming interfaces (APIs), it is referred to as software-defined storage (SDS).
Red Hat Ceph Storage is a deployment-ready, software-defined storage system that is open, adaptable, scalable, and globally supported. Combines innovation from open-source development with technical core and support from Red Hat. The solution provides tight integration with OpenStack and is designed from the ground up to provide next-generation storage for both cloud environments and other workloads.
Here we will consider an example of implementing this solution on Dell servers, in particular the Dell PowerEdge R730xd, which we offer for rent, and consider the benefits of building storage based on these platforms. Perhaps this information will be useful to architects, engineers and IT administrators who wish to explore the benefits of using Red Hat Ceph Storage on Dell PowerEdge servers and who need to design and plan implementations using proven best practices.
But at the beginning:
A bit about the platform itself, which option do we offer and why is it cost effective?
Dell PowerEdge R730xd is by far one of the best platforms for the enterprise sector and beyond, it has won many different awards and is a fairly effective solution for organizing data storage, as it can provide high-density storage for a reasonable price.
The PowerEdge R730xd is available in three chassis configurations with different form factors and drive combinations, as well as an optional rear expansion unit:
- 24 front accessible 2,5" SAS, SATA or nearline SAS drive bays plus 2 optional 2,5" rear bays. The 2,5" chassis can optionally support up to 4 x Dell PCIe Express Flash drives in the front.
- 12 front accessible 3,5" SAS, SATA or nearline SAS drive bays with 4 optional internal 3,5" hot-swap drive bays, plus two optional 2,5" rear bays.
- 18 front accessible 1,8" SATA bays, 8 3,5" SAS, SATA or nearline SAS drive bays plus 2 optional 2,5" rear bays.
It would seem that each chassis is good for some kind of task. It is certain. But are they really equally cost effective?
We opted for the second option to meet the needs of as wide a range of users as possible with a 12 drive bay chassis. Because we consider it the most effective. And that's why. The economic efficiency of the solution is already manifested in its versatility - this chassis can be equipped with drives of various form factors and it is cheaper to purchase, and, importantly, you can get better performance when working with data using several interconnected platforms rather than one with high storage density and obviously lower performance.
We believe that by connecting several such platforms into a multi-gigabit LAN (it is possible to connect each node to the LAN at speeds of 20 Gb / s or more, using the Intel X540-T2 dual-port ten-gigabit cards that we provide as an option), you can achieve much better results. and performance than platforms with higher storage densities. What is especially critical for the infrastructures that use these platforms to work with databases. Alas, with a large number of drives, the controller can be quite overloaded, and an additional controller in the xd version, alas, is not available. The performance that can be achieved when using platforms with exactly 12 bays and a local multi-gigabit network will be much higher, and the solution, in turn, will be more distributed and reliable. In a word - cost-effective!
Features of the structure and design, the use of video cards
The Dell PowerEdge R730xd platform, which is only 2U in size, supports up to 2 Intel Xeon E5-2600 v3 and higher processors, that is, it allows you to get up to 36 cores when using 18-core processors. We chose an intermediate option, but the latest generation - the 12-core E5-2650 v4 processor (24 cores at your disposal in total, and taking into account multithreading - 48 cores), as it turned out to be the most cost-effective. So in the fourth generation, more efficient processor instructions are implemented, for example, AES, which is responsible for data encryption, is 70% more efficient than the same processors, but the third generation. At the same time, the processor and platform can support up to 1,54 TB of RAM, which is important in some very specific cases. We have chosen the most mass-produced version, the most acceptable in terms of price and speed - 128GB DDR4 RAM and provided for the possibility of upgrading at the request of subscribers.
The front panel of the R730xd has 6 system status indicators that can notify you of various problems, so you can avoid a lot of critical problems by taking appropriate measures in a timely manner. DIMM slots for RAM are located directly on the motherboard. The R730xd supports Registered Error Correcting DIMM as well as LRDIMM (Load Reduced Dual In-Line Memory Modules), or the so-called Load Reduced Module, a relatively new type of memory for servers. But we do not use it, since LRDIMMs become useful only with large amounts of memory, when there is a goal to increase the speed of work.
The Internal Dual-SD Module (IDSDM) allows customers to deploy Dell's failover virtualization technology, which provides failover redundancy for embedded hypervisors. Although the PowerEdge R730 supports GPUs, which can be very useful in cases of building virtual office infrastructure (virtual desktops), as well as for high performance computing and co-processing, R730xd does not support GPUs, as proper cooling cannot be provided. video cards. Nevertheless, so far we have a small demand for this service and only one of our subscribers has ordered a video card for the server. It is for this reason that we did not order R730 platforms in bulk, but we can deliver them on request, equipping them with one of the recommended cards.
Due to this, unfortunately, the price for renting a solution cannot be so attractive and increases by more than 2 times, depending on the payment period and the term of the contract. We offer to buy the cards themselves through our supplier in the Netherlands, the prices are given below and only from this recommended list (such were the recommendations of Dell itself when we made the request), perhaps this information will be useful to someone:
NVIDIA Tesla M10 GPU CusKit: 2,884.98 EUR
NVIDIA Tesla M40 GPU: 4,913.33 EUR
NVIDIA Tesla M40 24GB GPU, Cust Kit: 6,458.95 EUR
NVIDIA M60 GPU, Passive, Requires GRID 2.0 SW for VDI Function, Cust Kit: 5,094.95 EUR
Licenses:
Nvidia GRID vApps Subscription License 3 yr, 1 CCU: 20 EUR
Nvidia GRID vPC Subscription License 3 yr, 1 CCU: 95 EUR
Nvidia GRID vWS Subscription License 3 yr, 1 CCU: 480 EUR
So, if you are ready for a minimum 2-year contract for renting a Dell R730 server (not xd, that's why the solution is much more expensive) - contact [email protected]we will be glad to help you! Using one of the recommended graphics cards above, in a DELL R730 2 x E5-2650 v4 / 128GB / 6 x 480GB SSD / 1Gbps 100TB + GPU configuration and a 2-year contract, it will cost about $6816 per year, instead of $2988 per year in case The Dell R730xd+ platform discussed in this article will require payment of the cost of the video card and license, as this is a very specific piece of equipment.
Nevertheless, even this price is much more attractive than the prices that data centers in Ukraine and Russia can offer for such solutions, if they even want to provide such solutions ... For, oddly enough, in the Netherlands, where cooperation with Dell is established directly, the promised delivery time for the server with a video card was almost 2 months from the moment our client placed the order (the client agreed to wait, since there are simply no alternatives on the market), due to the strong specificity of this product it just wasn't available from Dell itself. Nevertheless, the delivery was made according to the results faster than a month. Thanks to the data center supply department and the guys from Dell for the promptness. But for me, this experience was still unusual, since the standard Dell R730xd platforms are delivered within a few days.
PERC Controller Management and Capabilities
The server can be managed through the convenient iDRAC8 (Integrated Dell Remote Access Controller 8) with Dell's lifecycle controller, which reduces the time spent on management tasks, reduces error rates, improves security, and increases the overall efficiency of your IT environment.
For our subscribers, access to iDRAC8 is provided through RMI (Remote Management Interface), which is accessible only from the private network of the data center for security purposes, access to which we provide through the Open VPN tunnel for free. Once logged in, iDRAC displays an overview of the system as well as a virtual console view via iKVM.
I would like to pay special attention to monitoring, iDRAC8 allows you to get power consumption statistics for the last hour, day or week and limit the maximum power consumption:
The iDRAC temperature monitoring and recording system allows you to collect data from temperature sensors and determine the range in which it is located. So it is considered acceptable to find processors in the Warning temperature range only for 10% of the time during the year, and critical - for 1%. The time in the critical band also has an impact on the allowable time spent in the warning band. Temperature data collection starts when the system is turned on after leaving the factory and cannot be reset.
Dell's 13th generation of PowerEdge servers are equipped with PERC9 controllers, which replace the LSI CacheCade technology previously used in PERC8 controllers with the more efficient SanDisk DAS Cache.
During synthetic tests, DAS Cache provides a significant performance boost for HDD arrays in cases of HDD+SSD hybrid configurations. So, in the case of an array of 5 HDDs in RAID6 (the 6th level is selected to ensure maximum volume) and 5 SSD RAID10 (4 + 1 hot spare SSDs to maximize the performance of the array) when using DAS Cache, the performance of the array is 5 HDD RAID6 + DAS Cache on the specified SSD array turned out to be close to the performance of the SSD array itself:
So, having familiarized ourselves a little with the platform, we can now consider the advantages of this platform when building storages.
What makes the Dell R730xd platform ideal for data warehouse deployments, particularly Red Hat Ceph?
The Red Hat Ceph Storage environment uses industry standard servers for scalability, fault tolerance, and performance. Data protection methods play a key role in the cost of the solution. Ceph allows the user to select different protection methods for different storage pools.
Replicated storage pools produce full copies of stored objects and are ideal for fast recovery and data manipulation. For a replicated storage pool, Ceph defaults to a replication factor of three when three copies of the data reside on three separate Ceph nodes.
Corruption-resistant storage pools provide a single copy of data with parity, which is useful when long-term data storage is required and is cost-effective.
The PowerEdge R730xd is an extremely flexible and scalable XNUMXU rack server that delivers high performance processing and a wide range of workload-optimized local storage options, including hybrid tiering. The development includes a wide range of configurations, ideal for Ceph.
β R730xd provides the ability to create highly efficient infrastructure solutions for both small and large businesses.
β R730xd reduce commissioning time with automated deployment capabilities, which reduces user actions and reduces human error.
βPowerEdge servers improve the performance of IT infrastructure in data centers with innovative management tools such as iDRAC Quick Sync and iDRAC Direct for comprehensive system health monitoring and rapid deployment.
βPowerEdge servers have optimized energy usage to deliver better performance per watt and more precise control over power consumption and cooling.
The Dell PowerEdge R730xd delivers the benefits of maximizing performance through:
- Accelerate application efficiency with the latest technologies and dynamic local storage.
- Scale up quickly and easily with front-accessible storage that supports a wide range of drives from low-cost SATA HDDs to ultra-fast 2.5" SSDs, as well as ultra-low latency drives such as PowerEdge Express Flash NVMe PCIe SSDs.
βThe R730xd is well-adapted for hybrid storage by using new caching technologies in the PERC controller, which increases the cost-effectiveness of the Ceph solution.
An example of an infrastructure implementation with Red Hat Ceph Storage, which consists of 5 Dell R730xd servers, you can see below:
Independent testing of various Dell R730xd platform configurations yielded the following results:
Our version of the platform shows a very good result, and in the case of reading, perhaps the best. And if we take into account the fact that we ordered these platforms in a large batch and due to this we can offer a price more than 2 times lower than for other platform options, it turns out to be the best consolidated for reading and writing in terms of the cost-effective parameter. In the end, nothing prevents building a cluster of 7-8 platforms, instead of 5. At the same time, getting even more fault tolerance.
Are 10-gigabit switches required when building such infrastructures? No, they are not required, one of our clients built an infrastructure, though from 3 servers, using the following LAN connection scheme:
This allowed him to save quite a bit - $350 / month on switch rental, since 10-gigabit switches are quite expensive, and we provide only the best ones - Arista 7050tx-48-r. We see no point in solutions for several ports.
What are the Dell R730xd platforms capable of in principle?
Not so long ago, Microsoft conducted an interesting performance test of these platforms in the case of building data warehouses with a small number of nodes.
We used 4 Dell R730xd nodes connected to a local 100-gigabit network using a 32-port Arista DCS-7060CX-32S 100Gb switch, running EOS version 4.15.3FX-7060X.1.
The configuration of the nodes used is as follows:
2x Xeon E5-2660v3 2.6Ghz (10c20t)
256GB DRAM (16x 16GB DDR4 2133MHz DIMM)
4x Samsung PM1725 3.2TB NVME SSD (PCIe 3.0 x8 AIC)
Dell HBA330
4x Intel S3710 800GB SATA SSD
12x Seagate 4TB Enterprise Capacity 3.5β SATA HDD
2x Mellanox ConnectX-4 100Gb (Dual Port 100Gb PCIe 3.0 x16)
Mellanox FWv. 12.14.2036
Mellanox ConnectX-4 Driverv. 1.35.14894
Device PSID MT_2150110033
single port connected / adapter
With the help of VMFleet, 20 virtual machines were raised on each node, that is, a total of 80 virtual machines. Each virtual machine was configured with 1vCPU. VMFleet was then used to run the DISKSPD utility to test storage performance on each of the 80 nodes, the utility is available for free. . The test parameters are as follows - 1 thread, 512KiB sequential read for 4 I / O operations.
As you can see in the image above, we managed to achieve an incredible result of 60 gigabytes per second in total, which is approximately equal to 5 English versions of Wikipedia in compressed form (11.5GiB), which are downloaded every second. And the speed from each virtual machine was about one CD per second - 750 MB.
This test perfectly demonstrates how effective a solution can be when the three components of compute, storage, and network are balanced, minimizing potential bottlenecks that can occur in an unbalanced system.
What about platforms from other manufacturers?
Compare Dell R730xd vs HP ProLiant DL380
To compare the performance, we chose hardware that is weaker than what we offer for rent - servers with an E5-2620v3 processor, and not an E5-2650v4, which are much more productive. The main testing methodology used for this comparison was to measure the number of IOPS. Several different workload simulations were used, namely all Read and 30% Read / 70% Write (analogous to an OLTP system, a transactional system when processing is small transactions, but going in a large stream, and clients need to ensure a minimum response time).
The tests were carried out several times to exclude the negative impact of hidden factors on system performance. Initially, we ran baseline tests on a Dell R730xd and an HP ProLiant DL380 using only HDDs in storage (5x1TB HDD RAID5), which served as a benchmark for performance. The same test suite was then run with 5x1TB HDD RAID-5 storage using DAS cache installed on SAS SSDs (2xSAS 480GB Samsung SSD RAID1) on a Dell server and on an HP server using HPE SmartCache with a similar set of drives for storage and caching. Finally, the last set of benchmarks was run on a Dell server using DAS Cache on NVMe PCIe SSDs (2x400GB Samsung NVMe in software RAID) to show the maximum contribution of caching to application performance gains in read/write intensive cases. A similar test was not done on an HP server because Smart Cache does not support NVMe drives for caching.
Systems configuration:
Dell PowerEdge R730xd (13th Generation)
HP ProLiant DL380 (9th Generation)
Server & Hosting
CPU: x86-64 - Intel Xeon CPU E5-2620 v3 @2.40GHz;
Memory: 32GB DDR4.
CPU: x86-64 - Intel Xeon CPU E5-2620 v3 @2.40GHz;
Memory: 32GB DDR4.
Operating system / software
Windows 2012 R2 SP1;
SanDisk DAS Cache v1.4.
Windows 2012 R2 SP1;
HPE Smart Cache.
Storage / cache
5x1TB HDD RAID5;
2xSAS 480GB Samsung SSD RAID1;
2x400GB Samsung NVMe Software RAID.
5x1TB HDD RAID5;
2xSAS 480GB Samsung SSD RAID1.
Test
Simulate read-write OLTP workload;
300GB database size;
4 simultaneous IOMETER workers with a queue depth of 32.
Simulate read-write OLTP workload;
300GB database size;
4 simultaneous IOMETER workers with a queue depth of 32.
Several random read / write tests were carried out at different block sizes - 4 and 8KB. For each block size, in turn, tests were performed at 100% reads, as well as at 70% reads and 30% writes. Tests that dealt with caching performance acceleration were run for 900 seconds (15 minutes) to provide enough time for the cache to take full effect and get more accurate results.
Performance measurement in IOPS (Input/Output operations per second):
Server / tested configuration
4KBRR
100% read
4KBRR
70% Read 30% Write
8KBRR
100% read
8KBRR
70% Read 30% Write
Dell R730xD (no cache)
1650
974
1540
1040
HP ProLiant DL380 (no cache)
1370
628
1322
630
Dell R730xD
with DAS Cache
138884
66483
98368
56641
HP ProLiant DL380
with SmartCache
41273
33534
35984
39396
Dell R730xD with
DAS Cache and
NVMe PCIe SSD
264750
158157
257150
104490
Measuring performance in % processor utilization:
Server / tested configuration
4KBRR
100% read
4KBRR
70% Read 30% Write
8KBRR
100% read
8KBRR
70% Read 30% Write
Dell R730xD (no cache)
0,4%
0,28%
0,42%
0,3%
HP ProLiant DL380 (no cache)
0,8%
0,5%
0,8%
0,5%
Dell R730xD
with DAS Cache
13%
8,8%
11,34%
7,83%
HP ProLiant DL380
with SmartCache
6%
6%
5%
5%
Dell R730xD with
DAS Cache and
NVMe PCIe SSD
16%
10,1%
16%
5,78%
Test results show significant performance gains on Dell R730xd platforms with SanDisk DAS cache versus HPE SmartCache, and NVMe PCIe SSD support provides even greater benefits. In OLTP simulations, the R730xd showed up to 3x IOPS with SanDisk DAS Cache and SAS SSDs and 6x with SanDisk DAS Cache + NVMe PCIe SSDs compared to HPE's SmartCache. Higher CPU utilization with SanDisk DAS Cache, and even higher CPU utilization with SanDisk DAS Cache + NVMe PCIe SSDs are just a consequence of the much higher IOPS and shows, on the contrary, more optimal consumption of processor resources.
So, if in HP at 6% utilization the result in IOPS is 41 thousand, that is, 1% of processor resources are consumed per 0,15 thousand IOPS, then in the case of Dell R730xd we have 16% consumption at 264 thousand, which provides a more efficient indicator - 0,06% of CPU resources per 1000 IOPS.
That is, in terms of processor utilization, Dell also turns out to be better (2,5 times), although it has processor utilization values ββin% higher, but, as we could see, this is only a consequence of processing a significantly larger number of operations, and therefore better performance.
Thus, the Dell R730xd platform turns out to be much more efficient than the HP ProLiant DL380 (by several times), both in terms of performance and cost-effective parameter.
Dell R3xd Geekbench 730 results with E5-2640 v4 processor or why processor instructions and controller cache are important
As we could see above, we can achieve maximum results only when the three components - computing power, storage and network are balanced, as potential bottlenecks that can occur in an unbalanced system are minimized.
But it is also extremely important that the platform processor supports a number of instructions and their efficiency. I already mentioned above that a seemingly insignificant difference in generations can play a rather significant role in some cases. So, according to the results of independent tests, the E5-2650 v4 that we provide is 70% more efficient in the case of encryption (AES instructions) than the E5-2650 v3.
And then what about the βtastyβ solutions from competitors from the low-cost, but not the premium segment? Why are our solutions better? Why are our solutions premium? The answer is simple - our servers were and are balanced. Let them have had lower frequencies earlier or smaller amounts of memory, but they were always balanced in the three parameters described above. Including a reliable data center with the certificates that the corporate segment needs and excellent external communication channels that provide minimal latency from the Netherlands, both to the rest of Europe, and to Russia and Ukraine, and even
But now we have made a competitive offer for these indicators, and at once in 10 locations, in one in the Netherlands and in 9 locations in the USA, but we will not talk about us, but let's look at the test. Unfortunately, this test is currently only available for the E5-2640 v4 version, with 10, not 12 cores, for our platform.
It, of course, is not very objective, since it does not take into account all the advantages of the platform, the same SanDisk DAS cache that we talked about, real performance indicators can turn out to be much better than synthetic ones in a particular case, much! But still some understanding is possible to get.
Let's look at a lot of instructions. The most indicative may be the already mentioned AES, at approximately the same frequencies, the difference between the values ββfor the 1st core can be up to 1000 times depending on the processor, the rest of the instructions that were compared showed differences in values ββfrom several to 100 times.
You can yourself and make a comparison, but do not forget that this test is not completely indicative, since it does not take into account the hardware as a whole, but mainly characterizes the processor.
Nevertheless, I hope that these results may be useful to you when analyzing the effectiveness of a particular solution.
Using Dell R730xd servers like Bare Metal is simply better!
Some of our clients have decided to move away from VMware cloud solutions in favor of Bare Metal (iron servers) solutions and the Dell R730xd platform has helped them a lot. And this is not only about public clouds, but also about private ones.
Why do more and more people make such decisions? At the expense of public clouds, everything is more or less clear. I myself regard public clouds as a big marketing, if something falls there (and all clouds fall sooner or later), it falls there for a long time. An example is the well-known Amazon, which at one time lay for several days along with an important service for collecting telemetric medical data, including data on the state of the heart of patients using a pacemaker who were monitored by doctors, several people died ... Yes, and recently, when our corporate Bitrix was lying down, it turned out that the reason was that the employees who turned off some of the machines, as a result of a typo, turned off more nodes than they needed, affecting the control nodes, as a result of which the virtual machines restarted within 5 hours due to a large cluster size ... But not only this is a disaster, it may happen that you suddenly get a βbottleneckβ due to the use of the cloud by some other projects or the inability to quickly manage its infrastructure to meet your own needs.
As for private clouds, in theory, virtualization simplifies availability and disaster recovery. In practice, there are many situations where this simply does not make sense:
In case of technical problems on a node, a 15-30 minute downtime is acceptable.
Far from all projects, uptime at the level of about 100% is important, except perhaps for schoolchildren who are used to paying a dollar and demanding 100%. For some projects, 99,9% guaranteed uptime is more than acceptable. For 0,1% is unavailability of a maximum of 44 minutes per month, which can be observed for various unplanned reasons - due to hardware failure or network unavailability. We guarantee network uptime at the level of 99,99% for standard solutions, which allows unavailability of only 4 minutes per month. If the server is connected to 2 independent channels that pass through different communication nodes and whose traffic is processed by independent cores, the guaranteed network uptime increases several times more. This makes it possible to meet the uptime requirements of even banks, for which unavailability of about 40 minutes per year is acceptable, in accordance with certification. In reality, periods of unavailability are extremely rare. In 5 years of working with the data center, there has never been such a situation when all the servers became unavailable at the same time due to problems with the network or electricity. Even when there was a global blackout in the Netherlands and many other data centers became unavailable - someone forgot to refuel diesel engines, someone did not have enough UPS power, only less than a percent of servers in our data center became unavailable for a short period of time. Some of the servers that customers rent from us, even rather outdated, it would seem, hardware, with a higher probability of failure than new branded solutions, have not been rebooted even once in 3 years, just as they have never disappeared in 3 years net. Is an unavailability of 30 minutes acceptable in 3 years? More than, even for banks.
And a reasonable question arises, why then overpay? It goes without saying that you should not rely on chance and you should always proceed from the principle of reasonable reservation. The main rule that I discovered for myself is the following: a reservation should cost no more than the loss that may occur in the event of a period of unavailability due to the absence of such a reservation. That is, if the guaranteed uptime allows unavailability of 40 minutes per month, and in case of unavailability during such a period of time, the losses will amount to several hundred or even a thousand dollars, renting an additional solution to increase the uptime guarantee simply makes no sense. For in reality, the servers will not be unavailable every month for 40 minutes, 40 minutes of unavailability is a guarantee, the worst case scenario. Which is not implemented monthly, as practice has shown.
This way servers can be important but not critical, and when project owners don't mind manually switching the load to another dedicated server, we can avoid the complexities of building a failover cluster and shared failover storage.
Low productivity is unacceptable during the working period. Many of our customers choose bare metal solutions from Dell for this very reason, it is much cheaper and more profitable to use hardware to the fullest during standard operation, which is practically not subject to inaccessibility, as we have already considered, rather than overpay for the cloud option and the same performance in the cloud, which will cost many times more than the dedicated solution. The cloud and clustering simply will not be able to meet the resource needs, they will not be cost-effective for such clients.
Subscribers don't want to pay a lot for a service with limited features. Most of our clients are small companies that do not have their own technical departments and funding to purchase their own hardware. Buying iron could be a real blow for them and not provide proper performance, because over time, iron tends to become obsolete, and itβs a pity to throw away expensive hardware, even after 3 years, at a time when the company has grown and something better is needed. In the case of renting a Dell R730xd from us, you have the opportunity to switch to another server at any time or after a year if you choose the option of renting with a contract for a year. Moreover, there is an opportunity to save on paying for VMware licenses.
Using Dell R730xd Servers for Database Servers
Many of our customers who use these database servers are very fond of this platform. And not only because, unlike the Dell R730, the xd version can be provided with NVMe PCIe SSD support, which provide minimal latency, which is extremely important for such solutions. But also because the controller is incredibly efficient even with the usual 2,5" SSDs we offer to customers.
Of course, there is also a drawback in the Dell R730xd - there is only 1 controller, while the Dell R730 for 26 drives uses 2 hardware controllers. But this is not a significant problem, since we have chosen a platform with 12 drive bays, and therefore you will not get a "bottleneck" in the form of insufficient controller performance when working with this platform and a large number of SSDs. And the advantage in the form of the ability to install an NVMe PCIe SSD fully covers this drawback.
As mentioned above, not everyone needs complex clusters for databases, the performance that this platform can provide is much more important. One of our clients did just that, abandoned the excesses of virtualization and building complex clusters, renting a VPS with dedicated drives for database backups from the main Dell R730xd server: Of course, in the event of a problem with the main node, there will be some performance degradation and the need for manual switching will arise, however, this is more than a cost-effective solution given the limited budget.
We do not urge everyone to such savings, in some cases it is even very justified. Nevertheless, by increasing the budget by only $ 230 per month, you can allow an already identical E5-2650 v4 node and connect them to each other in a 20-gigabit local network, which, in principle, will allow you to build a small cluster with acceptable latency even without using PCIe NVMe SSD drives.
The solution itself will look like this (with a basic set of drives):
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV + Intel X540-T2 20GBPS LAN - $289 / month
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6x480GB SSD 1Gbps 100 TV + Intel X540-T2 20GBPS LAN - $289 / month
It does not always make sense to increase their number to increase productivity. So, in the course of tests with databases, we found that a solution with 8 SSDs is only 4% more efficient than a solution with 35 SSDs.
In terms of recommended RAID levels, RAID5 can be very optimal in some cases. Of course, RAID10 provides the best performance, but what if the drives are so fast that using them in RAID10 could cause the controller to become a bottleneck? In such cases, building a HW RAID array RAID5 is much more profitable using hot-spare drives than RAID10 (if the goal is not to lose half of the SSD quota).
Do I need to have the controller cache enabled or is it better to disable it? The RAID controller has a very limited amount of memory, what is 1GB, in the case of working with an SSD? In cases with fast SSDs, there is no point in having a read cache enabled, since clogging the memory with read operations that will not get any faster, since SSDs are already fast enough, we do not leave enough write cache, and, as you know, in inexpensive SSDs - the bottleneck is write operations, due to a drop in performance with insufficient over-provisioning and 100% utilization of free quota (a drop of dozens of times is possible). Therefore, we recommend leaving only the write cache enabled. Thus ensuring maximum efficiency of this solution.
Does NTFS block size matter that much? For most storages, 64KB is listed as the optimal block size for maximum performance. But very often we come across cases when the client is already using the server for databases, having applied the initial setting with the standard NTFS cluster size of 4KB. To fix this, you need to re-format the drives, but is there such a big difference in the general case? Do I need to expose myself to possible downtime and inconvenience associated with migration?
The controller in the Dell R730xd server provides such good performance even with 4 SSDs and write cache enabled (very important point, the read cache should be disabled because it does not provide any benefits, but only reduces the efficiency of write operations due to lack of space for a full-fledged cache, as stated above) that even with the standard block size of 4KB, we get very good performance, extremely good.
Nevertheless, we always go to a meeting and are ready to offer a similar platform for a while for subscribers to carry out work, if something was not taken into account by them.
RAMdrive is still faster than SSD drives. If your queries can be processed in TempDB, such as sorting or merging, then using a RAMdisk (a program that turns part of your RAM into a drive) can be quite justified. So, comparing the speed of 8 SSD drives in RAID10 with RAMdrive, we found that it is about 4 times faster with 4K write operations with a queue depth of 32, which just imitates the work of TempDB files. Even by combining 8 SSDs in RAID0, it was not possible to beat the results of RAMdrive.
RAID5 in the case of 8 SSD drives can be more efficient than RAID10. 4K reads with a queue depth of 32 are about 40% faster, which is not surprising given the characteristics of RAID5. RAID5 wins in performance in other tests as well, so it is 20% less efficient for sequential writes, it is only inferior to 4K writes with a large queue depth (in our case 32) and turns out to be less efficient by 30%. So the use of RAID5 can in some cases be very economically justified.
More storage doesn't always mean faster. When we are dealing with random small operations, adding drives to the array beyond a certain limit does not improve performance in any way. It would seem why? It turns out that everything depends on the capabilities of the controller, which is forced to write data to a whole bunch of drives. That is why we purchased platforms with a maximum of 12 drives, in order to reduce the likelihood of a "bottleneck". However, one has to look at the intensity of operations within each project, sometimes 4 larger drives, say 960GB, can be a better and more cost effective solution than a bunch of 480GB drives.
Using Dell R730xd Servers for Aerospike NoSQL Data Servers
Aerospike is an open-source, flash-optimized NoSQL database for high speed and scalability. Aerospike is designed to provide very low latency for read requests and heavy write load to meet today's business needs to be able to handle large amounts of I/O. Aerospike directly manages local storage on the nodes of its cluster, regardless of the file systems of the source media, be it DRAM, flash, or traditional hard drives. This allows you to store the most important data on the fastest media, the fastest processing of which is most critical, while the distributed nature of the Aerospike database provides a high degree of reliability in the event of cluster node failures.
Dell's 13th generation servers, namely the PowerEdge R730xd, with the latest E5-2650 v4 processors and fast DDR4 SDRAM, deliver incredible processing power. And the ability to use NVMe SSDs, such as the Samsung SM1715, can help get huge performance at minimal latency in nodes with high load. This makes it possible to comply with modern SLA requirements for the processing time of one transaction with a reliable level of protection against fraud even in very large databases, making the solution acceptable even for the financial sector and bank transfers, where transactions are sometimes carried out for several hours, or even days.
For the banking sector, we can offer a solution based on these platforms with a local network of 40 and 100 Gbit / s between cluster nodes and the use of appropriate switches.
Using Dell R730xd Servers in DMZ Networks
We are pleased to offer our subscribers the highest security standards. The data centers where we offer these servers have many certifications: , , , ΠΈ .
But in order to meet the requirements of the same Google, some customers need to build so-called demilitarized networks (DMZ networks) - networks with increased security, which implies the presence of a hardware firewall that restricts access to equipment and allows you to hide the necessary components behind a firewall and more improve security more.
And everything is OK when such a solution is ordered immediately, but it is very difficult to implement the transition with almost no downtime in the process, which recently happened with one of our clients, with a project to process visa applications for citizens from around the world, where unavailability even during hours can result in losses of tens of thousands of dollars. But we found a solution, went to a meeting, put additional equipment in a closet, moving the less important, and turned on the firewall according to the scheme, as the client wished, so that he could migrate without downtime.
Was / Became
Perhaps this scheme will also be useful for someone when implementing such a migration, since engineers in data centers do not always want to see the obvious things, but insist on a 2-hour downtime to switch behind the firewall, but I beg you - order a hardware firewall all well in advance if you need it. We are flexible and always meet the needs, but in some situations our possibilities may be limited.
In future publications, we will try to conduct a number of additional tests and share additional experience related to the use of these platforms, which will further increase the understanding of the selection of the most effective solution. After all, it is surprising to see the fact that RAID0 of two SSD drives, in cases of databases with a high load, can be slower than a single drive for writing, which is why we receive requests for NVMe PCIe SSD from time to time. It is likely that we will show the most effective scenarios for implementing solutions, share additional experience when building multiple RAID arrays within a single RAID controller turns out to be effective, we will show in which cases two independent RAID10 arrays of 4 drives turn out to be better than one RAID10 array of 8. When would it be efficient to use RAID1 for TempDB and RAID5 for everything else. And how can you do without NVMe PCIe SSD when you have a limited budget.
Dell R730xd: bonus for Habrahabr readers
We want to provide you with the best and have done everything for this by offering unique prices for the Dell R730xd:
- in the Netherlands
- and in the USA
Configurations can be very different, for example, available for instant activation in the Netherlands are::
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 6Γ480 SSD 1Gbps 100 TV -*$249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 12Γ240 SSD 1Gbps 100 TV -*$249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 4x4TB 4Γ480 SSD 1Gbps 100 TV - *$249
Dell R730xd 2 x Intel Dodeca-Core Xeon E5-2650 v4 128GB DDR4 12x4TB SATA 1Gbps 100 TV - *$249
And an honest dedicated 1Gbps Unmetered (excluding traffic) is available for them for +$120 / month. Plus, the ability to buy a firewall, network cards, a switch and build a multi-gigabit local network. But the condition for the price is valid for a contract for a year.
We want to improve the condition and offer a bonus for short-term payments and rentals without a contract, making the offer more affordable. Anyone who orders and pays for any of the proposed options in the Netherlands for a period of 1 month, we will provide exactly the same server for 1 month for free, or we will recalculate for the ordered configuration and calculate the ordered server at the price, as with an annual contract ($249 / month instead of $369 / month), returning the difference to the balance. To do this, just leave the order number in the comments. For those who have not grown up to dedicated solutions yet, we offer to rent any VPS (KVM) with dedicated drives in the Netherlands or the USA on these platforms, for example, , and get a bonus of 1-4 months of use when paying for 1, 3, 6, 12 months, respectively, indicating the order number in the comment. Now it's easier to build infrastructure!
Source: habr.com