One of the internal virtualization racks. They got confused with the color indication of cables: orange means odd power input, green means even.
Here we most often talk about “krupnyak” - chillers, diesel generator sets, main switchboards. Today we will talk about “little things” - sockets in racks, they are also Power Distribution Unit (PDU). In our data centers there are more than 4 thousand racks full of IT equipment, so I have seen a lot of things in action: classic PDUs, “smart” ones with monitoring and control, ordinary socket blocks. Today I will tell you what PDUs are and what is better to choose in a particular situation.
What are PDUs
Simple socket block. Yes, the one that lives at every home or office.
Formally, this is not quite a PDU in the sense of industrial use in racks with IT equipment, but these devices also have their fans. The only plus of such a solution is cheapness (the cost starts from 2 thousand rubles). They can also help out if you use open racks, where you can’t shove a standard PDU in any way, and you don’t want to lose units under a horizontal PDU. This is back to the issue of savings.
There are much more disadvantages: such devices do not always have internal protection against short circuit and overload, monitor indicators, and even more so, it will not be possible to control sockets. Most often they will be placed at the bottom of the rack. This is not the most convenient position for power outlets.
In general, "pilots" can be used if:
- you have thousands of servers and you need to save money,
- you can afford to blindly connect equipment without understanding what is happening with real consumption,
- ready for equipment downtime.
We do not use this, but we have clients who practice it quite successfully. True, they build the infrastructure for their services in such a way that the failure of dozens of servers does not affect the performance of the client application.
Cheap and cheerful.
Vertical placement.
“Dumb” PDUs. Actually, this is a classic PDU for use in racks with IT equipment, and this is already good. They have the appropriate form factor to be placed on the sides of the rack, making it convenient to connect equipment to them. There is internal protection. Such PDUs do not have monitoring, which means that we will not know what equipment consumes how much, and what is generally happening inside. We have almost no such PDUs left, and in general they are gradually leaving mass use.
Such PDUs cost from 25 thousand rubles.
“Smart” PDUs with monitoring. These devices have "brains", and they are able to track energy consumption parameters. There is a display where the main indicators are displayed: voltage, current current and power. You can track them by separate groups of outlets: sections or banks. You can connect to such a PDU remotely, configure sending data to the monitoring system. They write logs by which you can see everything that happened to it, for example, when exactly the PDU turned off.
They also know how to calculate consumption (kWh) for technical accounting in order to understand how much a rack consumes in a certain amount of time.
These are standard PDUs that we offer to our customers for rent, and most of these PDUs are in our data centers.
If you buy, get ready to shell out from 75 thousand rubles apiece.
Graph from our internal PDU monitoring.
"Smart" PDUs with control. In addition to the skills described above, these PDUs have control added. The coolest PDUs control and monitor each outlet: you can turn it on / off, which is necessary in situations where there is a task to remotely reboot the server by power. This is both the beauty and the danger of such PDUs: an ordinary user, out of ignorance, can go to the web interface, click something and reboot / turn off the entire system in one fell swoop. Yes, the system will warn twice about the consequences, but practice shows that even alarms do not always protect against rash user actions.
The big problem with smart PDUs is overheating and failure of the controller and display. PDUs are usually installed at the back of the rack where hot air is blown out. It's hot in there and the controllers can't handle it. In this case, the PDU does not need to be changed entirely, the controller can be changed to a hot one.
Well, the cost is quite biting - from 120 thousand rubles.
The controllable PDU can be recognized by the indication under each outlet.
In my opinion, the control function in the PDU is a matter of taste, but monitoring is a must have. Otherwise, consumption and load cannot be tracked. Why this is important, I will tell a little later.
How to calculate the required PDU power?
At first glance, everything is quite simple here: the power of the PDU is selected in accordance with the power of the rack, but there are nuances. Let's say you need a 10 kW rack. PDU manufacturers offer models for 3, 7, 11, 22 kW. Choose 11 kW, and, unfortunately, you will be wrong. We will have to choose 22 kW. Why do we need such a large supply. Now I will explain everything.
Firstly, manufacturers often indicate PDU power in kilowatts, and not in kilovolt-amperes, which is more correct, but not obvious to the layman.
Sometimes manufacturers themselves introduce additional confusion:
Here it first says about 11 kW,
And in the detailed description, we are already talking about 11000 VA:
If you are dealing with kettles and similar consumers, then there will be no difference between kW and kVA. A 10 kW rack with kettles will consume 10 kVA. But if we have IT equipment, then a coefficient (cos φ) appears there: the newer the equipment, the closer this coefficient is to one. On average for a hospital for IT equipment, you can take 0,93-0,95. Therefore, a 10 kW rack with IT will consume 10,7 kVA. Here is the formula by which we got 10,7 kVA.
Ptotal= Act./Cos(φ)
10/0.93=10.7 kVA
Well, you ask a reasonable question, 10,7 is less than 11. Why do we need a 22 kW remote control? There is a second point: the level of power consumption of the equipment will vary depending on the time of day, day of the week. When distributing power, you need to take this moment into account and lay down ~ 10% for fluctuations and jumps, so that at the time of increased consumption, PDUs do not go into overload and do not leave equipment without power.
Rack consumption schedule 10 kW for 4 days.
It turns out that we have to add another 10,7% to the 10 kW we have, and as a result, the remote control for 11 kW is no longer suitable for us.
Remote control model
phase
Manufacturer's power, kVA
Power DTLN, kW
1 f
3,7
3
1 f
7,4
6
3 f
11
9
3 f
22
18
A fragment of the power table of specific PDU models according to the DataLine version. Taking into account the conversion from kVA to kW and the reserve for jumps during the day.
Mounting Features
The PDU works best when it is mounted vertically on the left and right side of the rack. In this case, it does not take up useful space. As standard, up to four PDUs can be installed in the rack - two on the left and two on the right. Most often they put one PDU on each side. Each PDU comes with one power input.
The standard “body kit” of the rack is 2 PDUs and 1 ATS.
Sometimes there is no room in the rack for vertical PDUs, for example if it is an open rack. Then horizontal PDUs come to the rescue. The only thing is that in this case you will have to put up with the loss of 2 to 4 units in the rack, depending on the PDU model.
Here the PDU ate 4 units. This type of PDU is also used when two clients in the same rack need to be separated. In this case, each client will have a separate pair of PDUs.
It happens that the rack was chosen not deep enough, and the server sticks out, blocking the PDU. The saddest thing here is not that some of the outlets will be idle, but that if such a PDU breaks down, you will have to bury it right in the rack, or turn off and pull out all the interfering equipment.
Don't do this - 1.
Don't do this - 2.
Connecting equipment
Even the most sophisticated PDU will not help if the equipment is connected incorrectly and there is no way to monitor consumption.
What can go wrong? A little . Each rack comes with two power inputs, the standard rack has two PDUs. It turns out that each PDU has its own input. If something happens to one of the inputs (read PDU), then the rack continues to live on the second one. For this scheme to work, you need to follow some rules. Here are the main ones (you will find a complete list ):
The equipment must be connected to different PDUs. If the equipment has one power supply and one plug, then it is connected to the PDU via ATS (automatic transfer switch) or ATS (Automatic Transfer Switch). In the event of a problem with one of the inputs or the PDU itself, the ATS switches the equipment to a healthy PDU/input. The equipment will not feel anything.
Dual load on two inputs/PDUs. The backup input will save only if it can withstand the load of the fallen input. To do this, you need to leave a margin: load each input less than half of the rated power, and the total load on the two inputs was less than 100% of the nominal. Only in this case the remaining bushing will withstand the double load. If this is not the case for you, then the focus with switching to the reserve will not take place - the equipment will remain without power. To avoid the worst, we this setting.
Load balancing between PDU sections. PDU outlets are combined into groups - sections. Usually 2 or 3 pieces. Each section has its own power limit. It is important not to exceed it and distribute the load evenly across all sections. Well, the story with paired loads, which was mentioned above, also works here.
Summarize
- If possible, choose a PDU with a monitoring function.
- When choosing a PDU model, leave a headroom for power.
- Mount the PDU so that it can be replaced without disturbing the IT equipment.
- Connect correctly: connect equipment to two PDUs, do not overload sections, and be mindful of pair loads.
Source: habr.com