The Tushino data center is a commercial retail half-megawatt data center for everyone and everything. The client can not only rent already installed equipment, but also place his own equipment there, including non-standard devices such as servers in conventional cases for desktop PCs, mining farms or artificial intelligence systems. Simply put, these are a variety of popular tasks most in demand by domestic businesses of varying degrees of magnitude. This is what makes him interesting. In this post you will not find exclusive technical solutions and the flight of engineering thought. We will talk about standard problems and solutions. That is, about what 90% of specialists have 90% of their working time.
Tier - the more the better?
The fault tolerance of the Tushino data center corresponds to the Tier II level. In essence, this means that the data center is located in a normal prepared room, redundant power supplies are used, and there are redundant system resources.
However, contrary to a common misconception, Tier levels do not characterize the "toughness" of the data center, but the degree of its compliance with actual business tasks. And among them there are many for which high fault tolerance is either insignificant or not so important as to overpay 20-25 thousand rubles a year for it, which in a crisis can be very painful for the customer.
Where did such an amount come from? It is she who makes the difference between the prices for placing information in Tier II and Tier III data centers in terms of one server. The more data, the greater the potential savings.
What tasks do you mean? For example, storing backups or mining cryptocurrency. In these cases, a downtime server allowed by Tier II will cost less than Tier III.
Practice shows that in most cases savings are more important than increased fault tolerance. There are only five Tier III certified data centers in Moscow. And there are no fully certified Tier IVs at all.
How is the Tushino data center power supply system arranged?
The requirements for the power supply system of the Tushino data center complies with Tier II level conditions. These are redundancy of power lines according to the N + 1 scheme, redundancy of uninterruptible power supplies according to the N + 1 scheme and redundancy of the diesel generator set according to the N scheme. N + 1 in this case means a scheme with one reserve element that remains idle until the system is not one of the main elements will fail, and N is a non-redundant scheme, in which the failure of any element leads to the cessation of the entire system
Many energy-related problems are solved by choosing the right location for the data center. The Tushino data center is located on the territory of the enterprise, where two 110 kV lines from different city power plants already come. On the equipment of the plant itself, high voltage is converted to medium voltage, and two independent 10 kV lines are fed to the input of the data center.
The transformer substation inside the data center building converts the medium voltage into consumer 240-400 V. All lines are run in parallel, so the data center equipment is powered by two independent external sources.
Low voltage from transformer substations is connected to automatic transfer switches, which provide switching between city networks. The motor drives installed on the ATS require 1,2 seconds for this operation. All this time, the load falls on the uninterruptible power supply.
A separate ATS is responsible for automatically turning on the diesel generator in the event that power is lost on both lines. Starting a diesel generator is not a quick process and requires about 40 seconds, during which the power supply is completely borne by the UPS batteries.
On a full charge, the diesel generator ensures the operation of the data center for 8 hours. With this in mind, the data center entered into two contracts with diesel fuel suppliers independent of each other, who undertook to deliver a new portion of fuel within 4 hours after the call. The likelihood that both of them will have some kind of force majeure at once is extremely low. Thus, autonomy can last as long as repair teams need to restore power from at least one of the city's networks.
As you can see, there are no engineering frills here. This is due, among other things, to the fact that when building the engineering infrastructure, ready-made modules were used, the manufacturers of which are guided by a certain βaverage consumerβ.
Of course, any IT professional will say that averaging is βneither fish nor fowlβ and will suggest developing a unique set of components for a particular system. However, those who want to pay for this pleasure are clearly not lining up. Therefore, you have to be realistic. In practice, everything will be exactly like this: the purchase of ready-made equipment and the assembly of a system that will solve business-relevant tasks. Those who disagree with this approach will quickly be brought back from heaven to earth by the chief financial officer of the enterprise.
Switchboards
At the moment, nine switchboards ensure the operation of input distribution devices and four switchboards are used directly to connect the load. There were no serious restrictions on the place, but there is never a lot of it, so one interesting engineering moment was still present.
As it is easy to see, the number of "input" and "load" shields does not match - the second is almost two times less. This became possible because the designers of the data center infrastructure decided to use large shields to bring three or more incoming lines there. For each input automaton, there are approximately 36 outlet lines protected by separate automata.
Thus, sometimes the use of larger models saves scarce space. Simply because large shields will require less.
Uninterruptible Power Supply
The Eaton 93PM with a capacity of 120 kVA, operating in double conversion mode, is used as an uninterruptible power supply at the Tushino data center.
Eaton 93PM UPSs are available in different versions. Photo: Eaton
The main reasons for choosing this particular device are its following characteristics.
Firstly, the efficiency of this UPS is up to 97% in double conversion mode and 99% in energy saving mode. The device occupies less than 1,5 square meters. m and does not take up server room space from the main equipment. The result is low operating costs and the savings your business needs.
Secondly, thanks to the built-in thermal management system, the Eaton 93PM UPS can be placed anywhere. Even close to the wall. Even if it is not needed right away, it may be needed later. For example, to free up some space that is not enough for an additional rack.
Third, ease of operation. Including - Intelligent Power software used for monitoring and control. The metrics transmitted via SNMP allow you to control consumption and some global failures, which makes it possible to quickly respond to emergencies.
Fourth, modularity and scalability. This is perhaps the most important quality, due to which only one modular UPS is used in the Tushino data center redundancy system. It includes two working modules and one redundant one. This provides the N+1 scheme required for the Tier II level.
This is much simpler and more reliable than a three-UPS configuration. Therefore, the choice of a device that initially provides for the possibility of parallel operation is a completely logical move.
But why didn't the designers choose DRIBP instead of a separate UPS and a diesel generator? The main reasons here lie not in engineering, but in finance.
The modular structure is a priori tailored for upgrades - as the load grows, sources and generators are added to the engineering infrastructure. At the same time, the old ones worked and still work. With DRIBP, the situation is radically different: you need to buy such a device with a large margin of power. In addition, there are few βsmall combinesβ, and they cost very decently - they are incomparably more expensive than individual diesel generators and UPSs. DRIBP is also very capricious in transportation and installation. This, in turn, also affects the cost of the entire system.
The existing configuration solves its tasks quite successfully. The Eaton 93PM UPS can keep key data center equipment running for 15 minutes, more than 15 times the power.
Again, the pure sine wave that the UPS provides online saves the owner of the data center from having to buy separate stabilizers. And this is where the savings come in.
Despite the declared simplicity of the Eaton 93PM UPS, the device is quite complex. Therefore, its maintenance at the Tushino data center is carried out by a third-party company that has highly qualified specialists on its staff. Keeping a trained employee on your own staff for this purpose is an expensive pleasure.
Results and prospects
This is how the data center was created, which allows providing high-quality services to consumers whose tasks do not require a high level of redundancy and do not imply large economic costs. Such a service will always be in demand.
With the already planned construction of the second stage, an already purchased Eaton UPS will be used to create a backup power supply system. Due to the modular design, its modernization will be reduced to the purchase of an additional module, which is more convenient and cheaper than a complete replacement of the device. This approach will be approved by both the engineer and the financier.
Source: habr.com