The "Tushino" data center is a commercial retail half-megawatt facility designed for everyone and everything. Clients can not only rent already installed equipment but also place their own, including unconventional devices like servers in standard desktop cases, mining rigs, or artificial intelligence systems. Essentially, it caters to various popular tasks that are in high demand by domestic businesses of all sizes. This is why it is interesting. In this article, you won’t find exclusive technical solutions or engineering feats. We will discuss standard problems and solutions, meaning the everyday tasks that occupy 90% of specialists' working time.
Is Tier — the higher, the better?
The resilience of the "Tushino" data center corresponds to Tier II. This essentially means that the data center is located in a properly prepared building, utilizes backup power sources, and has redundant system resources.
However, contrary to popular belief, Tier levels do not characterize the "coolness" of a data center, but rather its alignment with current business needs. Among these, there are many tasks for which high resilience is either insignificant or not important enough to justify paying an extra 20,000-25,000 rubles per year, which can be quite burdensome for clients during a crisis.
Where did this figure come from? It represents the difference between the costs of information hosting in data centers Tier II and Tier III when calculated per server. The larger the data sets, the more noticeable the potential savings.
What tasks are we talking about? For instance, backing up data or mining cryptocurrency. In these cases, the acceptable Tier II downtime will be cheaper than Tier III. serverPractical experience shows that in most cases, saving money is more important than increased resilience. In Moscow, there are only five data centers certified to Tier III. There are none that are fully certified to Tier IV.
How is the power supply system of the "Tushino" data center structured?
How is the power supply system of the "Tushino" data center structured?
The power supply system requirements for the Data Center "Tushino" meet Tier II level conditions. This includes N+1 redundancy for power supply lines, N+1 redundancy for uninterruptible power supply sources, and N redundancy for the diesel generator setup. In this case, N+1 means a schema with one backup element that remains unused until a failure occurs in one of the primary components, while N refers to a non-redundant schema where a failure in any component leads to the cessation of the entire system's operation.
Many energy supply-related issues are resolved by choosing the right location for a data center. The "Tushino" data center is situated on the premises of an enterprise that is already served by two 110 kV lines from different urban power plants. At the factory's equipment, high voltage is transformed into medium voltage, allowing two independent lines of 10 kV to enter the data center.
The transformer substation within the data center building converts medium voltage into consumer-level 240-400 V. All lines are run in parallel, so the data center equipment receives power from two independent external sources.
Low voltage from the transformer substations is fed into the reserve input circuit breakers, which facilitate switching between the urban networks. The motor drives installed on the automatic transfer switches (ATS) require 1.2 seconds for this operation. During this time, the load is placed on the uninterruptible power supply sources.
A separate ATS is responsible for the automatic activation of the diesel generator in the event of power loss on both lines. The startup of the diesel generator is not a fast process and takes about 40 seconds, during which time the power supply is entirely reliant on the UPS batteries.
At full capacity, the diesel generator ensures data center operation for 8 hours. Considering this, the data center has signed two contracts with independent fuel suppliers, who are committed to delivering a new supply of fuel within 4 hours after a call. The likelihood that both will face any force majeure circumstances simultaneously is extremely low. Thus, the autonomy can last as long as needed for repair teams to restore power from at least one of the municipal networks.
It is not hard to notice that there are no engineering findings here. This is partly due to the fact that ready-made modules were used in the construction of the engineering infrastructure, which manufacturers target at some 'average consumer'.
Undoubtedly, any IT specialist will say that averaging is 'neither fish nor fowl' and will suggest developing a unique set of components for a specific system. However, those willing to pay for such pleasure clearly do not line up. Therefore, one must be realistic. In practice, this is how things will be: purchasing ready-made equipment and assembling a system that addresses the current business needs. Those who disagree with this approach will quickly be brought back down to earth by the company's chief financial officer.
Distribution Panels
Currently, nine distribution panels provide the operation of input-distribution devices, and four panels are directly used to connect loads. There were no serious space limitations, but there is never enough space, so one interesting engineering aspect was still present.
It is noticeable that the number of 'input' and 'load' panels does not match—the latter are almost half as many. This became possible because the data center infrastructure designers decided to use larger panels to accommodate three or more incoming lines. Each incoming circuit breaker has about 36 outgoing lines, protected by separate circuit breakers.
Thus, sometimes using larger models allows saving scarce space. Simply because fewer large panels are needed.
Uninterruptible Power Supplies
The data center 'Tushino' uses the Eaton 93PM uninterruptible power supply with a capacity of 120 kVA, operating in double conversion mode.
Eaton 93PM UPS units are available in various configurations. Photo: Eaton
The main reasons for choosing this device are based on its following characteristics.
First, the efficiency of this UPS in double conversion mode reaches 97%, and in energy-saving mode — 99%. The device occupies less than 1.5 square meters and does not take up server space from the main equipment. As a result, low operational costs and the necessary savings for the business are achieved.
Second, thanks to the built-in thermal regulation system, the Eaton 93PM UPS can be placed anywhere, even right next to a wall. Even if it's not needed right away, it may be required later. For example, to free up some space that is lacking for an additional rack.
Third, ease of use. This includes the Intelligent Power software used for monitoring and management. Metrics provided via SNMP allow for monitoring consumption and any significant failures, enabling quick response to emergencies.
Fourth, modularity and scalability. This is perhaps the most important quality, which is why only one modular UPS is used in the Tushino data center's backup system. It consists of two working modules and one spare. This ensures the N+1 configuration required for Tier II level.
This is significantly simpler and more reliable than a configuration of three UPS units. Therefore, the choice of a device that was originally designed for parallel operation is quite logical.
But why did the designers not choose a combined UPS instead of separate UPS units and a diesel generator? The main reasons here lie not in engineering, but in finance.
The modular structure is inherently designed for upgrades — as the load on the engineering infrastructure increases, sources and generators are added. The older ones continue to function as they always have. The situation with DRUPS is fundamentally different: purchasing such a device requires a significant power reserve. Moreover, there are very few 'small combine harvesters,' and they are quite expensive — incomparably more than individual diesel generators and UPSs. DRUPS are also very finicky in terms of transportation and installation, which in turn affects the overall cost of the system.
The existing configuration successfully addresses its tasks. The Eaton 93PM UPS can support the operation of the primary data center equipment for 15 minutes, which means more than a 15-fold reserve.
Once again, the pure sine wave produced by the UPS in online mode saves the data center owner from the need to purchase separate stabilizers. And this results in savings.
Despite the claimed simplicity, the Eaton 93PM UPS is quite a complex device. Therefore, its maintenance at the 'Tushino' data center is handled by an external company with highly qualified specialists. Keeping a trained staff member for this purpose in-house is an expensive proposition.
Results and Prospects
This is how a data center was created that allows for the provision of quality services to consumers whose tasks do not require a high level of redundancy and do not entail significant economic costs. Such a service will always be in demand.
In the already planned construction of the second phase, the previously acquired Eaton UPS will be used to create the backup power supply system. Thanks to its modular design, upgrading it will only require purchasing an additional module, which is more convenient and cheaper than completely replacing the device. Both the engineer and the financier will approve this approach.
Source: habr.com
