Somewhere in the middle between Moscow and St. Petersburg there is a small town Udomlya. Previously, it was known to the Kalinin NPP. In 2019, another attraction appeared nearby - the Udomlya megadata center for 4 racks.
After joining the Rostelecom-DPC team, DataLine specialists will also operate this data center. Surely you have already heard something about Udomlya. Today we decided to tell you in detail how everything works there.
Industrial landscapes: a 32 mΒ² data center and a nuclear power plant in the background. Udomlya sample of spring 000.
Under the cut, we collected more than 40 photos of the data center engineering systems with a detailed description. Those who make it to the end will be in for a pleasant surprise.
About logistics
The data center is located in the Tver region. The road from Moscow to Udomlya will take about three hours: 1 hour 45 minutes on Sapsan to Vyshny Volochek station, and from there, upon prior request, a shuttle will meet you and take you to the data center. From St. Petersburg to Vyshny Volochyok a little longer - 2 hours 20 minutes.
By car from Moscow can be reached in 4,5 hours, from St. Petersburg - in 5.
Yes, for the sake of a couple of units, you probably wonβt want to go here. But if you need a new home for dozens of racks, then you should take a closer look. There is enough space and electricity, even if you want to double this amount at any time. In Moscow, where, in our experience, data centers are booked at the construction stage, such a trick will not always work.
In addition, the location of the data center between Moscow and St. Petersburg can be used for geo-reservation. If the main facilities are in Moscow or St. Petersburg, then a reserve site will fit in well here.
The smart hands team will assist with all standard on-site operations. They will receive, unpack and install the equipment in racks, connect it to power and network, and provide remote access to the equipment. In case of malfunctions, they will help with diagnostics and replace failed components.
The first stage of the data center is 4 machine rooms, or modules, with 205 racks each. On the first floor there are 2 machine rooms and an energy center, on the second floor there are two more rooms and a cold center. Let's go see how it all works.
Physical security
The data center occupies a dedicated area, which cannot be entered without a pass and an identity document. Those who arrive by car also receive a transport pass and only after that they can enter the territory of the data center. For those who have everything in order with the pass, the data center is open 24x7.
The first post of round-the-clock security is the entrance to the territory.
We go further and get to the checkpoint directly at the entrance to the data center.
Security employees not only meet customers and issue passes, but also monitor the video wall around the clock, which displays an image of all the internal premises of the data center and surrounding areas.
Electric power supply
Electricity starts its way to the data center from the nuclear power plant. The data center receives 10 kV for 6 step-down transformers. Further - already 0,4 kV in two independent routes go to the low voltage switchgear (RUNN). Then, through the DIBP, power is supplied to IT and engineering equipment. There are two independent inputs to the rack, i.e. 2N redundancy. We will tell you more about how everything is arranged in terms of power supply in a separate article.
The path of electricity in the Udomlya data center
Power buses through which electricity comes from RUNN to DIBP power panels
Rows of RUNN
Despite the fact that there is a nuclear power plant nearby, in any reliable data center the main power supply is guaranteed. In our data centers, as you know, DGUs are responsible for it, but dynamic UPSs (DIBPs) are used here. They also provide uninterrupted power. DIBPs are reserved under the N+1 scheme.
DIBP brand Euro-Diesel (Kinolt) with a capacity of 2 MW. They roar so that itβs better not to go there without earplugs
And here's how it works. DIBP is a bundle of three main components: a diesel engine, a synchronous electric machine and a kinetic energy accumulator with a rotor. All of them are fixed on the main shaft.
The electric machine can work in the mode of the electric motor and the generator. When the DIBP is powered regularly from the city, the electric machine is an electric motor that turns the rotor and stores kinetic energy in the battery.
The gray block in the foreground is the DIBP synchronous machine
Diesel engine DIBP
If the city power is lost, the electric machine switches to generator mode. Thanks to the accumulated kinetic energy, the rotor makes the main shaft of the DIBP rotate, the electric machine continues to work without city power supply and the output voltage does not disappear. This ensures uninterrupted power supply in the data center. At the same time, the DIBP control system gives a signal to start the diesel engine. The same kinetic energy of the rotor starts the diesel engine and helps it reach its operating frequency. The rotor keeps the speed up to one minute, and this is enough for the diesel to come into play. After starting, the diesel engine already rotates the main shaft, and through it, the electric machine (here switching DIBP from one mode to another).
As a result, power in the racks does not disappear even for a second.
The tank of each diesel generator is designed for 3 hours. The data center also has its own fuel storage for 80 tons, which will allow keeping the entire load of the data center for 24 hours. In case of very hypothetical blackouts (the nuclear power plant nearby will not allow this), there are contracts with several contractors who will promptly deliver diesel fuel to the site on a call. In general, everything is as it should be.
Every week, the DIBPs self-test and start the diesel engine. Once a month, tests are carried out with a short-term shutdown of the city network.
DIBP control panel
Premises of ShCHP and ShchBP
"Highways" and "junctions" of power cables
Machine rooms
Each module is located in a containment area, in a special box. These additional walls and roof protect the machine room from dust, water and fire. When accepting a data center, the containment area is traditionally spilled with water to check for leaks.
The roof of the building and its own roof of the engine room with drainage pipes
Water that enters the roof of the containment area goes through the gutters into the drainage pipeline
Each hall is ready to receive 205 racks with an average power of 5 kW.
The arrangement of equipment in the hall is organized according to the scheme of cold and hot corridors.
The ceiling is wired for the Bolid early fire detection and gas extinguishing systems.
Smoke detectors are also located under the raised floor. It is enough to work with any two sensors, and the fire alarm siren will howl, but we'll talk about this a little later.
Immediately, along the rows of air conditioners, there are tape leakage sensors.
Each corridor between the racks is βshootableβ by CCTV cameras.
If desired, racks can be placed behind a special fence (cage) and additional cameras, access control systems and motion sensors, volume sensors, etc. can be installed on it.
Refrigeration
The Udomlya data center uses an ethylene glycol chiller circuit. There are air conditioners in the machine rooms, chillers on the roof, and on the second floor there is a cold center with pipelines, an automation and control system, pumps, storage tanks, etc.
Each room has 12 air conditioners, half of them with steam humidifiers. N+1 redundancy scheme.
The cold aisle is maintained at 21-25Β°C and 40-60% humidity.
Precision air conditioners Stulz Cyber ββCool
There are two rings around each machine room: a βcoldβ line that supplies cooled ethylene glycol to the air conditioner, and a βhotβ line that diverts heated glycol from air conditioners to chillers. If we open the raised floor in the corridor, we will see descents into the machine rooms from the refrigeration system.
The route of ethylene glycol is as follows: from the air conditioner, heated ethylene glycol first enters the return line around the machine room, and then into the common ring. Then the ethylene glycol goes to the pump and then to the chiller, where it is cooled down to 10 Β°C. After the chiller, the ethylene glycol returns to the air conditioner again through the supply line of the common ring, the accumulation tanks and the ring around the module.
Data center refrigeration scheme
This is how the cold center looks like, through which 100 m3 of ethylene glycol passes
Gray containers are expansion tanks. Heated ethylene glycol passes through them on its way to the chiller. In summer, ethylene glycol expands and requires additional space.
These impressive tanks are storage tanks, 5 m3 each. They provide uninterrupted cooling of the data center in case of chiller failure.
The cooled ethylene glycol from the tanks is fed into the system, and this keeps the air temperature at the outlet of the air conditioner at 19 Β°C for 5 minutes. Even if it is +40 Β°C outside.
Refrigeration pumps
Mesh pocket filters and separator tanks for cleaning ethylene glycol from mechanical particles and air
The thin red line on the floor under the pipes are the leak detectors. They go around the entire perimeter of the cold center.
If any of the pipes leak, then the ethylene glycol will leave through the drainage system and fall into a special tank in the water treatment room. There are also two tanks with "spare" ethylene glycol to feed the refrigeration system in case of large leaks.
And about chillers. There are 5 chillers on the roof with N+1 redundancy scheme. Every day, automation determines, depending on the operating time, which chiller to put on standby.
Chillers brand Stulz CyberCool 2 with a capacity of 1096 kW
Chillers support three modes:
- compressor - from 12 Β° C;
- mixed - at 0β12 Β° C;
- freecooling - from 0 and below. This mode cools the ethylene glycol with the fans instead of the compressor.
Fire safety
There are two gas fire extinguishing stations in the data center. Each has two batteries of 11 cylinders: the first is the main one, the second is a reserve.
The fire-fighting system of the data center is tied to the server of the Kalinin NPP, and, if necessary, the plant's own fire department will arrive at the site in minutes.
The photo shows a fire alarm system and an emergency exit button in the engine room. The latter is needed if the doors were not unlocked for some reason during a fire alarm: it breaks the power supply to the electric lock.
Telecom
Two Rostelecom highways come to the data center by independent routes. Each of the DWDM systems has a capacity of 8 Terabits.
The data center has two telecom inputs, which are located at a distance of more than 25 meters from each other.
Also on the site are operators Rascom, Telia Carrier Russia, Consist, and DataLine will appear in the near future.
From Udomlya, you can build a canal to Moscow, St. Petersburg or anywhere in Russia and the world.
Monitoring
Engineers on duty are on duty at the monitoring center around the clock.
All information on engineering systems comes here: climatic conditions in the hall, the state of inputs, DIBP, etc.
Every two hours, duty officers make a tour of all infrastructure facilities to inspect the condition of engineering and IT equipment.
Support infrastructure
An unloading area is provided for the delivery of equipment to the data center.
Unloading area from the inside.
If your hall is on the second floor, then this hydraulic forklift will deliver any equipment there.
Lockers for storing client tools, and more.
A little about life
For permanent staff, you can rent equipped workplaces in the office part. If you run into from time to time, you can stay in a temporary hotel with all amenities right on the territory of the data center.
The office part also has a dining room and a kitchen.
And there is awesome nature all around with forests, lakes, rivers, fishing and other outdoor activities. Come for a visit.
As promised, a nice bonus for those who made it to the end. For the first six months, the rental of a rack space in Udomlya with a connected power of 5 kW will be free. Pay only for the actual electricity consumed. Send an application to .
Source: habr.com