How we built a virtual infrastructure for cyber-exercise of industrial enterprises

How we built a virtual infrastructure for cyber-exercise of industrial enterprises

This year, we launched a large project to create a cyber training ground, a platform for cyber exercises for companies from various industries. To do this, it is necessary to create virtual infrastructures that are "identical to natural ones" - so that they repeat the typical internal structure of a bank, an energy company, etc., and not only in the corporate network segment. A little later we will talk about the banking and other infrastructures of the cyberpolygon, and today we will talk about how we solved this problem in relation to the technological segment of an industrial enterprise.

Of course, the topic of cyber exercises and cyber training grounds did not arise yesterday. In the West, a circle of competing proposals, various approaches to cyber exercises, as well as simply best practices, has long been formed. The "good form" of the information security service is to periodically practice readiness to repel cyber attacks in practice. For Russia, this is still a new topic: yes, there is a small supply, and it arose several years ago, but demand, especially in industrial sectors, has begun to gradually form only now. We believe there are three main reasons for this - they are problems that have already become very obvious.

The world is changing too fast

Even 10 years ago, hackers attacked mainly those organizations from which they could quickly withdraw money. For industry, this threat was less relevant. Now we see that the infrastructure of state organizations, energy and industrial enterprises are also becoming a subject of their interest. Here we often deal with espionage attempts, data theft for various purposes (competitive intelligence, blackmail), as well as obtaining points of presence in the infrastructure for further sale to interested comrades. Well, even banal ransomware like WannaCry caught a lot of similar objects around the world. Therefore, modern realities require information security specialists to take into account these risks and form new information security processes. In particular, regularly improve their skills and develop practical skills. Personnel at all levels of operational dispatch control of industrial facilities must have a clear understanding of what actions to take in the event of a cyber attack. But to conduct cyber exercises on your own infrastructure - thank you, the risks clearly outweigh the possible benefits.

Lack of understanding of the real capabilities of attackers to hack ICS and IIoT systems

This problem exists at all levels of organizations: not even all specialists understand what can happen to their system at all, what are the attack vectors on it. What can we say about leadership.

Security people often refer to an "air gap" that supposedly will not allow an attacker to go beyond the corporate network, but practice shows that in 90% of organizations there is a connection between the corporate and technology segments. At the same time, the elements of building and managing technological networks themselves also often have vulnerabilities, which, in particular, we saw when examining equipment MOXA ΠΈ Schneider Electric.

It is difficult to build an adequate threat model

In recent years, the process of increasing the complexity of information and automated systems, as well as the transition to cyber-physical systems, involving the integration of computing resources and physical equipment, has been constantly going on. Systems are becoming so complex that it is simply impossible to predict all the consequences of cyberattacks using analytical methods. We are talking not only about the economic damage to the organization, but also about the assessment of the consequences that are understandable for the technologist and for the industry - undersupply of electricity, for example, or another type of product, if we are talking about oil and gas or petrochemistry. And how to set priorities in such a situation?

Actually, all this, in our opinion, became the prerequisites for the emergence of the concept of cyber exercises and cyber training grounds in Russia.

How the technological segment of the cyberpolygon works

A cyberpolygon is a complex of virtual infrastructures that replicate the typical infrastructures of enterprises in various industries. It allows you to "train on cats" - to work out the practical skills of specialists without the risk that something will not go according to plan, and cyber exercises will damage the activities of a real enterprise. Large information security companies are starting to develop this area, and you can look at such cyber exercises in a game format, for example, at Positive Hack Days.

A typical scheme of the network infrastructure of a conditional large enterprise or corporation is a fairly standard set of servers, work computers and various network devices with a typical set of corporate software and information security systems. The industry cyberpolygon is all the same, plus serious specifics that dramatically complicate the virtual model.

How we brought the cyberpolygon closer to reality

Conceptually, the appearance of the industrial part of the cyberpolygon depends on the chosen method for modeling a complex cyberphysical system. There are three main approaches to modeling:

How we built a virtual infrastructure for cyber-exercise of industrial enterprises

Each of these approaches has its own advantages and disadvantages. In different cases, depending on the ultimate goal and existing restrictions, all three of the above modeling methods can be used. In order to formalize the choice of these methods, we have compiled the following algorithm:

How we built a virtual infrastructure for cyber-exercise of industrial enterprises

The pros and cons of different modeling methods can be presented in the form of a diagram, where the y-axis is the coverage of the study areas (i.e. the flexibility of the proposed modeling tool), and the abscissa is the accuracy of the simulation (the degree of correspondence to the real system). It turns out almost Gartner's square:

How we built a virtual infrastructure for cyber-exercise of industrial enterprises

Thus, the so-called semi-natural modeling (hardware-in-the-loop, HIL) is optimal in terms of the ratio of accuracy and flexibility of modeling. Within the framework of this approach, the cyber-physical system is partly modeled using real equipment, and partly using mathematical models. For example, an electrical substation can be represented by real microprocessor devices (relay protection terminals), servers of automated control systems and other secondary equipment, and the physical processes occurring in the electrical network themselves can be implemented using a computer model. Okay, we have decided on the modeling method. After that, it was necessary to develop the architecture of the cyberpolygon. For cyber exercises to be truly useful, all the interconnections of a real complex cyber-physical system must be recreated as accurately as possible on the test site. Therefore, in our country, as in real life, the technological part of the cyberpolygon consists of several interacting levels. Let me remind you that a typical industrial network infrastructure includes the lowest level, which includes the so-called β€œprimary equipment” - this is fiber optic, electrical network or something else - depending on the industry. It exchanges data and is controlled by specialized industrial controllers, and those, in turn, by SCADA systems.

We started the creation of the industrial part of the cyberpolygon from the energy segment, which is now a priority for us (we plan to include the oil and gas and chemical industries).

It is obvious that the level of primary equipment cannot be realized through full-scale simulation using real objects. Therefore, at the first stage, we developed a mathematical model of the power facility and the adjacent section of the power system. This model includes all the power equipment of substations - power lines, transformers, and so on, and is performed in a special software package RSCAD. The model created in this way can be processed by a real-time computing complex - its main feature is that the process time in a real system and the process time in the model are absolutely identical - that is, if a short circuit lasts two seconds in a real network, exactly the same time will be simulated in RSCAD). We get a "live" section of the electric power system, functioning according to all the laws of physics and even reacting to external influences (for example, operation of relay protection and automation terminals, switching off switches, etc.). Interaction with external devices was achieved using specialized customizable communication interfaces that allow the mathematical model to interact with the level of controllers and the level of automated systems.

But already the levels of controllers and automated control systems of a power facility can be created using real industrial equipment (although, if necessary, we can also use virtual models). At these two levels, respectively, controllers and automation equipment (RPA, PMU, USPD, meters) and automated control systems (SCADA, OIK, AIISKUE) are located. Full-scale simulation can significantly increase the realism of the model and, accordingly, the cyber exercises themselves, since the teams will interact with real industrial equipment, which has its own characteristics, bugs and vulnerabilities.

At the third stage, we implemented the interaction of the mathematical and physical parts of the model using specialized hardware and software interfaces and signal amplifiers.

As a result, the infrastructure looks something like this:

How we built a virtual infrastructure for cyber-exercise of industrial enterprises

All equipment of the test site interacts with each other in the same way as in a real cyber-physical system. More specifically, when building this model, we used the following equipment and computing tools:

  • Computing complex RTDS for real-time calculation;
  • Automated workstation (AWS) of the operator with installed software for modeling the technological process and primary equipment of electrical substations;
  • Cabinets with communication equipment, RPA terminals, and APCS equipment;
  • Amplifier cabinets designed to amplify analog signals from the digital-to-analog converter board of the RTDS simulator. Each amplifier cabinet contains a different set of amplification blocks used to generate current and voltage input signals for the studied RPA terminals. The input signals are amplified to the level required for normal operation of the RPA terminals.

How we built a virtual infrastructure for cyber-exercise of industrial enterprises

This is not the only possible solution, but, in our opinion, it is optimal for conducting cyber exercises, as it reflects the real architecture of the vast majority of modern substations, and at the same time it can be customized in such a way as to recreate some features of a particular object as accurately as possible.

In conclusion

Cyberpolygon is a huge project, and there is still a lot of work ahead. On the one hand, we study the experience of our Western colleagues, on the other hand, we have to do a lot based on our experience of working with Russian industrial enterprises, since not only different industries, but also different countries have specifics. This is both a complex and interesting topic.
Nevertheless, we are convinced that we in Russia have reached, as they say, a β€œlevel of maturity”, when the industry also understands the need for cyber exercises. This means that soon the industry will have its own best practices, and we hope to strengthen the level of security.

Authors

Oleg Arkhangelsky, Lead Analyst and Methodologist of the Industrial Cyber ​​Polygon project.
Dmitry Syutov, chief engineer of the project "Industrial cyberpolygon";
Andrey Kuznetsov, Project Manager "Industrial Cyber ​​Testing Ground", Deputy Head of the Cybersecurity Laboratory for Industrial Control Systems for Production

Source: habr.com

Add a comment