Introduction to Operating Systems
Hey Habr! I would like to bring to your attention a series of articles-translations of one interesting literature in my opinion - OSTEP. This material discusses quite deeply the work of unix-like operating systems, namely, work with processes, various schedulers, memory and other similar components that make up a modern OS. You can see the original of all materials here . Please note that the translation was made unprofessionally (quite freely), but I hope I retained the general meaning.
Lab work on this subject can be found here:
- original:
- original:
- my personal adaptation:
You can also check out my channel at =)
Program operation
What happens when a program is running? A running program does one simple thing - it executes instructions. Every second, millions and even possibly billions of instructions are retrieved by the processor from RAM, in turn it decodes them (for example, it recognizes what type these instructions belong to) and executes them. This could be adding two numbers, accessing memory, checking a condition, jumping to a function, and so on. After the execution of one instruction, the processor proceeds to the execution of another. And so instruction after instruction, they are executed until the program ends.
This example is naturally considered simplified - in fact, to speed up the processor, modern hardware allows you to execute instructions out of turn, calculate possible results, execute instructions simultaneously, and similar tricks.
Von Neumann model of computation
The simplified form of work described by us is similar to the Von Neumann model of computation. Von Neumann is one of the pioneers of computer systems, he is also one of the authors of game theory. While the program is running, a bunch of other events take place, many other processes and third-party logic work, the main purpose of which is to simplify the launch, operation and maintenance of the system.
There is a set of software that is responsible for making programs easy to run (or even allowing multiple programs to run at the same time), that allows programs to share the same memory, and to communicate with different devices. Such a set of software (software) is essentially called the operating system and its tasks include monitoring that the system works correctly and efficiently, as well as ensuring ease of management of this system.
Operating system
An operating system, abbreviated as an OS, is a set of interrelated programs designed to manage computer resources and organize user interaction with a computer..
The OS achieves its effectiveness in the first place, through the most important technique - the technique virtualization. The OS interacts with a physical resource (processor, memory, disk, etc.) and transforms it into a more general, more powerful, and easier-to-use form of itself. Therefore, for a general understanding, you can very roughly compare the operating system with a virtual machine.
In order to allow users to give commands to the operating system and thus use the capabilities of the virtual machine (such as running a program, allocating memory, accessing a file, and so on), the operating system provides some interface called API (application programming interface) and to which you can make calls (call). A typical operating system allows hundreds of system calls to be made.
Finally, since virtualization allows multiple programs to run (thus sharing the CPU), and simultaneously access their instructions and data (thus sharing memory), and access disks (thus sharing I/O devices). ), the operating system is also called a resource manager. Each processor, disk and memory is a resource of the system, and thus one of the roles of the operating system becomes the task of managing these resources, doing it efficiently, honestly, or vice versa, depending on the task for which this operating system is designed.
CPU virtualization
Consider the following program:
(https://www.youtube.com/watch?v=zDwT5fUcki4&feature=youtu.be)
It does not perform any special actions, in fact, all it does is call a function spin moves(), whose task is to cycle through the time check and return after one second has passed. Thus, it repeats indefinitely the string that the user passed as an argument.
Let's run this program and pass it the character "A" as an argument. The result is not particularly interesting - the system simply executes a program that periodically displays the character "A".
Now let's try the option when many instances of the same program are running, but outputting different letters to make it clearer. In this case, the result will be somewhat different. Despite the fact that we have one processor, the program is executed simultaneously. How does it happen? But it turns out that the operating system, not without the help of hardware capabilities, creates an illusion. The illusion that the system has multiple virtual processors, turning a single physical processor into a theoretically infinite number and thereby allowing seemingly programs to run simultaneously. This illusion is called CPU virtualization.
This picture raises many questions, for example, if several programs want to run at the same time, which one will be launched? The “policies” of the OS are responsible for this question. Policies are used in many places in the OS and answer questions like this, and are the basic mechanisms that the OS implements. Hence the role of the OS as a resource manager.
Memory virtualization
Now let's look at memory. The physical model of memory in modern systems is represented as an array of bytes.. To read from memory, you need to specify cell addressto access it. To write or update data, you must also specify the data and the address of the cell where to write it.
Memory is accessed constantly during program execution. A program stores its entire data structure in memory and accesses it by executing various instructions. The instructions, meanwhile, are also stored in memory, so it is also accessed for each request for the next instruction.
malloc() call
Consider the following program, which allocates a region of memory using the call malloc () (https://youtu.be/jnlKRnoT1m0):
The program does several things. First, it allocates some memory (line 7), then prints the address of the allocated cell (line 9), writes zero to the first slot of the allocated memory. Next, the program enters a loop in which it increments the value stored in memory at the address in the “p” variable. It also prints the process ID of itself. The process ID is unique for each running process. Having launched several copies, we will stumble upon an interesting result: In the first case, if you do nothing and just run several copies, then the addresses will be different. But this does not fall under our theory! Correct, since modern distributions have memory randomization enabled by default. If it is disabled, we get the expected result - the memory addresses of two simultaneously running programs will match.
As a result, it turns out that two independent programs work with their own private address spaces, which in turn are mapped by the operating system in physical memory. Therefore, the use of memory addresses within one program will not affect others in any way, and it seems to each program that it has its own piece of physical memory, entirely given to it. The reality, however, is that physical memory is a shared resource managed by the operating system.
Consistency
Another of the important topics within operating systems is − consistency. This term is used when talking about problems in the system that can occur when working with many things at the same time within the same program. Consistency issues arise even within the operating system itself. In the previous memory and processor virtualization examples, we realized that the OS manages many things at the same time - it starts the first process, then the second, and so on. As it turned out, this behavior can lead to some problems. So, for example, modern multi-threaded programs experience such difficulties.
Consider the following program:
The program in the main function creates two threads using the call pthread_create(). In this example, a thread can be thought of as a function running in the same memory space alongside other functions, with clearly more than one function running at the same time. In this example, each thread starts and executes the function worker() which in turn simply increments the variable,.
Let's run this program with an argument of 1000. As you might have guessed, the result should be 2000 because each thread incremented the variable 1000 times. However, everything is not so simple. Let's try to run the program with an order of magnitude more repetitions.
By inputting a number, for example, 100000, we expect to see the output as the number 200000. However, if we run the number 100000 several times, we will not only not see the correct answer, but also get different incorrect answers. The answer lies in the fact that to increase the number, three operations are required - extracting the number from memory, incrementing, and then writing the number back. Since all these instructions are not executed atomically (all at the same time), strange things like this can happen. This problem is called in programming race condition. When unknown forces at an unknown moment can affect the performance of any of your operations.
Source: habr.com