I publish on Habr the original article, the translation of which is posted in the corporate .
The need to do something asynchronously, without waiting for the result here and now, or to divide a lot of work between several units that perform it, existed before the advent of computers. With their appearance, this need has become very tangible. Now, in 2019, typing this article on a laptop with an 8-core Intel Core processor, on which not one hundred processes, but even more threads are running in parallel. Nearby, there is a slightly shabby phone, bought a couple of years ago, it has an 8-core processor on board. Thematic resources are full of articles and videos where their authors admire the flagship smartphones of this year, where they put 16-core processors. MS Azure provides a virtual machine with 20 cores and 128 TB of RAM for less than $2/hour. Unfortunately, it is impossible to extract the maximum and curb this power without being able to manage the interaction of threads.
Vocabulary
Process - OS object, isolated address space, contains threads.
Thread - OS object, the smallest unit of execution, part of the process, threads share memory and other resources among themselves within the process.
Multitasking - property of the OS, the ability to execute several processes at the same time
Multi-core - property of the processor, the ability to use multiple cores for data processing
multiprocessing - a property of a computer, the ability to simultaneously work with several processors physically
Multithreading - a property of the process, the ability to distribute data processing between several threads.
Parallelism - performing several actions physically simultaneously in a unit of time
Asynchrony - execution of the operation without waiting for the completion of this processing, the result of the execution can be processed later.
Metaphor
Not all definitions are good and some need further explanation, so I will add a metaphor about cooking breakfast to the formally introduced terminology. Making breakfast in this metaphor is a process.
Preparing breakfast in the morningCPU) come to the kitchen (Компьютер). I have 2 handsColors). The kitchen has a number of appliances (IO): oven, kettle, toaster, refrigerator. I turn on the gas, put a frying pan on it and pour oil into it, without waiting until it warms up (asynchronously, Non-Blocking-IO-Wait), I take the eggs out of the refrigerator and break them into a plate, after which I beat them with one hand (Thread#1), and second (Thread#2) hold the plate (Shared Resource). Now I would still turn on the kettle, but there are not enough hands (Thread Starvation) During this time, the frying pan is heated (Processing the result) where I pour what I whipped. I reach for the kettle and turn it on and stupidly watch the water boil in it (Blocking-IO-Wait), although during this time I could wash the plate where I whipped the omelette.
I cooked an omelette using only 2 hands, and I don’t have more, but at the same time, at the time of whipping the omelet, 3 operations took place at once: whipping the omelette, holding the plate, heating the pan. The CPU is the fastest part of the computer, IO is what is more often slows down everything, therefore often an effective solution is to occupy the CPU with something while data is being received from IO.
Continuing the metaphor:
- If in the process of cooking an omelet, I would also try to change clothes, this would be an example of multitasking. An important nuance: computers are much better at this than people.
- A kitchen with multiple chefs, such as a restaurant, is a multi-core computer.
- Many restaurants at the food court in the mall - data center
.NET Tools
In threading, as in many other things, .NET is good. With each new version, it introduces more and more new tools for working with them, new layers of abstraction over OS threads. When working with building abstractions, framework developers use an approach that leaves the possibility, when using a high-level abstraction, to go down one or more levels below. Most of the time this isn't necessary, moreover, it opens up the possibility of shooting yourself in the foot with a shotgun, but sometimes, in rare cases, it may be the only way to solve a problem that doesn't solve at the current level of abstraction.
By tools I mean both programming interfaces (APIs) provided by the framework and third-party packages, as well as a whole software solution that makes it easier to find any problems associated with multi-threaded code.
Starting a thread
The Thread class is the most basic threading class in .NET. The constructor accepts one of two delegates:
- ThreadStart - No parameters
- ParametrizedThreadStart - with one parameter of type object.
The delegate will be executed in the newly created thread after calling the Start method, if a delegate of the ParametrizedThreadStart type was passed to the constructor, then an object must be passed to the Start method. This mechanism is needed to pass any local information to the stream. It is worth noting that creating a thread is an expensive operation, and the thread itself is a heavy object, at least because it allocates 1MB of memory per stack and requires interaction with the OS API.
new Thread(...).Start(...);
The ThreadPool class represents the concept of a pool. In .NET, the thread pool is a piece of engineering and the developers at Microsoft have put a lot of effort into making it work optimally in a wide variety of scenarios.
General concept:
From the moment of launch, the application creates several threads in the background in reserve and provides the opportunity to take them for use. If threads are used frequently and in large numbers, then the pool expands to meet the needs of the calling code. When there are no free threads in the pool at the right time, it will either wait for one of the threads to return, or create a new one. It follows from this that the thread pool is great for some short actions and poorly suited for operations running as services throughout the application.
To use a thread from the pool, there is a QueueUserWorkItem method that takes a delegate of the WaitCallback type, which matches the signature of ParametrizedThreadStart, and the parameter passed to it performs the same function.
ThreadPool.QueueUserWorkItem(...);
A lesser-known thread pool method, RegisterWaitForSingleObject, is used to organize non-blocking IO operations. The delegate passed to this method will be called when the WaitHandle passed to the method is Released.
ThreadPool.RegisterWaitForSingleObject(...)
.NET has a thread timer and it differs from WinForms/WPF timers in that its handler will be called on a thread taken from the pool.
System.Threading.Timer
There is also a rather exotic way to send a delegate for execution to a thread from the pool - the BeginInvoke method.
DelegateInstance.BeginInvoke
I also want to briefly dwell on the function that calls many of the above methods - CreateThread from Kernel32.dll Win32 API. There is a way, thanks to the mechanism of extern methods, to call this function. I saw such a call only once in the most terrible example of legacy code, and the motivation of the author who did just that is still a mystery to me.
Kernel32.dll CreateThread
Viewing and Debugging Threads
Threads created by you, all third-party components, and the .NET pool can be viewed in the Threads window of Visual Studio. This window will display information about threads only when the application is under debugging and in break mode. Here you can conveniently view the stack names and priorities of each thread, switch debugging to a specific thread. With the Priority property of the Thread class, you can set the priority of the thread, which the OC and CLR will take as a recommendation when dividing processor time between threads.
Task Parallel Library
The Task Parallel Library (TPL) was introduced in .NET 4.0. Now it is the standard and the main tool for working with asynchrony. Any code that uses older approaches is considered legacy. The basic unit of the TPL is the Task class from the System.Threading.Tasks namespace. Task is an abstraction over a thread. With the new version of the C# language, we got an elegant way to work with Tasks - async / await statements. These concepts made it possible to write asynchronous code as if it were simple and synchronous, this made it possible even for people with little understanding of the inner workings of threads to write applications that use them, applications that do not freeze when performing long operations. Using async/await is a topic for one or more articles, but I'll try to get the point across in a few sentences:
- async is a modifier for a method returning Task or void
- and await is Task's non-blocking wait operator.
Once again: the await operator, in the general case (there are exceptions), will release the current thread of execution further, and when the Task finishes its execution, and the thread (in fact, it’s more correct to say the context, but more on that later) will be free to continue executing the method further. Inside .NET, this mechanism is implemented in the same way as yield return, when the written method turns into a whole class, which is a state machine and can be executed in separate pieces depending on these states. Anyone who is interested can write any simple code using async / await, compile and view the assembly using JetBrains dotPeek with Compiler Generated Code enabled.
Consider options for launching and using Task. In the example code below, we create a new task that does nothing useful (Thread.Sleep(10000)), but in real life it must be some complex CPU-intensive work.
using TCO = System.Threading.Tasks.TaskCreationOptions;
public static async void VoidAsyncMethod() {
var cancellationSource = new CancellationTokenSource();
await Task.Factory.StartNew(
// Code of action will be executed on other context
() => Thread.Sleep(10000),
cancellationSource.Token,
TCO.LongRunning | TCO.AttachedToParent | TCO.PreferFairness,
scheduler
);
// Code after await will be executed on captured context
}
Task is created with a number of options:
- LongRunning is a hint that the task will not be completed quickly, which means that you should probably think about not taking a thread from the pool, but creating a separate one for this Task so as not to harm the rest.
- AttachedToParent - Tasks can line up in a hierarchy. If this option has been used, then the Task may be in a state where it has itself completed and is waiting for its children to complete.
- PreferFairness - means that it would be nice to execute Tasks sent for execution earlier before those that were sent later. But this is just a recommendation and the result is not guaranteed.
The second parameter passed to the method is CancellationToken. To correctly handle the cancellation of an operation after it has started, the code being executed must be filled with checks on the state of the CancellationToken. If there are no checks, then the Cancel method called on the CancellationTokenSource object will be able to stop the execution of the Task only before it starts.
The last parameter is a scheduler object of type TaskScheduler. This class and its descendants are designed to control strategies for distributing Tasks across threads, by default Task will be executed on a random thread from the pool.
The await operator is applied to the created Task, which means that the code written after it, if any, will be executed in the same context (often this means on the same thread) as the code before the await.
The method is marked as async void, which means that it is allowed to use the await operator, but the calling code will not be able to wait for execution. If this capability is required, then the method must return a Task. Methods marked async void are quite common: as a rule, these are event handlers or other methods that work on the principle of fire and forget. If it is necessary not only to give the opportunity to wait for the completion of the execution, but also to return the result, then it is necessary to use Task.
On the Task that the StartNew method returned, however, as on any other, you can call the ConfigureAwait method with the false parameter, then execution after await will continue not on the captured context, but on an arbitrary one. This should be done whenever the execution context is not important for the code after the await. It is also a recommendation from MS when writing code that will be delivered packaged in a library.
Let's dwell a little more on how you can wait for the completion of the Task'i. Below is a code example, with comments on when the wait is conditionally good and when conditionally bad.
public static async void AnotherMethod() {
int result = await AsyncMethod(); // good
result = AsyncMethod().Result; // bad
AsyncMethod().Wait(); // bad
IEnumerable<Task> tasks = new Task[] {
AsyncMethod(), OtherAsyncMethod()
};
await Task.WhenAll(tasks); // good
await Task.WhenAny(tasks); // good
Task.WaitAll(tasks.ToArray()); // bad
}
In the first example, we wait for the Task to complete without blocking the calling thread, we will return to processing the result only when it is already there, until then the calling thread is left to itself.
In the second option, we block the calling thread until the result of the method is calculated. This is bad not only because we have occupied the thread, such a valuable program resource, with simple idleness, but also because if there is await in the code of the method that we are calling, and the synchronization context involves returning to the calling thread after await, then we will get a deadlock : the calling thread is waiting for the result of the asynchronous method to be evaluated, the asynchronous method tries in vain to continue its execution in the calling thread.
Another disadvantage of this approach is complicated error handling. The fact is that errors in asynchronous code when using async / await are very easy to handle - they behave in the same way as if the code were synchronous. Whereas if we apply a synchronous wait exorcism to a Task, the original exception is wrapped in an AggregateException, i.e. To handle the exception, you will have to explore the InnerException type and write the if chain yourself inside one catch block or use the catch when construct, instead of the catch block chain more familiar in the C # world.
The third and last examples are also marked bad for the same reason and contain all the same problems.
The WhenAny and WhenAll methods are extremely convenient when waiting for a group of Tasks, they wrap a group of Tasks into one that will work either on the first Task from the group, or when everyone has finished their execution.
Stopping Threads
For various reasons, it may be necessary to stop a thread after it has started. There are a number of ways to do this. The Thread class has two methods with appropriate names - these are Abortion и Interrupt. The first is highly discouraged for use, because. after its call at any random moment, during the processing of any instruction, an exception will be thrown ThreadAbortedException. You don't expect such an exception to be thrown when an integer variable is incremented, right? And when using this method, this is a very real situation. If you need to prevent the CLR from throwing such an exception in a certain section of code, you can wrap it in calls Thread.BeginCriticalRegion, Thread.EndCriticalRegion. Any code written in the finally block turns into such calls. For this reason, in the bowels of the framework code, you can find blocks with an empty try, but not an empty finally. Microsoft discourages this method so much that they didn't include it in .net core.
The Interrupt method works more predictably. It can interrupt the thread with an exception ThreadInterruptedException only when the thread is in a waiting state. It enters this state by hanging while waiting for WaitHandle, lock or after calling Thread.Sleep.
Both options described above are bad for their unpredictability. The way out is to use the structure CancellationToken and class CancellationTokenSource. The bottom line is this: an instance of the CancellationTokenSource class is created and only the one who owns it can stop the operation by calling the method Cancel. Only the CancellationToken is passed to the operation itself. CancellationToken owners cannot cancel the operation themselves, but can only check if the operation has been canceled. There is a boolean property for this IsCancellationRequested and method ThrowIfCancelRequested. The latter will throw an exception TaskCancelledException if the Cancel method was called on the CancellationTokenSource instance that paried the CancellationToken. And this is the method I recommend. This is better than the previous options by giving you full control over when an exception operation can be aborted.
The most brutal way to stop a thread is to call the Win32 API TerminateThread function. The behavior of the CLR after this function is called can be unpredictable. On MSDN, the following is written about this function: “TerminateThread is a dangerous function that should only be used in the most extreme cases. “
Converting Legacy API to Task Based Using the FromAsync Method
If you're lucky enough to be working on a project that was started after Tasks were introduced and no longer aroused the quiet horror of most developers, then you won't have to deal with a lot of old APIs, both third-party ones and those your team has tortured in the past. Luckily, the .NET Framework development team took care of us, although the goal may have been to take care of ourselves. Be that as it may, .NET has a number of tools for painlessly converting code written in the old asynchronous programming approaches to the new one. One of them is the FromAsync method of the TaskFactory. In the code example below, I'm wrapping the old asynchronous methods of the WebRequest class in a Task using this method.
object state = null;
WebRequest wr = WebRequest.CreateHttp("http://github.com");
await Task.Factory.FromAsync(
wr.BeginGetResponse,
we.EndGetResponse
);
This is just an example and you are unlikely to have to do this with built-in types, but any old project is simply teeming with BeginDoSomething methods returning IAsyncResult and EndDoSomething methods accepting it.
Converting Legacy API to Task Based Using the TaskCompletionSource Class
Another important tool to consider is the class TaskCompletionSource. In terms of functions, purpose and principle of operation, it can somewhat resemble the RegisterWaitForSingleObject method of the ThreadPool class, which I wrote about above. Using this class, you can easily and conveniently wrap old asynchronous APIs in Tasks.
You will say that I already talked about the FromAsync method of the TaskFactory class for this purpose. Here we have to recall the entire history of the development of asynchronous models in .net that Microsoft has been offering over the past 15 years: before the Task-Based Asynchronous Pattern (TAP), there was the Asynchronous Programming Pattern (APP), which was about methods BeginDoSomething returning IAsyncResult and methods EndDoSomething its host and for the legacy of these years, the FromAsync method is just great, but over time, it was replaced by the Event Based Asynchronous Pattern (EAP), which assumed that an event would be raised when the asynchronous operation completed.
TaskCompletionSource is just perfect for wrapping Task and legacy APIs built around the event model. The essence of its work is as follows: an object of this class has a public property of the Task type, the state of which can be controlled through the SetResult, SetException, etc. methods of the TaskCompletionSource class. In places where the await operator was applied to this Task, it will be executed or crashed with an exception, depending on the method applied to the TaskCompletionSource. If it's still not clear, then let's look at this code example, where some old EAP-era API is wrapped in a Task using TaskCompletionSource: when the event fires, the Task will be set to the Completed state, and the method that applied the await operator to this Task will resume its execution getting an object result.
public static Task<Result> DoAsync(this SomeApiInstance someApiObj) {
var completionSource = new TaskCompletionSource<Result>();
someApiObj.Done +=
result => completionSource.SetResult(result);
someApiObj.Do();
result completionSource.Task;
}
TaskCompletionSource Tips & Tricks
Wrapping old APIs isn't all you can do with TaskCompletionSource. Using this class opens up an interesting possibility of designing various APIs on Tasks that do not take up threads. And the flow, as we remember, is an expensive resource and their number is limited (mainly by the amount of RAM). This limitation is easy to achieve when developing, for example, a loaded web application with complex business logic. Let's consider the possibilities that I'm talking about on the implementation of such a trick as Long-Polling.
In short, the essence of the trick is this: you need to receive information from the API about some events occurring on its side, while the API for some reason cannot report the event, but can only return the state. An example of such is all APIs built on top of HTTP before the times of WebSocket or when it is impossible for some reason to use this technology. The client can ask the HTTP server. An HTTP server cannot initiate communication with a client on its own. A simple solution is to poll the server on a timer, but this creates additional load on the server and additional delay on average TimerInterval / 2. To get around this, a trick called Long Polling was invented, which involves delaying the response from the server until Timeout expires or an event will occur. If the event has occurred, then it is processed, if not, then the request is sent again.
while(!eventOccures && !timeoutExceeded) {
CheckTimout();
CheckEvent();
Thread.Sleep(1);
}
But such a solution will prove horrendous as soon as the number of clients waiting for an event grows, because Each such client occupies a whole thread while waiting for an event. Yes, and we get an additional delay of 1ms when the event is triggered, most often this is not significant, but why make the software worse than it can be? If we remove Thread.Sleep(1), then in vain we load one processor core to 100% idle, spinning in a useless cycle. Using TaskCompletionSource, you can easily rewrite this code and solve all the above problems:
class LongPollingApi {
private Dictionary<int, TaskCompletionSource<Msg>> tasks;
public async Task<Msg> AcceptMessageAsync(int userId, int duration) {
var cs = new TaskCompletionSource<Msg>();
tasks[userId] = cs;
await Task.WhenAny(Task.Delay(duration), cs.Task);
return cs.Task.IsCompleted ? cs.Task.Result : null;
}
public void SendMessage(int userId, Msg m) {
if (tasks.TryGetValue(userId, out var completionSource))
completionSource.SetResult(m);
}
}
This code is not production-ready, just demo. For use in real cases, you also need to, at a minimum, handle the situation when a message arrives at a time when no one expects it: in this case, the AsseptMessageAsync method must return an already completed Task. If this case is the most frequent, then you can think about using ValueTask.
When we receive a request for a message, we create and place a TaskCompletionSource in the dictionary, and then wait for what happens first: the specified time interval expires or a message is received.
ValueTask: why and how
The async/await statements, like the yield return statement, generate a state machine from the method, and this is the creation of a new object, which is almost always not important, but in rare cases can create a problem. This case can be a method called really often, we are talking about tens and hundreds of thousands of calls per second. If such a method is written in such a way that in most cases it returns the result bypassing all await methods, then .NET provides a tool to optimize this - the ValueTask structure. To make it clear, consider an example of its use: there is a cache that we go to very often. There are some values in it and then we simply return them, if not, then we go to some slow IO after them. I want to do the latter asynchronously, which means that the whole method turns out to be asynchronous. So the obvious way to write a method is as follows:
public async Task<string> GetById(int id) {
if (cache.TryGetValue(id, out string val))
return val;
return await RequestById(id);
}
Out of a desire to optimize a little, and a slight fear about what Roslyn will generate when compiling this code, this example can be rewritten as follows:
public Task<string> GetById(int id) {
if (cache.TryGetValue(id, out string val))
return Task.FromResult(val);
return RequestById(id);
}
Indeed, the optimal solution in this case would be to optimize the hot-path, namely, getting the value from the dictionary without any unnecessary allocations and load on the GC, while in those rare cases when we still need to go to IO for the data, everything will remain plus / minus the old way:
public ValueTask<string> GetById(int id) {
if (cache.TryGetValue(id, out string val))
return new ValueTask<string>(val);
return new ValueTask<string>(RequestById(id));
}
Let's take a closer look at this code fragment: if there is a value in the cache, we create a structure, otherwise the real task will be wrapped in a meaningful one. The calling code doesn't care what path this code was executed in: ValueTask, in terms of C# syntax, will behave the same as a regular Task in this case.
TaskSchedulers: managing strategies for launching Tasks
The next API that I would like to consider is the class task Scheduler and its derivatives. I already mentioned above that TPL has the ability to manage the strategies for distributing Tasks across threads. Such strategies are defined in the heirs of the TaskScheduler class. Almost any strategy you might need will be found in the library ParallelExtensionsExtras, developed by microsoft, but not part of .NET, but supplied as a Nuget package. Let's briefly look at some of them:
- CurrentThreadTaskScheduler - executes Tasks on the current thread
- LimitedConcurrencyLevelTaskScheduler - limits the number of Tasks executed simultaneously by the parameter N, which is accepted in the constructor
- OrderedTaskScheduler - defined as LimitedConcurrencyLevelTaskScheduler(1), so the tasks will be executed sequentially.
- WorkStealingTaskScheduler — implements task distribution approach. In fact, it is a separate ThreadPool. Solves the problem that in .NET ThreadPool is a static class, one for all applications, which means that its overload or misuse in one part of the program can lead to side effects in another. Moreover, it is extremely difficult to understand the cause of such defects. That. there may be a need to use separate WorkStealingTaskSchedulers in those parts of the program where the use of the ThreadPool can be aggressive and unpredictable.
- QueuedTaskScheduler - allows you to execute tasks according to the rules of the priority queue
- ThreadPerTaskScheduler - creates a separate thread for each Task that is executed on it. Can be useful for tasks that take an unpredictable amount of time to complete.
There is a good detailed about TaskSchedulers on the microsoft blog.
For convenient debugging of everything related to Tasks, Visual Studio has a Tasks window. In this window, you can see the current state of the task and jump to the currently executing line of code.
PLinq and the Parallel class
In addition to Tasks and everything said in .NET, there are two more interesting tools: PLinq(Linq2Parallel) and the Parallel class. The first promises to run all Linq operations in parallel on multiple threads. The number of threads can be configured with the WithDegreeOfParallelism extension method. Unfortunately, most often PLinq in its default mode will not have enough information about the internals of your data source to provide a significant speed gain, on the other hand, the cost of trying is very low: you just need to call the AsParallel method before the Linq method chain and perform performance tests. Moreover, it is possible to pass additional information about the nature of your data source to PLinq using the Partitions mechanism. You can read more и .
The static Parallel class provides methods for iterating through a Foreach collection in parallel, executing a For loop, and executing multiple delegates in an Invoke parallel. The execution of the current thread will be stopped before the completion of the calculations. The number of threads can be configured by passing ParallelOptions as the last argument. You can also specify TaskScheduler and CancellationToken using options.
Conclusions
When I started writing this article based on the materials of my report and the information that I collected during my work after it, I did not expect that it would turn out so much. Now, when the text editor in which I am typing this article reproachfully tells me that the 15th page has gone, I will sum up the intermediate results. Other tricks, APIs, visual tools, and pitfalls will be covered in the next article.
Conclusions:
- You need to know the tools for working with threads, asynchrony and parallelism in order to use the resources of modern PCs.
- There are many different tools in .NET for this purpose.
- Not all of them appeared at once, because you can often find legacy ones, however, there are ways to convert old APIs without much effort.
- Threading in .NET is represented by the Thread and ThreadPool classes.
- The Thread.Abort, Thread.Interrupt, TerminateThread Win32 API functions are dangerous and not recommended. Instead, it is better to use the CancellationTokens mechanism
- The flow is a valuable resource, their number is limited. Avoid situations where threads are busy waiting for events. To do this, it is convenient to use the TaskCompletionSource class.
- The most powerful and advanced .NET tools for dealing with concurrency and asynchrony are Tasks.
- c# async/await statements implement the concept of non-blocking wait
- You can control the distribution of Tasks among threads using TaskScheduler derivative classes
- The ValueTask structure can be useful in optimizing hot-paths and memory-traffic
- The Tasks and Threads windows of Visual Studio provide a lot of useful information for debugging multithreaded or asynchronous code.
- PLinq is a cool tool, but it may not have enough information about your data source, however this can be fixed using the partitioning mechanism
- To be continued ...
Source: habr.com