Once again about the asynchronous state machine and where exactly are the allocations

Once again about the asynchronous state machine and where exactly are the allocations

Despite the fact that about async/await many words have already been said and many reports have been recorded, but in my teaching and mentoring practice, I often encounter a misunderstanding of the device async/await even Middle+ level developers.

As you know, when compiling an asynchronous method, the compiler converts the code, breaking it into separate steps. Then, during execution, each step is interrupted by an asynchronous operation. When it ends, it is necessary to understand exactly where to return the control – to which step. Therefore, all steps are numbered and the compiler very strictly monitors where you can go from where to where. In computer science, such a solution is called state machine. Also, they call it Russian finite state machine. Hereafter, for brevity, I will use the abbreviation SM (state machine).

So, in this article, we will consider in detail the car wasgenerated by the C# compiler from an asynchronous method to understand how asynchrony works in C#.

“High Level” C#

First, let’s look at an example of simple code in regular (“high-level”) C #.

using System;
using System.Threading.Tasks;
using System.IO;

public class Program {
    private string _fileContent;
    
    public async Task Main() {
        await Task.Delay(100);
        
        int delay = int.Parse(Console.ReadLine());
        await Task.Delay(delay);
        
        _fileContent = await File.ReadAllTextAsync("file1");
        
        await Task.Delay(delay);
    }
}

The code first waits for 100 ms, then reads from the console how long it will wait, waits some more, reads data from the file and waits some more. You should not look for logic in the sequence of these calls, the main thing for us here is that these are simply understandable asynchronous calls.

“Low Level” C#

What follows is the code that the compiler generates from “high-level” (regular) C#.
I will say right away that the original code generated by the compiler looks as if the developers of the code generator did everything so that nothing was incomprehensible to a person. However, for those interested, the original code can be viewed at sharplab.io.

In general, I asked the neural network to do a little refactoring to make the code more readable, and accompanied significant and non-obvious places with detailed comments. Here’s what came out (class content Program):

// Машина состояний (SM)
private sealed class AsyncStateMachine : IAsyncStateMachine
{
    // Определение состояний для машины состояний
    public enum State
    {
        NotStarted,               // Машина состояний не запущена - начальное состояние
        WaitingAfterInitialDelay, // Ожидание после начальной задержки
        WaitingForFileRead,       // Ожидание чтения файла
        WaitingAfterFinalDelay,   // Ожидание после последней задержки
        Finished                  // Завершено
    }

    public State CurrentState; // Текущее состояние машины состояний

    public AsyncTaskMethodBuilder Builder; // Строитель задачи асинхронного метода

    public Program Instance; // Экземпляр программы (оригинального класса)

    private int DelayDuration; // Длительность задержки (переменная delay стала полем машины состояний)

    private string FileContentTemp; // Временное хранение содержимого файла

    private TaskAwaiter DelayAwaiter; // Ожидатель задержки

    private TaskAwaiter<string> ReadFileAwaiter; // Ожидатель чтения файла

    private void MoveNext()
    {
        try
        {
            switch (CurrentState)
            {
                case State.NotStarted:
                    // Запуск начальной задержки
                    DelayAwaiter = Task.Delay(100).GetAwaiter();
                    if (DelayAwaiter.IsCompleted)
                    {
                        /* 
                            В случае если таска сразу после запуска завершилась, произойдет переход к выполнению следующего этапа машины состояний (WaitingAfterInitialDelay)
                            Такое бывает, например, когда в методе с модификатором async нет асинхронных вызовов, либо если мы эвэйтим уже завершенную таску.
                        */
                        goto case State.WaitingAfterInitialDelay;
                    }
                    // Конкретно в этом кейсе, исполнение не зайдет в if, который выше, а выполнит две нижние строки
                    CurrentState = State.WaitingAfterInitialDelay;
                    Builder.AwaitUnsafeOnCompleted(ref DelayAwaiter, ref this);
                    /* 
                        AwaitUnsafeOnCompleted запланирует, что указанная машина состояний (ref this) будет продвинута вперед после завершения работы указанного awaiter'а (будет вызван метод MoveNext).
                        По смыслу это похоже на ContinueWith.
                        [ссылка на исходник под кодом] *
                    */
                    break;

                case State.WaitingAfterInitialDelay:
                    DelayAwaiter.GetResult();
                    /*
                        В случае если в асинхронном методе случился эксепшн, тогда он будет выброшен при вызове GetResult и мы сразу попадем в блок catch.
                    */

                    DelayDuration = int.Parse(Console.ReadLine());
                    DelayAwaiter = Task.Delay(DelayDuration).GetAwaiter();
                    if (DelayAwaiter.IsCompleted)
                    {
                        goto case State.WaitingForFileRead;
                    }
                    CurrentState = State.WaitingForFileRead;
                    Builder.AwaitUnsafeOnCompleted(ref DelayAwaiter, ref this);
                    break;

                case State.WaitingForFileRead:
                    /*
                        Важно, что если выполнение идет по реально асинхронному сценарию (т. е. мы попадаем сюда не из goto case), и используется контекст синхронизации по умолчанию, либо он не задан (что по умолчанию в запросах ASP.NET Core, например), то метод MoveNext() будет вызван из какого-то потока пула потоков. То есть, разные состояния SM могут быть запущены разными потоками.
                        Обычно, нам, программистам, эта особенность не мешает. Но есть редкие кейсы, где это может быть важно - как, например, кейс в одной из задачек на самопроверку ниже в статье.
                    */
                    DelayAwaiter.GetResult();
                    ReadFileAwaiter = File.ReadAllTextAsync("file1").GetAwaiter();
                    if (ReadFileAwaiter.IsCompleted)
                    {
                        goto case State.WaitingAfterFinalDelay;
                    }
                    CurrentState = State.WaitingAfterFinalDelay;
                    Builder.AwaitUnsafeOnCompleted(ref ReadFileAwaiter, ref this);
                    break;

                case State.WaitingAfterFinalDelay:
                    // Завершение чтения файла и установка результата
                    FileContentTemp = ReadFileAwaiter.GetResult();
                    Instance._fileContent = FileContentTemp;
                    FileContentTemp = null;
                    DelayAwaiter = Task.Delay(DelayDuration).GetAwaiter();
                    if (DelayAwaiter.IsCompleted)
                    {
                        CurrentState = State.Finished;
                        return;
                    }
                    CurrentState = State.Finished;
                    Builder.AwaitUnsafeOnCompleted(ref DelayAwaiter, ref this);
                    break;
            }
        }
        catch (Exception exception)
        {
            CurrentState = State.Finished;
            Builder.SetException(exception);
        }
    }
}

private string _fileContent; // Содержимое файла

[AsyncStateMachine(typeof(AsyncStateMachine))]
public Task Main()
{
    AsyncStateMachine stateMachine = new AsyncStateMachine();
    stateMachine.Builder = AsyncTaskMethodBuilder.Create();
    stateMachine.Instance = this;
    stateMachine.CurrentState = AsyncStateMachine.State.NotStarted;
    stateMachine.Builder.Start(ref stateMachine);
    /* 
        Первый вызов MoveNext происходит прямо в stateMachine.Builder.Start. Т. е. первое состояние нашей SM фактически выполняется синхронно (и далее до первого реального асинхронного вызова).
        Исходник **
    */
    return stateMachine.Builder.Task;
}

* Method source code AsyncTaskMethodBuilder.AwaitUnsafeOnCompleted available at the link
** Source code for this method AsyncStateMachine.Builder.Start available at the link.

From the code above, you can see that actually for every use of the keyword await the compiler generates an additional state machine (SM) state. In addition, it is important to note that the compiler will generate the state machine itself if the modifier is used in the method type definition async.

By the way, this code won’t run because it doesn’t have some helper methods that the compiler generates. But lets you understand how asynchrony works in C#.

Let’s deal with allocations

It is interesting that in Debug mode AsyncStateMachine for rakes is presented in the form of a class, and in Release – in the form of a structure (struct). But even though this is a structure, if the execution really goes according to the asynchronous scenario, under the hood in Runtime there will still be an allocation for AsyncStateMachine.

When execution follows an asynchronous script (call DelayAwaiter.IsCompleted returns false), the CLR’s state machine needs to be moved from the stack to the managed heap, for this it is packaged (boxed) in AsyncStateMachineBox by the runtime.
For Task this happens inside AsyncTaskMethodBuilder.GetStateMachineBox.
For ValueTask it happens inside the chain (AsyncValueTaskMethodBuilder.AwaitUnsafeOnCompleted -> AsyncTaskMethodBuilder.AwaitUnsafeOnCompleted -> AsyncTaskMethodBuilder.GetStateMachineBox).

Pooling

It is also interesting that the CLR provides the possibility of pooling* AsyncStateMachineBox to minimize allocations (StateMachineBox RentFromCache() method).

* Pooling in this context is a data reuse technique to save space in the program heap and resources GC. In the case of state machine pooling, the CLR will be able to reuse it for ValueTaskto save on heap allocations Although even Steven Taub doubts the real effectiveness of this approach.

I would like to note separately that the use ValueTask often does not cancel allocations in the case of an asynchronous scenario.

And the final remark – if you look closely, what happened to the variable delaythen we will see that she was captured and transferred to the state machine field (DelayDuration in cleaned code and <delay>5__2 code from the compiler). Accordingly, it is important to understand that if the execution follows an asynchronous scenario (which is quite often), the value type variables will be boxed together with the state machine and will be moved to the managed heap (heap) – therefore SpanYou are not allowed to be used in methods with a modifier async.

What else to read on the topic

For example, a whole guide by Steven Taub “How Async/Await Really Works in C#” (“How Async/Await Really Works in C#”). Translations are also available on habra.

Or the article Prefer ValueTask to Task, always; and don’t await twice, which describes not only features ValueTaskbut also how to implement even asynchronous logic via IValueTaskSource or ManualResetValueTaskSourceCoreMinimizing the number of memory allocations in the heap.

It is worth mentioning that the approach from this article is similar to what Sergey Teplyakov @SergeyT did in his article back in 2017. The difference is that in my example, the SM is preserved more neatly in its original form, immediately inside the SM the code is accompanied by comments, the issue with allocations is analyzed and various cases are given.

Exercises for self-testing

Exercises for self-examination under the spoiler
  1. What will the program output?

void Main()
{
    RunAsync(); //"fire-and-forget"
    Console.WriteLine("Main");
    Thread.Sleep(1500);
}

async Task RunAsync()
{
    Console.WriteLine("RunAsync 1");
    await Task.Delay(1000);
    Console.WriteLine("RunAsync 2");
}
Answer

T. to. the first state is executed exactly synchronously (it is executed by the calling thread), then immediately at the call RunAsync “RunAsync 1” will be output to the console, then after startup Task.Delay(1000) the calling thread will immediately resume execution and transition to execution Console.WriteLine("Main")after which it will go into the state WaitSleepJoin (Wait:ExecutionDelay) and sleeps, then after about a second another thread (thread pool worker) will write “RunAsync 2” to the console. As a result, we get the conclusion:

RunAsync 1
Main
RunAsync 2
  1. How many SM states will be generated for such code?

async Task Delay1()
{
    await Task.Delay(1);
}
Answer
  • the first state is the stage at which thrust is initiated Task.Delay(1)

  • the second state – continuation (continuation) with a call GetResultwhich will be executed after the completion of a previously started task.

  • Total: 2 states.

By the way, technically the state field there can take another value -2which is set after all operations are completed, but in fact it is equivalent to the initial state.

  1. And for this?

async Task MultiDelay()
{
    var task = Task.Delay(1);
    await task;
    await task;
    await task;
}
Answer
  • 1 to all eveyts on launch Task.Delay

  • 1 continiation with challenge GetResult after the first await’a

  • then 2 add. states that make actually synchronous consumption of already completed tasks (they are executed in a chain through goto)

  • Total: 4 states.

  1. And the final task. How many SM states will be generated for such code?

Task Delay1()
{
    return Task.Delay(1);
}
Answer

The state machine for this code will not be generated at all because the modifier is missing async in the method declaration Delay1.

  1. Bonus: A question sometimes asked in interviews about why it’s not allowed await within lock. To answer it, it is quite conceptual.simplified) reproduce the code to which the keyword is expanded lock:

object _syncObj = new();

async Task DelayLocked()
{
    Monitor.Enter(_syncObj);
    await Task.Delay(1);
    Monitor.Exit(_syncObj);
}
Answer

If you haven’t come across this task before and don’t know the answer to it, for better learning, I would recommend trying to solve it yourself – run the code from the above example, see what it outputs, then look at the generated state machine and try to understand what’s going on. And you can write about the results of your research in the comments to this article. Spoiler alert: the execution of this code depends on the synchronization context.

The material is current for version .NET 8.0.1. I will be happy to answer your questions in the comments.

In conclusion, I want to thank Marka Shevchenko @markshevchenko and Yevhen Peshkov @epeshk for the roar of this article. And also, in future versions. async/await can change a lot.

Related posts