Async/Await C#. Part 5. Enumeration function and loop through recursion, asynchronous call without Async/Await

Async/Await C#. Part 5. Enumeration function and loop through recursion, asynchronous call without Async/Await

In one of the previous articles devoted to the analysis of this (all the same) Post, I wrote that “I will not undertake to rewrite all this tricky logic related to the formation of a sequence of calls to a generated function in human language.” But analyzing in detail the 5th chapter of that Post, I somehow still managed to do it. I ask you to evaluate the results of this work.

Where I have to quote (let’s call it that) the content of the source text in the translation, it will be highlighted underlined italics.In the previous article, I tried to experiment with special formatting, but it is not always convenient for the reader, as far as I understand.

So, the author of the original Post claims that in order to get rid of callbacks during asynchronous calls, callbacks which he also calls “continuation” or according to our “continuation”, we will be helped by iterators, or rather functions which the compiler uses to implement interfaces IEnumerable<T> and/or an IEnumerator<T> the concept of which appeared as early as C# 2.0. For now, let’s just note that interfaces do not exist without objects that implement them, that is, such functions implicitly create an object of some type (class) generated by the compiler, which implements interfaces. IEnumerable<T> and/or an IEnumerator<T>, in this sense, such functions are object functions, since they are actually converted to objects by the compiler.

It is very useful to consider an example with the implementation of an iterator for calculating the elements of the Fibonacci sequence, given by the author of the Post:

public static IEnumerable<int> Fib()
{
    int prev = 0, next = 1;
    yield return prev;
    yield return next;

    while (true)
    {
        int sum = prev + next;
        yield return sum;
        prev = next;
        next = sum;
    }
}

And also the method of calling this function is as follows:

foreach (int i in Fib())
{
    if (i > 100) break;
    Console.Write($"{i} ");
}

Or this one:

using IEnumerator<int> e = Fib().GetEnumerator();
while (e.MoveNext())
{
    int i = e.Current;
    if (i > 100) break;
    Console.Write($"{i} ");
}

We can see that to fully use this function, to get a sequence of values, we need a loop, foreach or while. In fact, the second usage option is more explicit and demonstrative, there we see that the function is not used as an ordinary function, but as a more complex entity, which first returns an interface to an object of unknown type, which is stored in the variable “e”, and the most interesting thing is that then, we repeatedly call the same function through a method of this interface in a loop MoveNext().

To be honest, I was once amazed that someone had thought to implement such a compiler capability. If you think about how such a function is executed, you can notice that it decomposes into several functions that are executed one after the other, it is interesting to note that in the given example this sequence is infinite, but we can count how many functions we get from this original . The author of the post gave an example of a compiled function in the form of a state machine with a switch-case operator, but I want to show an alternative way of implementing such a function, which clearly repeats parts of the original (compiled) function and therefore seems to me to be clearer, and therefore more useful for understanding what happens with the following function when compiling:

class FibFuncDataStorage
    {
        int prev;
        int next;
        int sum;
        public delegate int NextFunc();
        public NextFunc nextFunc;
        public FibFuncDataStorage()
        {
            nextFunc = func1;
        }
      private bool MoveNext()
      {//is it typo it is private? 
          nextFunc();
          return true;
      }
        int func1()
        {
            nextFunc = func2;
            prev = 0; next = 1;
            return prev;
        }
        int func2()
        {
            nextFunc = func3;
            return prev;
        }
        int func3()
        {
            nextFunc = func4;
            return prev;
        }
        int func4()
        {
            nextFunc = func5;
            sum = prev + next;
            return sum;
        }
        int func5()
        {
            //nextFunc = func5;
            prev = next;
            next = sum;
            sum = prev + next;
            return sum;
        }
    }

As you can see in the state, the machine can be replaced by a simple delegate, where the original function is decomposed into many functions that represent individual pieces of that original function. Such a class can also be formalized-inherited from the right interfaces so that it also works well in foreach constructs or through the IEnumerator iterator. When you complete this design (which is a very useful learning task in itself) you will be able to compare the complexity or clarity-readability of this option with the option offered to us by the real C# compiler shown by the author of the Post, look in the Post for the code that begins with the line:

public static IEnumerable<int> Fib() => new <Fib>d__0(-2);
...

As I understand it, each of the implementations has its own strengths and weaknesses, that is, I would not take it upon myself to judge which one is better overall, they are simply different implementations of the same functionality that are better or worse for specific application conditions.

Now that we have thoroughly (I hope) dealt with функциями-объектами для перечисления we can also try to dig deep into the examples that the author of the Post gave us for implementing asynchronous calls. I’ll list them here in one piece of code because they can’t be considered in isolation from each other:

    static internal class AyncAwait
    {
static Task IterateAsync(IEnumerable<Task> tasks)
{
    var tcs = new TaskCompletionSource();

    IEnumerator<Task> e = tasks.GetEnumerator();

    void Process()
    {
        try
        {
            if (e.MoveNext())
            {
                e.Current.ContinueWith(t => Process());
                return;
            }
        }
        catch (Exception e)
        {
            tcs.SetException(e);
            return;
        }
        tcs.SetResult();
    };
    Process();

    return tcs.Task;
}
static Task CopyStreamToStreamAsync(Stream source, Stream destination)
{
    return IterateAsync(Impl(source, destination));

    static IEnumerable<Task> Impl(Stream source, Stream destination)
    {
        var buffer = new byte[0x1000];
        while (true)
        {
            Task<int> read = source.ReadAsync(buffer, 0, buffer.Length);	
             Console.WriteLine($"read {read.Result}");
            yield return read;
            int numRead = read.Result;
            if (numRead <= 0)
            {
                break;
            }

            Task write = destination.WriteAsync(buffer, 0, numRead);
             Console.WriteLine($"write {write.AsyncState}");
            yield return write;
            write.Wait();
        }
    }
}
  }

As you can see, I even had to push these methods into a special class AyncAwaitso that they can be compiled. You can also pay attention that I had to add several lines of leagues, the result of which we will see below.

It seems to me that it is especially important to pay attention to what the author of the original post missed. If you remember what our previous analysis of the compilation of functions-objects for enumeration is based on, you will understand that we are missing the code that calls our function-example of an asynchronous operation: CopyStreamToStreamAsync(). We should write such code, for example, in this form:

static void Main(string[] args)
{
    string StartDirectory = @"c:\tmp";
    string filename = Directory.EnumerateFiles(StartDirectory).First();
    Stream source = File.Open(filename, FileMode.Open);
    Stream destination = File.Open(@"c:\tmp\test.bin", FileMode.OpenOrCreate);

    Task tsk = AyncAwait.CopyStreamToStreamAsync(source, destination);

    Console.WriteLine($"{tsk} was started");
    Console.WriteLine("was it asynch?"); 
…
}

And we can get the following console output:

read 4096
write
System.Threading.Tasks.Task was started
was it asynch?
read 4096
write

which shows us that the function was indeed executed asynchronously, as we can see from the logs added to the original example

read 4096
write

that the called operation continues to run after returning from the function called to start that operation!

Full asynchronous call without async/await modifiers

Let’s figure out how we got a full-fledged asynchronous call WITHOUT async/await modifiers. It all starts with two challenges:

IterateAsync(Impl(source, destination));

Here it is necessary to remember that the function Impl() is an enumeration object function, each time it is called it returns this enumeration object. This means that we have created an enumeration object and passed it to the function IterateAsync()nothing more!

Next, we analyze what happens in the function IterateAsync(). And there we have 4 (four) lines of code, if you do not count the definition of the function Process(). We have created:

1. created an object TaskCompletionSource which will represent our asynchronous operation CopyStreamToStreamAsync() in the form of a task,

2. received an iterator to our enumeration object;

3. called the function Process();

4. returned a task representing the created asynchronous operation.

The point is that the function Process() recursive, recursion in the case is a way of organizing the loop, which, in turn, is necessary to pass from the enumeration from the function-object to the enumeration Impl(). Closing the function Process() on a variable with an iterator to our enumeration object, I think there is no need to explain. And we remember that MoveNext() this is also a call to the enumeration function Impl() in this case, only to certain parts of this function, each time to the next part.

The idea is that when we called an operation from main, we started a loop in which the successive tasks of that operation call each other in sequence with a function. Process()which organizes the recursion loop. This loop then exists – is executed in parallel with the execution of the next code after the call from the Main function, in our case. Tasks are executed asynchronously, so the entire operation (calling the CopyStreamToStreamAsync function) occurs asynchronously. Recursion does not result in a stack dump because each time a new recursive function call is made Process() is performed in the context of another task, that is, a function Process() does not perform itself directly, it gives itself another task to be called upon completion of this task. By the way, here it is appropriate to mention the concept of continuation again! This feature Process() is just such a continuation in this case, in fact, it is a function that allows us to build our tasks into a sequence and execute them in a loop, according to the order from that sequence.

There is one more nuance, which I also did not immediately understand, did not immediately pay attention to. We have a sequence of tasks that our enumeration function generates for us, these are the tasks that this function returns to us with yield returnBy the way, it is useful to notice that in our example they are generally of different types: Task-simple and Task. But whoever called our operation should have a task object that represents this one big operation, consisting of this sequence of tasks of different types, and such an object is created at the very beginning:

var tcs = new TaskCompletionSource();

is a task-getting wrapper through which the calling code can monitor whether an asynchronous operation that the calling code ran sometime before has completed, if of course it needs to monitor it. Note that this wrapper object is used to create and return a task representing the current complex-composite asynchronous task operation immediately when this task operation is started:

return tcs.Task;

And only at the very end of this complex operation, it will be given the result obtained by the results of the chain of function calls Process():

            tcs.SetException(e);
///////// или ////////////////////
        tcs.SetResult();

Conclusion

In conclusion, I will repeat that the author of the post reminds us that this solution was implemented and used even before async/await burst onto the scene (according to the author of the post):

In fact, some enterprising developers used iterators in this fashion for asynchronous programming before async/await hit the scene.

As far as I understand the main idea of ​​this solution and this outlined implementation is the basis of the c async/await constructs compilation method in C#. And then the original Post immerses us in the details of the implementation of methods of compiling structures using async/await in C#.

I hope my interpretation of the code examples from the original post will help someone better understand the way to compile c async/await constructs in C#, well, it will just be interesting as another description of a rather complex technique (maybe even a design pattern for which I don’t know the name) that can be applied somewhere on your own.

PS: you still have to force yourself to read the end of the original Post, you might find something else interesting.

Related posts