Asynchronous Code: async and await

Open almost any real C# API code and you will see words like async, await and Task<Book> all over it. This lesson is here so that you can read that code without getting lost, and write it with a simple rule.

You do not need to understand how it works inside. We will say that again later, because it is the most important sentence in this lesson.

We tell the story in four steps: what is slow, why waiting hurts, how people first fixed it, and the modern fix.

What is slow?

Think about what a program actually does. There are two very different kinds of work:

• Thinking: adding numbers, comparing text, looping over a list, reading a variable. This uses the CPU and memory, and it is extremely fast.
• Waiting for something outside the program: reading or writing a file, calling another service over HTTP, asking a database a question over the network. These are called I/O, which stands for input / output.

I/O is slow because the program is not in charge. It must wait for a disk to spin up, or for a message to travel across a network and come back.

How slow? Here are rough numbers. A nanosecond is one billionth of a second, a microsecond is a thousand nanoseconds, and a millisecond is a thousand microseconds.

These are rough, order-of-magnitude numbers. The exact values depend on the machine. The lesson is in the shape: our own code is fast, and I/O is a thousand or a million times slower. The graph shows it.

The bars are drawn on a stretched scale: each step along the line is many times bigger than the last. If we drew it to true scale, the first bars would be invisible and the last would run off the page, which is the whole point.

So when a program feels slow, the cause is usually not the code we wrote. It is the program waiting.

The original problem: blocking

The simplest way to write code is to call something and wait for the answer:

decimal price = ReadPrice();   // reads the price from a file

While ReadPrice waits for the disk, our code stops on that line. The thread that runs our code is stuck there and does nothing else. This is called blocking.

A thread is a worker that runs your code, one line at a time. A blocked thread is one that is stuck on a line, waiting, and doing nothing else. The restaurant story below shows what that looks like.

Here is a small program with three slow steps. It reads a price from a file, asks an exchange-rate service for a rate, and saves a line to a log file. To keep the demo short, each step is pretend I/O that simply waits half a second (Thread.Sleep):

// each of these waits half a second, like real I/O would
var price = ReadPrice();       // a file
var rate = GetRate();          // another service over HTTP
SaveLine($"{price * rate}");   // a log file

Output, with the time since the start and the thread that ran the line:

    9 ms  thread 1  start
  513 ms  thread 1  price 12.5
 1014 ms  thread 1  rate 0.9
 1514 ms  thread 1  saved 11.25

One thread, three waits, one after another. For those 1.5 seconds, thread 1 did nothing useful. The CPU was idle. It only waited.

For a small program on your own laptop, that is fine. For a web server it is a disaster.

Why this hurts a web server

A web app runs on a limited number of threads. Each request is handled by one thread, from the start of the request to the end. And what does a typical simple web app spend its time doing? Waiting on the database.

A server has only a limited number of threads ready to serve requests. Each thread costs memory, so you cannot have endless amounts. A real server might have something like 32 threads (it depends on the machine). A request takes one thread, and the thread is busy until the request finishes.

To keep the picture small, imagine a server with only 4 threads. Six requests arrive. Each request asks the database a question and then blocks, waiting for the answer.

Look at what is happening. Four threads are parked, doing nothing but waiting for the database. Requests 5 and 6 cannot start, because there is no free thread, so they wait too. They wait for no good reason, because the work they need from the CPU is tiny. They are stuck behind threads that are only sitting there.

Users see slow pages, or timeouts. And the strange part: if you look at the server, the CPU is almost idle. Nothing looks busy, yet the app is choking.

The real numbers are bigger, but the problem is the same. With 32 blocked threads, request number 33 waits. (The server does add more threads when it sees this, but only about one or two a second, which is far too slow for a sudden crowd.)

The fix is not a faster CPU, and it is not "more threads". The fix is to stop wasting a thread while it waits. If threads never sit blocked, those same 32 threads can keep thousands of requests going at once, because each thread is only busy for the short moments when there is real code to run.

When a request reaches await, its thread does not stay with it. It jumps to the next request that is ready, so all the waiting happens somewhere else (in the database and the network), not on a thread. In the picture, each block is a turn for some request, and you can see how quickly a thread moves from one to the next. (The colours only make the blocks easier to tell apart.)

A story: the restaurant

Before the fix, a story. Picture a packed restaurant with only 4 waiters.

Waiters who wait

Each waiter takes a guest's order, walks to the kitchen, hands it over, and then stands at the kitchen window until the food is ready.

After four guests have ordered, all four waiters are at the kitchen window. They are not busy. They are only standing there. But the restaurant is full of guests, and guest 5, guest 6 and everyone after them are starving, because no waiter is free to even take their order. They wait for no good reason. The cause is that cooking is slow.

Waiters who keep moving

Now the better way. A waiter hands the order to the kitchen and goes straight back to the tables to take the next order. Then the next. When the kitchen has a dish ready, a waiter picks it up and delivers it.

1. A waiter takes an order from the guests.
2. The waiter hands it to the kitchen.
3. The waiter goes straight back to the tables to take more orders. They do not wait at the kitchen.
4. The kitchen finishes a dish.
5. A waiter picks it up and delivers it to the right guest.

The kitchen cooks many dishes at the same time, and waiters are fast, so four waiters can look after a whole busy restaurant. The kitchen is slow, but nobody stands around waiting for it.

Who is who

This is the whole lesson in one picture. The first restaurant is blocking. The second is what await gives you, and it is what the rest of this lesson is about.

An old fix: callbacks (history only)

You can skip this section. It is here for history, so you know why async/await exists. If it feels confusing, that is fine, because nobody writes new code this way any more. Jump to the next section.

The first idea was: do not stand and wait. Hand over the job together with a note that says "call me back when you are done". That note is a callback: a method you pass in, to be run later with the result.

ReadPriceThen(price =>
{
    GetRateThen(rate =>
    {
        SaveLineThen($"{price * rate}", () =>
        {
            Console.WriteLine("saved");
        });
    });
});

The thread is free while the file is being read, so no more waste. But look at the code. Each step is inside the previous step, so three steps already drift to the right. Real code has ten steps, loops and error handling, and it turns into a pyramid. It is very hard to write, very hard to read and understand, and hard to maintain, because changing one step means digging through the nesting. People called it callback hell.

So callbacks fixed the waste but broke the readability. We needed both.

The modern fix: async and await

C# (and later JavaScript, Python and others) came up with async and await. The same three steps look like this:

var price = await ReadPriceAsync();
var rate = await GetRateAsync();
await SaveLineAsync($"{price * rate}");

It reads top to bottom, like the blocking version. No nesting. But it does not waste the thread.

The key word is await. It means:

"This will take a while. Pause this method here and free the thread to do other work. When the answer is ready, continue from this line."

Back to the restaurant: the waiter hands the order to the kitchen and goes back to the tables. When the dish is ready, a waiter (not necessarily the same one) delivers it, carrying on from where that guest's order was left.

Be clear about what this does and does not do. await does not make one request faster. The three steps still happen one after another, so this request still takes about 1.5 seconds. The win is for the server: during each wait, the thread goes back to serve other users instead of standing at the kitchen window.

Here is the life of one thread in a server that uses await. It does not stay with any one request. It runs a request until that request reaches an await, leaves it there, and moves on to the next request that is ready.

With three users at once the difference grows. Each user's request needs a short bit of code, then a wait, then a short bit more. We have one thread. Watch what it does:

User A finishes at the same moment either way. But B and C finish far sooner, because the thread started their work while A was waiting, instead of leaving them in a queue. This is why one thread can serve many users.

You do not need the internals

This is the most important thing to know. How await works inside the compiler is interesting, but you do not need it. Almost nobody thinks about it when writing an API.

In the restaurant, a waiter does not need to know how the kitchen's order system works. The waiter only needs to know that slow orders get handed to the kitchen. It is the same here. You only need one rule:

The rule: whenever something is I/O (a file, another service, and above all a database call), call its async version and await it.

That is all you do. You are giving the compiler a hint: "this one may take a while, so feel free to leave and come back." The compiler and the .NET libraries do the hard work for you.

The libraries already provide the async versions:

Some things, such as HttpClient (for calling other services), only offer the async version: await http.GetStringAsync(url).

And your own plain code, such as adding numbers or looping over a list, stays normal. No async for CPU work.

Reading async code

Now the vocabulary, so the code you meet is no longer strange. Here is a method of our own that follows the rule. It uses the two slow steps from earlier, reading the price and getting the rate:

public async Task<decimal> GetPriceInEurosAsync()
{
    var price = await ReadPriceAsync();
    var rate = await GetRateAsync();
    return price * rate;
}

• Task<decimal> is the return type. A Task<decimal> means "a decimal that will be ready later". Think of a receipt you get from the counter while your order is being prepared. It is not the order itself, only the promise of it.
• await collects the order: it turns a Task<decimal> into the plain decimal inside it, once it is ready. So await ReadPriceAsync() gives a decimal. This is also the line where the waiter hands the order to the kitchen and moves on, and the line where the method carries on once the dish is ready.
• return price * rate; returns a plain decimal. You write the normal value, and the compiler wraps it in the Task<decimal> for you.
• async is a flag on the method that says "I use await inside". You cannot await in a method that is not marked async.
• The Async ending in ReadPriceAsync is a naming habit. It tells the reader "this returns a Task, so you must await it". Follow the habit.

If a method does something slow but returns nothing, its return type is plain Task (not Task<T>), like SaveLineAsync in the earlier example.

Async spreads upwards

If method A awaits method B, then A must be async and returns a Task. So whoever calls A should await it, and so on, all the way up.

In an API this chain ends at the endpoint: the endpoint is async too, and ASP.NET Core awaits it for you. You will see this when we build the endpoints. This is normal, so do not fight it.

Common mistakes

1. Forgetting await. Without await, you hold the receipt but not the order:

decimal price = ReadPriceAsync();

error CS0029: Cannot implicitly convert type 'System.Threading.Tasks.Task<decimal>' to 'decimal'

The compiler catches this one, and the message tells you the cause. A good rule of thumb: when you see Task<something> in an error, you probably forgot an await.

There is a quieter version. If a method returns plain Task and you call it without await, the compiler stays silent. The call still runs, but your code carries on without waiting for it to finish, and you never see its errors. Your IDE shows a warning for this, so do not ignore it.

2. Waiting with .Result or .Wait(). These force a blocking wait on a Task:

var price = ReadPriceAsync().Result;   // blocks the thread: don't

It works, but it blocks the thread again, so you lose the whole benefit. It is the waiter standing at the kitchen window again. In some kinds of application it can even freeze the program for good. Use await.

3. Wrapping a blocking call in Task.Run to make it "async". This hires a second waiter to stand at the kitchen window for the first one. A thread is still stuck, so nothing is fixed. The cure is to choose the real async version of the call (the table above), not to dress up the blocking one.

One more word: async void. You may see this in old code. It is meant for a few special cases only. Always write async Task instead.

Recap

• I/O (files, other services, the database) is slow compared with CPU and memory. A program spends most of its time waiting on it.
• Blocking means a thread waits and does nothing. A web server that does this can run out of threads while the CPU sits idle, so users see slow pages. In the restaurant, the waiters are stuck at the kitchen window and guests starve.
• Callbacks fixed the waste but made code hard to read. They are history.
• await frees the thread during the wait and continues afterwards, while the code still reads top to bottom. In the restaurant, the waiter hands over the order and goes back to the tables. It makes the server efficient, not one request faster.
• A Task<T> is a receipt for a T that will be ready later, and await turns it into the T.
• The rule: I/O means use the async version and await it. Your method becomes async and returns Task or Task<T>, and that spreads up to the endpoint.
• You do not need the internals. Avoid .Result, .Wait(), Task.Run as a fix and async void.

Questions to think about

Pick one answer for each. The answers are not given here on purpose, so check yourself against the lesson.

1. Which of these is I/O?

   • A. Adding up the prices in a list of 1,000 books
   • B. Sorting a list of titles in memory
   • C. Asking the database for a book
   • D. Looping over an array

2. A web server has its CPU at 2%, yet users complain that pages are very slow. Every thread is stuck at a database call. What is this called, and what is the cause?

   • A. A memory leak, because the CPU should be busy
   • B. Blocking, because threads wait on I/O and none are left for new requests
   • C. A slow network cable between the users and the server
   • D. A bug in the CPU

3. You change a method to use await on its database call. What happens for one request?

   • A. It becomes much faster
   • B. It takes about the same time, but the thread is free during the wait
   • C. It uses more CPU, so it is slower
   • D. It runs in parallel with itself

4. A method returns Task<Book>. What do you write to get the Book out of it?

   • A. .Wait()
   • B. Task.Run(...)
   • C. await
   • D. Nothing, it already is a Book

5. Your method contains await GetRateAsync(). What must be true of your method?

   • A. It must be marked async and return Task or Task<T>
   • B. It must be static
   • C. It must be named with the Async ending
   • D. It must return void

6. A teammate wants to "make a blocking file read async" by writing await Task.Run(() => File.ReadAllText(path)). What is the better fix?

   • A. Nothing, this is perfect
   • B. Use .Result instead
   • C. Use await File.ReadAllTextAsync(path)
   • D. Wrap it in a callback