One Integration Test

Here is a small accident that is easy to make. A developer tidies ReportsController.cs and, while moving lines around, deletes the one line above the class: [Authorize(Roles = "Manager")]. The code still compiles. Every test the course has written still passes. The report now opens for anyone with a token, and bob, who is staff, can read the stock values. Nobody notices until the owner does.

Why did no test notice? Because every test so far looks at one piece, standing alone. The controller tests call GetStock() as a plain method, as protect endpoints and roles said: a lock is an attribute that ASP.NET reads when a request arrives, and a method call is not a request. The service tests use a real database, but no web request. We have tested every room of the house, and nobody has ever walked in through the front door.

An integration test does that. It starts the whole API, sends it a real request (log in, then ask for the report with the token), and checks the answer. The locks, the token check, the JSON and the database all run together, as in real life.

Our goal: a test project that starts the real API in memory, logs in as bob and as alice, and checks the locks on the books and the report.

Here is the route:

1. Say what an integration test is and how it differs from the unit tests so far.
2. Add a test project, BookShop.Api.IntegrationTests, and the package that starts the API inside a test.
3. Give the tests their own database, so they never touch bookshop.db.
4. Write the first test (bob must not read the report), and run it.
5. Break the lock on purpose and watch this test fail while the unit tests stay green.
6. Add two more tests in the same shape.

Left out: testing with a database server in a container (named at the end), testing the web pages in a browser, and making the tests faster by sharing one API between them.

The solution map

This lesson adds a project to the solution. Here is the finished map, with the new project marked:

Where we are

This is the solution from the end of the trace id. One project is new:

BookShop/
  Directory.Packages.props                  <- changes (one package)
  src/
    BookShop.Api/                           (unchanged)
    BookShop.Data/                          (unchanged)
  tests/
    BookShop.Api.Tests/                     (unchanged)
    BookShop.Data.Tests/                    (unchanged)
    BookShop.Api.IntegrationTests/          <- NEW
      BookShop.Api.IntegrationTests.csproj  <- NEW
      MSTestSettings.cs                     <- NEW (made by the template)
      BookShopFactory.cs                    <- NEW
      LockTests.cs                          <- NEW

Run every command from the solution folder, BookShop/.

How it differs from a unit test

A unit test checks one class, one public method, with stand-ins for the rest (unit testing basics). It is small and fast, and when it fails it points at one place. An integration test checks several parts working together. The service tests of the books service were already a small step this way, since they used a real database, and the lesson called them integration tests. Today's test goes all the way: it uses the real API.

Neither kind replaces the other. Unit tests are cheap, so a project has many. Integration tests are costly, so a project has fewer, for the paths that matter most. A lock on the report is one of them.

The test project

Create it as in the books service, with a different name, and give it a reference to the API project:

dotnet new mstest -n BookShop.Api.IntegrationTests -o tests/BookShop.Api.IntegrationTests --no-restore
dotnet add tests/BookShop.Api.IntegrationTests reference src/BookShop.Api
dotnet sln add tests/BookShop.Api.IntegrationTests

Delete Test1.cs, and leave MSTestSettings.cs. Visual Studio: right-click the solution, Add, New Project, MSTest Test Project, name it BookShop.Api.IntegrationTests in the tests folder, add a project reference to BookShop.Api, and delete Test1.cs. As before, the template's project file does not fit the shared settings (NU1008), so open tests/BookShop.Api.IntegrationTests/BookShop.Api.IntegrationTests.csproj and make the whole file:

<Project Sdk="Microsoft.NET.Sdk">

  <ItemGroup>
    <PackageReference Include="MSTest" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="Microsoft.VisualStudio.TestTools.UnitTesting" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\..\src\BookShop.Api\BookShop.Api.csproj" />
  </ItemGroup>

</Project>

Then the package that starts an API inside a test. It is Microsoft's, and it is called Microsoft.AspNetCore.Mvc.Testing:

dotnet add tests/BookShop.Api.IntegrationTests package Microsoft.AspNetCore.Mvc.Testing

Visual Studio: right-click the project, Manage NuGet Packages, and install Microsoft.AspNetCore.Mvc.Testing. The command writes the version into Directory.Packages.props and the name into the project file. The two whole files are now (your version numbers may be newer):

<Project>
  <PropertyGroup>
    <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
  </PropertyGroup>
  <ItemGroup>
    <PackageVersion Include="Microsoft.AspNetCore.Authentication.JwtBearer" Version="10.0.14" />
    <PackageVersion Include="Microsoft.AspNetCore.Mvc.Testing" Version="10.0.14" />      <!-- new -->
    <PackageVersion Include="Microsoft.AspNetCore.OpenApi" Version="10.0.12" />
    <PackageVersion Include="Microsoft.EntityFrameworkCore.Design" Version="10.0.13">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageVersion>
    <PackageVersion Include="Microsoft.EntityFrameworkCore.Sqlite" Version="10.0.13" />
    <PackageVersion Include="MSTest" Version="4.0.1" />
    <PackageVersion Include="Scalar.AspNetCore" Version="2.18.2" />
  </ItemGroup>
</Project>

<Project Sdk="Microsoft.NET.Sdk">

  <ItemGroup>
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" />                    <!-- new -->
    <PackageReference Include="MSTest" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="Microsoft.VisualStudio.TestTools.UnitTesting" />
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\..\src\BookShop.Api\BookShop.Api.csproj" />
  </ItemGroup>

</Project>

(The <!-- new --> marks are only for this page.) What the package gives us is a class, WebApplicationFactory<Program>. Program is the class that Program.cs makes, the start-up code of the API. The factory runs that start-up code inside the test, in memory: no port is opened, and nothing is shown in a browser. It also gives the test an HttpClient that talks straight into that API. Older guides tell you to add public partial class Program { } to the end of Program.cs, so the test can see the class. With .NET 10 and this package we did not need it (we tried both), so the API project does not change at all.

Its own database

If the test started the API as it is, the API would open bookshop.db, the reader's real file. Tests that read and write there would change the real books, and would depend on what happens to be in them. A test needs its own database, empty at the start and gone at the end.

We do what the service tests of the books service did: SQLite in memory, a database that lives only in the program's memory. One rule of in-memory SQLite is new here. It disappears the moment the last connection to it closes, and EF Core normally opens a connection for each request and closes it again. So the test opens one connection itself, keeps it open for the test's whole life, and gives that same connection to the API.

The factory is a small class that starts the API and swaps the database. Create BookShopFactory.cs in tests/BookShop.Api.IntegrationTests/. The whole file is:

using BookShop.Data;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Data.Sqlite;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;

namespace BookShop.Api.IntegrationTests;

public class BookShopFactory : WebApplicationFactory<Program>
{
    private readonly SqliteConnection _connection = new("DataSource=:memory:");

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureServices(services =>
        {
            services.RemoveAll<DbContextOptions<BookShopDbContext>>();
            services.RemoveAll<IDbContextOptionsConfiguration<BookShopDbContext>>();

            _connection.Open();
            services.AddDbContext<BookShopDbContext>(options => options.UseSqlite(_connection));
        });
    }

    protected override void Dispose(bool disposing)
    {
        base.Dispose(disposing);
        _connection.Dispose();
    }
}

• : WebApplicationFactory<Program> is the class from the package. We make our own version of it to change one thing.
• ConfigureWebHost runs while the API is being put together, before it starts. ConfigureServices lets us change what dependency injection holds. This is why the API registered its database through AddDbContext and not by hand: a test can replace one registration and leave the rest.
• The two RemoveAll lines remove the setting that says "use the file bookshop.db". AddDbContext keeps that setting in two places, the options and the list of instructions for building them, so both go. Removing only the first would still pass today, because the next line registers SQLite again and its settings are applied last. It would break the day the API used another database.
• _connection.Open() and UseSqlite(_connection) register the same database again, this time on our open in-memory connection.
• Dispose closes the connection at the end, and the database disappears.

Everything else the API needs stays as Program.cs set it up: the login, the token check, the locks and the services. The factory starts the app in the Development environment, as dotnet run does, so it reads appsettings.Development.json and finds the signing key there.

The first test

The question: a staff member who has logged in must not be able to read the stock report. In words, a test of the front door: log in as bob, ask for the report with the token, expect 403.

Create LockTests.cs in the same folder. The whole file is:

using System.Net;
using System.Net.Http.Json;
using BookShop.Data;
using Microsoft.Extensions.DependencyInjection;

namespace BookShop.Api.IntegrationTests;

[TestClass]
public class LockTests
{
    private BookShopFactory _factory = null!;
    private HttpClient _client = null!;

    [TestInitialize]
    public void Setup()
    {
        _factory = new BookShopFactory();
        _client = _factory.CreateClient();

        using var scope = _factory.Services.CreateScope();
        var context = scope.ServiceProvider.GetRequiredService<BookShopDbContext>();
        context.Database.EnsureCreated();
    }

    [TestCleanup]
    public void Cleanup()
    {
        _client.Dispose();
        _factory.Dispose();
    }

    private async Task<string> LoginAsync(string userName, string password)
    {
        var response = await _client.PostAsJsonAsync("/auth/login", new { userName, password });
        var login = await response.Content.ReadFromJsonAsync<LoginResponse>();
        return login!.Token;
    }

    [TestMethod]
    public async Task GetStockReport_LoggedInAsStaff_Returns403()
    {
        var token = await LoginAsync("bob", "bob-pass-1");
        _client.DefaultRequestHeaders.Authorization = new("Bearer", token);

        var response = await _client.GetAsync("/reports/stock");

        Assert.AreEqual(HttpStatusCode.Forbidden, response.StatusCode);
    }
}

• Setup runs before each test, as in the other test projects. It makes a new factory (so a new API and a new, empty database), asks it for the HttpClient, and creates the tables with EnsureCreated(). That is the same call the service tests use. It builds the tables from the model, so it also builds the price check, and it does not run the migrations, so the five sample books are not there.
• Cleanup closes everything, and the database goes with it. Each test starts clean.
• LoginAsync is the first half of every test here: POST the name and password to /auth/login and take the token out of the answer. It uses LoginResponse, the same record the API returns. PostAsJsonAsync writes the JSON for us.
• The test follows Arrange, Act, Assert from unit testing basics. It logs in as bob, puts Authorization: Bearer <token> on the client, asks for the report, and expects 403 Forbidden. The test name says the same: what is asked, in what situation, what comes back.

Run it:

dotnet test tests/BookShop.Api.IntegrationTests

Passed!  - Failed:     0, Passed:     1, Skipped:     0, Total:     1, Duration: 1 s - BookShop.Api.IntegrationTests.dll (net10.0)

One request after another has gone through the whole API: the login endpoint, the password check, the token being made, the token being read and checked, the role being compared with the lock, and the 403. That is more of the app than all the tests before it, in about thirty lines.

Break the lock on purpose

A test that never fails proves nothing, so make it fail. In src/BookShop.Api/Controllers/ReportsController.cs, comment out the one line above the class, the same accident as in the opening:

// [Authorize(Roles = "Manager")]
[ApiController]
[Route("reports")]
public class ReportsController : ControllerBase

First the unit tests:

dotnet test tests/BookShop.Api.Tests

Passed!  - Failed:     0, Passed:    15, Skipped:     0, Total:    15, Duration: 1 s - BookShop.Api.Tests.dll (net10.0)

(15 is the count in the scratch solution we ran this in; yours is higher. None fail.) They do not see the change. Now the integration test:

dotnet test tests/BookShop.Api.IntegrationTests

  Failed GetStockReport_LoggedInAsStaff_Returns403 [2 s]
  Error Message:
   Assert.AreEqual failed. Expected:<Forbidden>. Actual:<OK>. 'expected' expression: 'HttpStatusCode.Forbidden', 'actual' expression: 'response.StatusCode'.
Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: 2 s - BookShop.Api.IntegrationTests.dll (net10.0)

(We cut the stack trace.) Expected Forbidden, actual OK: bob got the report. This is the accident from the opening, caught before the owner found it. Put the line back and run again to see it pass.

Two more tests, same shape

The lock on the books and the success path of the report take the same few lines. Open tests/BookShop.Api.IntegrationTests/LockTests.cs and add two tests at the end. The whole file is:

using System.Net;
using System.Net.Http.Json;
using BookShop.Data;
using Microsoft.Extensions.DependencyInjection;

namespace BookShop.Api.IntegrationTests;

[TestClass]
public class LockTests
{
    private BookShopFactory _factory = null!;
    private HttpClient _client = null!;

    [TestInitialize]
    public void Setup()
    {
        _factory = new BookShopFactory();
        _client = _factory.CreateClient();

        using var scope = _factory.Services.CreateScope();
        var context = scope.ServiceProvider.GetRequiredService<BookShopDbContext>();
        context.Database.EnsureCreated();
    }

    [TestCleanup]
    public void Cleanup()
    {
        _client.Dispose();
        _factory.Dispose();
    }

    private async Task<string> LoginAsync(string userName, string password)
    {
        var response = await _client.PostAsJsonAsync("/auth/login", new { userName, password });
        var login = await response.Content.ReadFromJsonAsync<LoginResponse>();
        return login!.Token;
    }

    [TestMethod]
    public async Task GetStockReport_LoggedInAsStaff_Returns403()
    {
        var token = await LoginAsync("bob", "bob-pass-1");
        _client.DefaultRequestHeaders.Authorization = new("Bearer", token);

        var response = await _client.GetAsync("/reports/stock");

        Assert.AreEqual(HttpStatusCode.Forbidden, response.StatusCode);
    }

    [TestMethod]                                                                            // new
    public async Task GetBooks_NoToken_Returns401()                                         // new
    {                                                                                       // new
        var response = await _client.GetAsync("/books");                                    // new
                                                                                            // new
        Assert.AreEqual(HttpStatusCode.Unauthorized, response.StatusCode);                  // new
    }                                                                                       // new

    [TestMethod]                                                                            // new
    public async Task GetStockReport_LoggedInAsManager_ReturnsTheReport()                   // new
    {                                                                                       // new
        using (var scope = _factory.Services.CreateScope())                                 // new
        {                                                                                   // new
            var context = scope.ServiceProvider.GetRequiredService<BookShopDbContext>();   // new
            context.Books.Add(new Book                                                      // new
            {                                                                               // new
                Title = "Clean Code", Author = "Robert C. Martin", Price = 34.50m,          // new
                Stock = 7, Category = "Programming"                                         // new
            });                                                                             // new
            context.SaveChanges();                                                          // new
        }                                                                                   // new
        var token = await LoginAsync("alice", "alice-pass-1");                              // new
        _client.DefaultRequestHeaders.Authorization = new("Bearer", token);                 // new
                                                                                            // new
        var response = await _client.GetAsync("/reports/stock");                            // new
                                                                                            // new
        Assert.AreEqual(HttpStatusCode.OK, response.StatusCode);                            // new
        var report = await response.Content.ReadFromJsonAsync<List<CategoryStock>>();      // new
        Assert.HasCount(1, report!);                                                        // new
        Assert.AreEqual("Programming", report[0].Category);                                 // new
        Assert.AreEqual(7, report[0].Copies);                                               // new
    }                                                                                       // new
}

• GetBooks_NoToken_Returns401 is the simplest: no login, no token, and the books must refuse with 401.
• GetStockReport_LoggedInAsManager_ReturnsTheReport is the happy path: the same lock lets alice in. It first puts one book into the test database through the API's own DbContext (the test can reach the services the API runs on, through _factory.Services), then reads the report and checks it. The report must hold exactly one line, for the one book we added. That also proves the test used its own empty database: the reader's bookshop.db has five books and would have given more.

dotnet test tests/BookShop.Api.IntegrationTests

Passed!  - Failed:     0, Passed:     3, Skipped:     0, Total:     3, Duration: 2 s - BookShop.Api.IntegrationTests.dll (net10.0)

Three tests, three seconds. Each test makes its own API and its own database, which is why they take a second or two together and not a few milliseconds. That is the cost of an integration test, and the reason to keep them for what matters. A test that shares one API between all tests is faster, and it is left out here to keep the tests independent of each other.

The 400 that a bad book gets, which create a book could only check by hand, is also a job for this kind of test: post a book with no title and expect 400. That is left as an exercise, in the same shape as the three above.

One step further

SQLite in memory is a stand-in. It is quick and needs nothing installed, but it is not the database a live server uses, and a query that works on SQLite can behave differently on another one. Teams that want the real database in their tests start it in a container for the test run, with a library that does that (Testcontainers is the usual name). It is the step up from this lesson, and is left out.

Where we ended

Compare with your solution. One project was added, one file changed:

BookShop/
  Directory.Packages.props                  plus Microsoft.AspNetCore.Mvc.Testing
  tests/
    BookShop.Api.IntegrationTests/
      BookShop.Api.IntegrationTests.csproj  references BookShop.Api, uses Mvc.Testing
      MSTestSettings.cs                     template settings, left as made
      BookShopFactory.cs                    starts the API in memory, swaps in a SQLite in-memory database
      LockTests.cs                          three tests through the front door

Recap

• A unit test checks one class with stand-ins. An integration test checks several parts together, here the whole API with a real request.
• Locks, the token check and the JSON are invisible to unit tests, because those tests never send a request. An integration test is how you check them.
• WebApplicationFactory<Program> starts the API inside the test, in memory, and gives an HttpClient that talks to it.
• The test gets its own database: in-memory SQLite on one open connection that the test keeps alive, swapped in with ConfigureServices.
• A test that cannot fail is worth nothing: break the code on purpose and watch it fail.
• Integration tests are slower, so a project has fewer of them than unit tests, for the paths that matter.

Questions to think about

Try to answer each one before you open the answer.

1. Why did the unit tests stay green when the [Authorize] line was deleted?

2. What does an integration test cover that a unit test does not?

3. Why does the test project give the API its own database?

4. Integration tests are slower than unit tests. Why write them anyway?

Next lesson: switching to a real database server, changing the database under the API and seeing what that costs.