Foreward - Execution order

For test containers working in tandem with xUnit, execution order/print display of system message could be confusing so to avoid possible confusion here is how Testcontainer working with xUnit showcases messages in order

1. Testcontainer starts building its containers
2. xUnit starts running its tests in the containers
3. xUnit finishes its test and calls Testcontainer to delete all of its containers
4. xUnit prints and shows it's test results

During development there was some confusion over this message order, but this happens due to a failsafe for xUnit to change its test completion messages to failure incase any containers failed to get deleted due to bugs or no deletion for the containers created in the dispose async

public async Task DisposeAsync()
    {
        if (_postgres != null) await _postgres.DisposeAsync().AsTask();
        if (_serviceContainer != null) await _serviceContainer.DisposeAsync().AsTask();
        if (_generatorContainer != null) await _generatorContainer.DisposeAsync().AsTask();
        if (_readerContainer != null) await _readerContainer.DisposeAsync().AsTask();
        if (_middlewareContainer != null) await _middlewareContainer.DisposeAsync().AsTask();
    }

Also ensure that docker desktop is turned on when you want to run tests otherwise test container won't even be able to do step 1 of building its containers

Testcontainers

Testcontainer is the industry standard for making integration tests with real databases using docker. To start ensure you have all necessary packages installed inside your test folder.

dotnet add package Testcontainers
dotnet add package Testcontainers.Xunit
dotnet add package Testcontainers.PostgreSql
dotnet add package Microsoft.AspNetCore.Mvc.Testing
dotnet add package xunit
dotnet add package xunit.runner.visualstudio

Some of these might already be installed in the test folder you are working in so make sure to check the csproj file if it exists. If you are unsure where to install packages (the test folder) take a look inside the UnixInstallation script for energyService.Tests there you will see the filepath we cd into to install for energyService tests.

Make sure you add these add package commands to the UNIX and windows installation scripts after you are done so your teammates and future students can easily install necessary testing packages

These commands will install all necessary packages to run micro service and API tests by utilizing testing containers, xunit tests and ASP.NET core for creating a test web host server.

Using testcontainers

You will need at least two different classes in your test file.cs (currently, possible changes in the future) one for creating the containers and configuring the web host (Class A) and the other for running tests (Class B)

To build a test container its the same as docker, you need an image "dockerfile" and then you need something to take that image and build a container, then you need to start that container and have some way for it to be disposed of by xUnit/Testcontainer itself later on. With test container we can just utilize the existing dockerfiles we have, no new dockerfiles need to be created for testing (expected that, if working on a new service you created a dockerfile for it)

Testcontainers come with a lot of possible parameters beyond just naming and building so make sure to take a look at the Testcontainer .NET documentation on their website to see what you can do, they have many code examples as well on a github page: https://github.com/testcontainers/testcontainers-dotnet

The most important parameters to ensure you have is WithNetwork. because that will allow for the containers using the same network to communicate with each other. Testcontainer uses random ports so you don't need to specify any ports you are using in specific.

You need to make sure you create a network so containers can communicate with each other AND a postgres based on our own PostgreSQL container so that we actually have a real database to use

Class A

public sealed class EnergyServiceContainers<TProgram> : WebApplicationFactory<TProgram>, IAsyncLifetime where TProgram : class
   {

       public IFutureDockerImage _testImage;

       public DotNet.Testcontainers.Containers.IContainer _testContainer;

       public PostgreSqlContainer _postgres;

        public INetwork _network;
       
       public async Task InitializeAsync()
          {
              //we create a network test containers can use to communicate with each other
             _network = new NetworkBuilder()
            .WithName($"test-network{Guid.NewGuid()}")
            .Build();

             await _network.CreateAsync().ConfigureAwait(false);
             
             _postgres = new PostgreSqlBuilder("postgres:18.3-alpine3.23") //found in postgres dockerfile, builds a postgres container with a specifik version, using the one from our real postgres                                                                                                     
            .WithEnvironment("POSTGRES_USER", dbUser)                                //dockerfile makes it match 1 to 1
            .WithEnvironment("POSTGRES_PASSWORD", dbPassword)
            .WithEnvironment("POSTGRES_DB", "sensors")
            .WithEnvironment("POSTGRES_HOST_AUTH_METHOD", "trust")
            .WithNetwork(_network)
            .WithNetworkAliases("db-test-server")
            .Build();

             _testImage = new ImageFromDockerfileBuilder()
            .WithDockerfileDirectory(CommonDirectoryPath.GetSolutionDirectory(), @"{Path to the folder where the dockerfile you want is}") //example @"service/energyService/energyService
            .WithDockerfile("Dockerfile") //name of the dockerfile
            .Build();

            //you have setup for the creation of the image, before you build it you have to create that image so that it can be used
            await _testImage.CreateAsync().ConfigureAwait(false);

           _testContainer = new ContainerBuilder(_testImage)
            .WithNetwork(_network)
            .WithNetworkAliases("testservice")
            .Build();
            
            //once you have built your containers now all you need to do is start them at the end of the initialization
            await Task.WhenAll(_testContainer.StartAsync(), {any other container you need or have}).ConfigureAwait(false);
          }
   }

What containers you need to build is based on what containers your service/api uses during build time for example energyService needs to build containers for its generator, reader, service, the middleware and the postgres. Roombooking for example only needs to build containers for its Service and the postgres.

You also need a way to dispose/delete the containers you are creating during tests so that they don't stay around.

inside Class A

public async Task DisposeAsync()
    {
        if (_testContainer != null) await _testContainer.DisposeAsync().AsTask();
    }

Finally inside of Class A, between the initialize and dispose fucntion, you need to configure the web host you create from the WebApplicationFactory. The reason we need a web host test server is to be able to tests http calls correctly and have it match with the real database. To do this we need to use the webhost configuration to change and configure or ngsql data connection and data connection string to make our test containers utilize the newly created postgreses test container connection string instead. We need to do this because or connection strings are hard coded but cant be used by the test containers as they use a different network environment to communicate(The one we created earlier).

Another issue is that while we have created a test container based on our postgres, making a real database ,all of its tables are empty. While we usually allow generators or have other ways of populating these tables overtime, Testcontainer containers are meant to be fast and run quick and wont have that same wait time. So after the new connection string has been applied to the webhosts network we can manually create the table we want. Make sure it matches 1 to 1 with how the table would have been generated in the actual build to ensure accuracy when testing on the real database in the test containers

inside Class A

protected override void ConfigureWebHost(IWebHostBuilder builder)
{
        builder.UseEnvironment("Testing");'

        builder.ConfigureServices(services =>
        {
            var descriptor = services.SingleOrDefault(
                d => d.ServiceType == typeof(NpgsqlDataSource));

            if (descriptor != null)
            {
                services.Remove(descriptor);
            }

            string testcontainerConnectionString = _postgres.GetConnectionString();
            var testDataSource = NpgsqlDataSource.Create(testcontainerConnectionString);
            services.AddSingleton(testDataSource);
       
            //example taken from the energyService Configuration
            using var connection = testDataSource.OpenConnection();
            connection.Execute(@"
                CREATE TABLE IF NOT EXISTS t_sensors_energy (
                    id SERIAL PRIMARY KEY,
                    esensorid VARCHAR(15),
                    timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
                    kilowatt REAL
                );
            ");
        });
}

With this Class A is finished make sure you remember the standard we have for function orders inside Class A

public async Task InitializeAsync(){}
protected override void ConfigureWebHost(IWebHostBuilder builder){}
public async Task DisposeAsync(){}

For the webhost to be able to recognize your containers/ be able to create a web server for them you need to make sure your micro service/api startpoint includes this line in its Program.cs file

public partial class Program { }

You can look at the energyService Program.cs file for an example and detailed explanation. this one line will allow the WebApplicationFactory to recognize your service as a valid startpoint to create the test server and connect the containers to it.

Class B

Class B is simpler but is also important because it will describe to the web application factory what program they are making tests for and connects them to the correct web test server.

We run tests in a separate class (currently in the same file but this should be changed in the future) because that allows the test containers to only need to be created once for x amounts of tests that need to be run on them. Make sure to include all the necasseray service includes you need for communications between endpoints, databases, models etc... to function in the actual build (again see energyService tests or roomBooking tests for examples)

//example based on energy service
public sealed class EnergyServiceTesting : IClassFixture<EnergyServiceContainers<Program>>
{
      private readonly EnergyServiceContainers<Program> _containers;
      private readonly HttpClient _httpClient;

      public EnergyServiceTesting(EnergyServiceContainers<Program> containers)
      {
        _containers = containers;
        _httpClient = _containers.CreateClient();
      }
}

_containers allows us to connect to our containers and any of the functions found inside and httpClient allows us to make tests using links and http calls

Test Examples

Foreward - As with needing to create the database table in the postgres container when configuring the webhost, some tests will require you to populate the table created with entries to some degree. All the entries created don't get added to the real time build PostgreSQL database as we use the testcontainers that create its own table and instantly deletes itself so you don't need to worry about overpopulating the tables, this also allows us to safely test wrongful table entries and injections aswell.

The testing class utilizes xUnit so each test function is first started with a [Fact] above the function (see the xUnit wiki for more info)

Although the reasoning for using testcontainer with xunit and ASP.NET core is to do integration tests on real databases and not in memory databases, we can still do unit tests if we want thanks to xUnit. But if unit tests are the only tests planned for a service/api/anything that will need testing then utilizing testcontainer is overkill and mostly a time loss, still possible but wholly unnecessary, look instead to do moq tests with xUnit using in memory databases instead in those situations.

    [Fact]
    public async Task Verify_Postgres_Container_Is_Connected_And_Responding()
    {
        //Test if connections work between containers to the postgresql container/ database
        // Get the active connection string from the running Testcontainer
        string connectionString = _containers._postgres.GetConnectionString();

        // Connect from your host machine executing the tests into the container
        using var connection = new NpgsqlConnection(connectionString);
        await connection.OpenAsync();

        // Execute a raw query to check database responsiveness
        var result = await connection.QueryFirstOrDefaultAsync<int>("SELECT 1;");

        // Assert that the container processed the query successfully
        Assert.Equal(1, result);
    }


    [Fact]
    public async Task HealthCheck_Returns_OK()
    {

        // check your endpoints and most likely add what you use after /api/ in the full web version and you should be fine with that as the link
        var response = await _httpClient.GetAsync("/energy/health");

        // Assert
        response.EnsureSuccessStatusCode();
        var content = await response.Content.ReadAsStringAsync();
        Assert.Equal("\"OK\"", content);
    }

The test on top checks that the postgres is up and connected and the test on the bottom checks that the web host is correctly configured by being able to reach the postgres database through an endpoint

integration test example

when making integration tests there might be situations where you want access to some functions found in your services or api's we can gain access to these by using our _containers services to create a scope that can then be used to get any required services or functions from a namespace and assign them to a variable (again make sure you include all necessary pieces for your services communication in the test containers and on top of the file with "using {necassary dependancies};")

Pay close attention to the "var energryService" as thats where you define what service or class you want to get functions from. In this case IEnergyService is the energy service interface and thus gives the energyService variable access to the functions defined in the interface

Taken from the energyService

    [Fact]
    public async Task Full_Integration_Test_Example()
    {
        //Test if energyservice reaches databasequeries
        using var scope = _containers.Services.CreateScope();
        var energyService = scope.ServiceProvider.GetRequiredService<IEnergyService>();

        string connectionString = _containers._postgres.GetConnectionString();

        using (var connection = new NpgsqlConnection(connectionString))
        {
            await connection.OpenAsync();
            await connection.ExecuteAsync(@"
            INSERT INTO t_sensors_energy (esensorid, kilowatt) 
            VALUES ('sensor-abc', 120.5);
        ");
        }
        ;

        var result = await energyService.GetLatestAsync("sensor-abc");
        //energy response contains Data but no methods or ways to pick specifc values from it so convert it to EnergyData
        //or whatever type you have to compare the values you want or need to test
        var _energyData = result.Data as EnergyData;

        Assert.NotNull(_energyData);
        Assert.Equal(120.5, _energyData.kilowatt);
      }