Implementing new Services (and Mockups)

• Introduction
• Script usage
  • Create Service
  • Create Mockup
• Examples
• Common errors

Introduction

As visualized in the mockup models, the services and mockups in the project should follow a similar structure to help with debugging and ease-of-use. To achieve this two scripts per system has been created. A create service script and a create mockup script (explained more in detail below).

Script usage

The scripts can be found under /scripts.

Create Service

To create a new service execute either WindowsCreateService.bat or UnixCreateService.sh (system dependable). You will be prompted to enter a name for the Service. Please try to name the Service without spaces or special characters, as it will not conform with the C# code conventions, as the script uses the name for, well, naming.

Furthermore, the Windows script require you to first enter the Service name once in lower letters only (example: studying) and once capitalized (example: Studying). This is because Windows batch doesn't have a good way of doing this on its own.

The CreateService scripts add the following:

• A project under /services, with a Tests directory linked to that project.
• A docker compose file under /DockerCompose/stacks where you need to add the necessary components to launch the service (examples below).
• A init_sql directory for initializing things to the database for the Service.
• A echoed message in the terminal that you need to paste into /entryPoint/nginx.conf.

After running the script you need to write the compose.yaml file. If you are familiar with docker you can do this on your own, but try to follow how other compose files are made/structured. You can either look in existing files or look at the examples below.

The newly created service also needs to be added to the Rebuild scripts, see examples below.

Create Mockup

To create new mockups, you execute either WindowsCreateMockup.bat or UnixCreateMockup.sh (system dependable), but for the mockup to be linked and created, a service with the same name needs to exist before running the scripts. The name entered in the script should be the same entered into the Service, so to use the same example as made above, the entered prompts should be studying and Studying (or just studying if on an Unix system).

The CreateMockup scripts add the following:

• A project under /generateMockup, with a Tests directory linked to that project.
• A project under /readMockup, with a Tests directory linked to that project.

The generateMockup part is the simplest of the two, in it there exist a part that says "Generate data here" where the generation of the mockup data should be made. There exist no standard for this since data might differ in many ways. Our recommendation is to look at out mockups that might be similar and see how they work.

readMockup might be more complex, depending on the data retrieved from generateMockup. The goal of the reader is to take the generated data and send it via MQTT so that the middleware can add it to the database. Everything in the read is basically setup as it should, but the data need to be sent as one object. This means that MQTT cannot send arrays or Lists of data, so the use of for each statements are best used here. As mentioned above with the generated data, our recommendation is to look at other readers and see how the work. Some examples exist below.

After scripts

After running the script and adding everything needed, the service should at least boot and logs should be able to be seen in Docker. Below is a checklist of what needs to be done, for each Create script:

• Run the CreateService script and:
  • All components added to the Docker compose.
  • Added at least one init_sql file.
  • Added the echoed message to nginx.conf.
  • Added all necessary parts to the Rebuild script (again, see examples).
• Run the CreateMockup script and:
  • Generate the data in /generateMockup
  • Setup the MQTT sender in /readMockup
  • Added all necessary parts to the Rebuild script (again, see examples).

When all containers boot and no error messages can be seen, you can safely expand on the Mockup and Service. If you add something new and suddenly error messages appears, see common errors for what they could mean.

Examples

• compose.yaml
• Rebuild script
• readMockup

compose.yaml

Example for a studying service using a database. Notice that the database is only connected to the internal network, and can thus only be accessed by the studying service.

services:
  studying-db:
    image: postgres:18.3-alpine3.23
    container_name: studying-db
    environment:
      POSTGRES_USER: ${DATABASE_USER}
      POSTGRES_PASSWORD: ${DATABASE_PASSWORD}
    networks:
      - internal
    restart: unless-stopped
    volumes:
      - pgdata:/var/lib/postgresql
      - ./init_sql:/docker-entrypoint-initdb.d:z
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${DATABASE_USER}"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s

  studyingservice:
    image: studyingservice
    container_name: studying-service
    environment:
      DATABASE_USER: ${DATABASE_USER}
      DATABASE_PASSWORD: ${DATABASE_PASSWORD}
    depends_on:
      studying-db:
        condition: service_healthy
    networks:
      - internal
      - frontend
    restart: unless-stopped

networks:
  internal:
    internal: true
  frontend:
    external: true
    name: shared-frontend

volumes:
  pgdata:

Rebuild script

Example of what would be needed to be added to the rebuild scripts if a studying service were to be introduced. The docker build command needs to be pointed to the directory where the Dockerfile for the service is located (if using the scripts to create a Service the naming below is correct for the studying example).

# studying
docker build -t studyingservice ../service/studyingService/studyingService
docker compose -f stacks/studying/compose.yaml --env-file ./.env up -d

readMockup

In this example the studying mockup returns a list of books that is needed for ongoing courses. To be able to send this over the MQTT broker, each book have to be JSON serialized before sending it. The best approach is to run a for-each statement over the list and serialize every book by itself and send it as a MQTT message. It could look something like this:

    private static async Task RunParkingLoop(IMqttClient mqttClient)
    {
      var httpClient = new HttpClient();
      while (true)
      {
        try
        {
          // Fetch JSON from studyingmockup webservice
          var response = await httpClient.GetFromJsonAsync<List<Books>>(
              // Connect to mockup docker container endpoint
              "http://studyingMockup:8080/studying"
          );

          if (response != null)
          {
            foreach (var book in response)
            {

              Console.WriteLine($"SQL Datatypes: {book.sql_datatypes}");
              Console.WriteLine($"Book ID: {book.id}");
              Console.WriteLine($"Book Name: {book.name}");

              // Convert to JSON for MQTT
              var json = JsonSerializer.Serialize(book);

              var message = new MqttApplicationMessageBuilder()
                  .WithTopic("sensors/parking:insert")
                  .WithPayload(json)
                  .WithQualityOfServiceLevel(MQTTnet.Protocol.MqttQualityOfServiceLevel.AtLeastOnce)
                  .Build();
              await mqttClient.PublishAsync(message);
              Console.WriteLine($"MQTT Published: {json}");
            }
          }
        }
        catch (Exception ex)
        {
          Console.WriteLine($"ERROR: {ex.Message}");
        }
        await Task.Delay(300 * 1000); // 300 seconds = 5 minutes
      }
    }

Common errors

Here are some common errors we have encountered when implementing Services and Mockups. Note that most of these error messages are found in container logs, usually middleware or the Service worked on.

• Name or service not known
• Connecting with MQTT server failed (NotAuthorized)
• 42P01: relation "t_sensors_studying" does not exist
• 28P01: password authentication failed for user ...

Name or service not known

Usually happens whenever the connectionString in the Service points to the wrong hostname. The naming in the connectionString happens automatically by the script, but it will try to find a container with that name. To use the studying example from above, it will try to find a container named studying-db. It can be a plentliful of things, but all have to do with Docker. The Service and the database have to be in the same network for them to be able to communicate, and the compose file needs to have the database named exactly studying-db.

Connecting with MQTT server failed (NotAuthorized)

This happens when the middleware is trying to connect to the MQTT broker, but the given username or password is wrong. When you create a new Service you shouldn't need to edit the middleware at all. This error should never happen unless you are actively editing the middleware.

42P01: relation "t_sensors_studying" does not exist

This error happens when the database query in DatabaseQueries.cs in your Service cannot find the table you are specifying. Since this is a string it is common that mistypes is the issue. If you are unsure what the name for the table is you can see its name in the middleware logs, looking something like:

[insert]->sensors
Creating table t_sensors_studying
Added data from sensors/studying to table t_sensors_studying in postgres

28P01: password authentication failed for user ...

Wrong username or password for database Happens when the wrong username or password is entered in the connectionString to the Service specific database. This can happen in both middleware and the Service worked on since both needs to access it. The connectionString in the middleware and Service worked on should use the username and password provided by the environment variables, so potential issues can be that the wrong variables are being retrieved or something similar. It might also be that the environment file is empty or are missing the username or password for that database. Running the installation script could fix the issue.