Webpage for the project down #959

Closed
opened 2026-05-14 12:31:55 +00:00 by Andras · 15 comments
Owner

http://193.93.250.90/app/

is down on a non-work day. We need service windows, but it is crucial that the service is available when not being actively maintained.

[http://193.93.250.90/app/](http://193.93.250.90/app/) is down on a non-work day. We need service windows, but it is crucial that the service is available when not being actively maintained.
Collaborator

We're aware of this, this happened during deployment on Wednesday at 19:30. Since it was already out of work hours i didn't spend too much time on figuring out why the deployment failed.

This will hopefully be resolved today.

We're aware of this, this happened during deployment on Wednesday at 19:30. Since it was already out of work hours i didn't spend too much time on figuring out why the deployment failed. This will hopefully be resolved today.
Collaborator

Tried deploying again and it failed, i suspected that the "nohup" in the command wasnt working and that it wasnt executed remotely on the server with SSH-disonnects. I can now dismiss this theory as i kept my SSH-connected through out the whole deployment session.
LOG:
image

Tried deploying again and it failed, i suspected that the "nohup" in the command wasnt working and that it wasnt executed remotely on the server with SSH-disonnects. I can now dismiss this theory as i kept my SSH-connected through out the whole deployment session. **LOG:** ![image](/attachments/deea4691-961d-4659-8d62-65727577103c)
119 KiB
Collaborator

For the question about availability, we have no current monitoring for the server. We don't actually know when or if the webpage goes down. Our suggestion on using Prometheus for notifications on down time was disregarded by the customer, how are we gonna support availability if we cant get information about it?

For the question about availability, we have no current monitoring for the server. We don't actually know **when** or **if** the webpage goes down. Our suggestion on using Prometheus for notifications on down time was disregarded by the customer, how are we gonna support availability if we cant get information about it?
Collaborator

Tried to reinstall the whole project on the server but it didnt work either, apperantly the docker container "RoomBookingApiMockup" doesnt want to shutdown and has been up for 6 days.

Tried to reinstall the whole project on the server but it didnt work either, apperantly the docker container "RoomBookingApiMockup" doesnt want to shutdown and has been up for 6 days.
Collaborator

@b24johka wrote in #959 (comment):

Tried to reinstall the whole project on the server but it didnt work either, apperantly the docker container "RoomBookingApiMockup" doesnt want to shutdown and has been up for 6 days.

did you try to close all containers manually and the start all?

@b24johka wrote in https://git.webug.se/Andras/BoundlessFlowCampus2K/issues/959#issuecomment-10686: > Tried to reinstall the whole project on the server but it didnt work either, apperantly the docker container "RoomBookingApiMockup" doesnt want to shutdown and has been up for 6 days. did you try to close all containers manually and the start all?
Collaborator

Had to terminate the container "RoomBookingApiMockup" by remove orphans. Apparently containers which was is not in the compose.yaml will not be removed with docker compose down -v, which is executed on the server through the rebuild script.
This is likely the causing issue since RoomBookingApiMockup was removed from compose.yaml in the merge to main.

image

Currently i have restarted the server, run docker stop all and the deployment script is executing.

Had to terminate the container "RoomBookingApiMockup" by remove orphans. Apparently containers which was is not in the compose.yaml will not be removed with docker compose down -v, which is executed on the server through the rebuild script. This is likely the causing issue since RoomBookingApiMockup was removed from compose.yaml in the merge to main. ![image](/attachments/d6414b8e-6cf7-4f5f-87f3-b30ba84da668) Currently i have restarted the server, run docker stop all and the deployment script is executing.
Collaborator

@a24sakma wrote in #959 (comment):

@b24johka wrote in #959 (comment):

Tried to reinstall the whole project on the server but it didnt work either, apperantly the docker container "RoomBookingApiMockup" doesnt want to shutdown and has been up for 6 days.

did you try to close all containers manually and the start all?

I haven't restarted them manually since the rebuild script is the deploy standard. If the current execution wont work i will try to rebuild everything manually. Good Idea!

@a24sakma wrote in https://git.webug.se/Andras/BoundlessFlowCampus2K/issues/959#issuecomment-11056: > @b24johka wrote in #959 (comment): > > > Tried to reinstall the whole project on the server but it didnt work either, apperantly the docker container "RoomBookingApiMockup" doesnt want to shutdown and has been up for 6 days. > > did you try to close all containers manually and the start all? I haven't restarted them manually since the rebuild script is the deploy standard. If the current execution wont work i will try to rebuild everything manually. Good Idea!
Collaborator

It was identified that the monolithic rebuild script (Which the server uses), had not been updated which explains a lot.
Fixed it with a updater rebuild script, now most of the containers are starting but some are not starting properly.
image

image

It was identified that the monolithic rebuild script (Which the server uses), had not been updated which explains a lot. Fixed it with a updater rebuild script, now most of the containers are starting but some are not starting properly. ![image](/attachments/55eef146-6976-4f23-9143-71cdc5acae33) ![image](/attachments/f2fedfca-612e-4446-8fbb-305f9d966027)
Collaborator

The script was the biggest problem with the deployment, we should also consider adding a docker compose down --remove-orphans to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file.

We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform.
The website is now deployed and all of the containers are healthy.
http://193.93.250.90/app/

image

The script was the biggest problem with the deployment, we should also consider adding a **docker compose down --remove-orphans** to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file. We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform. The website is now deployed and all of the containers are healthy. http://193.93.250.90/app/ ![image](/attachments/94bf6777-1f05-430f-806b-64c107a76186)
115 KiB
Collaborator

@b24johka wrote in #959 (comment):

The script was the biggest problem with the deployment, we should also consider adding a docker compose down --remove-orphans to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file.

We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform. The website is now deployed and all of the containers are healthy. http://193.93.250.90/app/

image

Well done! I think sometimes the containers does not get cleaned up when you only use the script therefor some manual steps and retry with script can do some magic :)

Yeah I agree. You can open a research on automation of the scripts and a sync between them

@b24johka wrote in https://git.webug.se/Andras/BoundlessFlowCampus2K/issues/959#issuecomment-11136: > The script was the biggest problem with the deployment, we should also consider adding a **docker compose down --remove-orphans** to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file. > > We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform. The website is now deployed and all of the containers are healthy. http://193.93.250.90/app/ > > [![image](/attachments/94bf6777-1f05-430f-806b-64c107a76186)](/Andras/BoundlessFlowCampus2K/attachments/94bf6777-1f05-430f-806b-64c107a76186) Well done! I think sometimes the containers does not get cleaned up when you only use the script therefor some manual steps and retry with script can do some magic :) Yeah I agree. You can open a research on automation of the scripts and a sync between them
Collaborator

Should "docker compose down --remove-orphans" be added to the script or should the issue be closed @b24johka ?

Should "docker compose down --remove-orphans" be added to the script or should the issue be closed @b24johka ?
Collaborator

@a24sakma wrote in #959 (comment):

@b24johka wrote in #959 (comment):

The script was the biggest problem with the deployment, we should also consider adding a docker compose down --remove-orphans to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file.
We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform. The website is now deployed and all of the containers are healthy. http://193.93.250.90/app/
image

Well done! I think sometimes the containers does not get cleaned up when you only use the script therefor some manual steps and retry with script can do some magic :)

Yeah I agree. You can open a research on automation of the scripts and a sync between them

The server is up and running all the time except for a short moment during rebuild, therefor old containers which exists in prior versions will be kept running unless we include the --remove-orphans in the rebuild script.
We should really avoid going in and do manual changes on server because starting containers can take up to 1h, and that would increase down time.

@a24sakma wrote in https://git.webug.se/Andras/BoundlessFlowCampus2K/issues/959#issuecomment-11180: > @b24johka wrote in #959 (comment): > > > The script was the biggest problem with the deployment, we should also consider adding a **docker compose down --remove-orphans** to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file. > > We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform. The website is now deployed and all of the containers are healthy. http://193.93.250.90/app/ > > [![image](/attachments/94bf6777-1f05-430f-806b-64c107a76186)](/Andras/BoundlessFlowCampus2K/attachments/94bf6777-1f05-430f-806b-64c107a76186) > > Well done! I think sometimes the containers does not get cleaned up when you only use the script therefor some manual steps and retry with script can do some magic :) > > Yeah I agree. You can open a research on automation of the scripts and a sync between them The server is up and running all the time except for a short moment during rebuild, therefor old containers which exists in prior versions will be kept running unless we include the **--remove-orphans** in the rebuild script. We should really avoid going in and do manual changes on server because starting containers can take up to 1h, and that would increase down time.
Collaborator

@a24julot wrote in #959 (comment):

Should "docker compose down --remove-orphans" be added to the script or should the issue be closed @b24johka ?

I think it should be added to the rebuild script, i will create an issue on this.

@a24julot wrote in https://git.webug.se/Andras/BoundlessFlowCampus2K/issues/959#issuecomment-11250: > Should "docker compose down --remove-orphans" be added to the script or should the issue be closed @b24johka ? I think it should be added to the rebuild script, i will create an issue on this.
Collaborator

@b24johka wrote in #959 (comment):

@a24sakma wrote in #959 (comment):

@b24johka wrote in #959 (comment):

The script was the biggest problem with the deployment, we should also consider adding a docker compose down --remove-orphans to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file.
We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform. The website is now deployed and all of the containers are healthy. http://193.93.250.90/app/
image

Well done! I think sometimes the containers does not get cleaned up when you only use the script therefor some manual steps and retry with script can do some magic :)
Yeah I agree. You can open a research on automation of the scripts and a sync between them

The server is up and running all the time except for a short moment during rebuild, therefor old containers which exists in prior versions will be kept running unless we include the --remove-orphans in the rebuild script. We should really avoid going in and do manual changes on server because starting containers can take up to 1h, and that would increase down time.

I meant like debugging when the script is not working but if it takes that that much time then maybe it is irrelevant here. :)

@b24johka wrote in https://git.webug.se/Andras/BoundlessFlowCampus2K/issues/959#issuecomment-11253: > @a24sakma wrote in #959 (comment): > > > @b24johka wrote in #959 (comment): > > > The script was the biggest problem with the deployment, we should also consider adding a **docker compose down --remove-orphans** to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file. > > > We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform. The website is now deployed and all of the containers are healthy. http://193.93.250.90/app/ > > > [![image](/attachments/94bf6777-1f05-430f-806b-64c107a76186)](/Andras/BoundlessFlowCampus2K/attachments/94bf6777-1f05-430f-806b-64c107a76186) > > > > > > Well done! I think sometimes the containers does not get cleaned up when you only use the script therefor some manual steps and retry with script can do some magic :) > > Yeah I agree. You can open a research on automation of the scripts and a sync between them > > The server is up and running all the time except for a short moment during rebuild, therefor old containers which exists in prior versions will be kept running unless we include the **--remove-orphans** in the rebuild script. We should really avoid going in and do manual changes on server because starting containers can take up to 1h, and that would increase down time. I meant like debugging when the script is not working but if it takes that that much time then maybe it is irrelevant here. :)
Collaborator

This has been fixed and an issue for fixing script is created. I close this issue

This has been fixed and an issue for fixing script is created. I close this issue
Sign in to join this conversation.
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Andras/BoundlessFlowCampus2K#959
No description provided.