Webpage for the project down #959
Labels
No labels
_CRITICAL_
API
app
backEnd
Blocked, waiting for further changes
bug
cleanup
close
design
duplicate
enhancement
feature request
frontEnd
help wanted
invalid
low priority
needs input
needs review
project documentation
question
research
reviewed
script
security
SQL
style
testing
topLevel
wontfix
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
Andras/BoundlessFlowCampus2K#959
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
http://193.93.250.90/app/
is down on a non-work day. We need service windows, but it is crucial that the service is available when not being actively maintained.
We're aware of this, this happened during deployment on Wednesday at 19:30. Since it was already out of work hours i didn't spend too much time on figuring out why the deployment failed.
This will hopefully be resolved today.
Tried deploying again and it failed, i suspected that the "nohup" in the command wasnt working and that it wasnt executed remotely on the server with SSH-disonnects. I can now dismiss this theory as i kept my SSH-connected through out the whole deployment session.

LOG:
For the question about availability, we have no current monitoring for the server. We don't actually know when or if the webpage goes down. Our suggestion on using Prometheus for notifications on down time was disregarded by the customer, how are we gonna support availability if we cant get information about it?
Tried to reinstall the whole project on the server but it didnt work either, apperantly the docker container "RoomBookingApiMockup" doesnt want to shutdown and has been up for 6 days.
@b24johka wrote in #959 (comment):
did you try to close all containers manually and the start all?
Had to terminate the container "RoomBookingApiMockup" by remove orphans. Apparently containers which was is not in the compose.yaml will not be removed with docker compose down -v, which is executed on the server through the rebuild script.
This is likely the causing issue since RoomBookingApiMockup was removed from compose.yaml in the merge to main.
Currently i have restarted the server, run docker stop all and the deployment script is executing.
@a24sakma wrote in #959 (comment):
I haven't restarted them manually since the rebuild script is the deploy standard. If the current execution wont work i will try to rebuild everything manually. Good Idea!
It was identified that the monolithic rebuild script (Which the server uses), had not been updated which explains a lot.

Fixed it with a updater rebuild script, now most of the containers are starting but some are not starting properly.
The script was the biggest problem with the deployment, we should also consider adding a docker compose down --remove-orphans to the rebuild script to ensure that we delete containers that is not referenced by the compose.yaml file.
We also have to check for a better way than to have 2 separate rebuild scripts, since both needs to be uniform.
The website is now deployed and all of the containers are healthy.
http://193.93.250.90/app/
@b24johka wrote in #959 (comment):
Well done! I think sometimes the containers does not get cleaned up when you only use the script therefor some manual steps and retry with script can do some magic :)
Yeah I agree. You can open a research on automation of the scripts and a sync between them
Should "docker compose down --remove-orphans" be added to the script or should the issue be closed @b24johka ?
@a24sakma wrote in #959 (comment):
The server is up and running all the time except for a short moment during rebuild, therefor old containers which exists in prior versions will be kept running unless we include the --remove-orphans in the rebuild script.
We should really avoid going in and do manual changes on server because starting containers can take up to 1h, and that would increase down time.
@a24julot wrote in #959 (comment):
I think it should be added to the rebuild script, i will create an issue on this.
@b24johka wrote in #959 (comment):
I meant like debugging when the script is not working but if it takes that that much time then maybe it is irrelevant here. :)
This has been fixed and an issue for fixing script is created. I close this issue