Adds health checks to web and data_handler container; restarting on failure #24

Open

ViMaSter wants to merge 15 commits from healthcheck into main

pull from: healthcheck

merge into: CCCHH:main

CCCHH:main

CCCHH:renovate/actions-checkout-5.x

CCCHH:renovate/quay.io-fedora-fedora-minimal-44.x

CCCHH:sunders-18

CCCHH:impressum-as-var

Conversation 1 Commits 15 Files changed 12 +106 -16

ViMaSter commented 2025-10-25 21:30:35 +02:00

Collaborator

Copy link

Increases the resilience of most containers a bit:
fd6b1d680c: web health check verifies the camera endpoint returns at least one position when zoomed all the way out (shows most of Europe and parts of Asia)
58f048adba: Adding a health check to data_handler required more verbose output; before this PR, all logs generated by dcrond and its jobs were silenced; they're now stored in /var/log
f94080f45f: Previously, update_camera.sh was only started every hour at xx:05; now it's also run immediately after the container starts
73e9c279df: Adds a health check to data_handler, which remains healthy unless the newest update_camera_ log file contains the word error
82b41342fc: Restarts any container if the associated health check fails (shutdowns by users still allowed)

Increases the resilience of most containers a bit: fd6b1d680c: `web` health check verifies the camera endpoint returns at least one position when zoomed all the way out (shows most of Europe and parts of Asia) 58f048adba: Adding a health check to `data_handler` required more verbose output; before this PR, all logs generated by `dcrond` and its jobs were silenced; they're now stored in `/var/log` f94080f45f: Previously, `update_camera.sh` was only started every hour at `xx:05`; now it's also run immediately after the container starts 73e9c279df: Adds a health check to `data_handler`, which remains `healthy` unless the newest `update_camera_` log file contains the word `error` 82b41342fc: Restarts any container if the associated health check fails (shutdowns by users still allowed)

ViMaSter added 5 commits 2025-10-25 21:30:36 +02:00

feat: Adds healthcheck to web server fd6b1d680c

refactor: Increases verbosity of data_handler 58f048adba

chore: Immediately starts update_camera on container startup f94080f45f

feat: Adds healthcheck that restarts data_handler, when any error is printed to log files 73e9c279df

fix: Restarts any container unless stopped by user 82b41342fc

ViMaSter added 2 commits 2025-11-01 20:50:41 +01:00

first changes for sync of instances ea2d8ebe37

modified health check for web 5ff2918d07

ViMaSter added 6 commits 2025-11-01 21:29:25 +01:00

feat: web healthcheck returns 503 status code if database is unavailable 729af9ced9

docs: Adds instructions about dev override ... adab9b3175

chore: Applies docker naming convention to files

fix: Enables PHP log output when running in development 907c20bad2

fix: Handles issue where error handler was skipped, if database DNS request fails 590fb8647a

docs: Removes incorrect comment dbd0ff677f

docs: Adds information about changes when running in a development environment 34339b8021

ViMaSter commented 2025-11-01 21:43:19 +01:00

Author

Collaborator

Copy link

• 729af9ced9 applys the changes from #25 to the production healthcheck in docker-compose.yml. I noticed the health check never failed, as curl just asserts the 200 status code and doesn't care about the JSON content; I made it return 503 as well
• 590fb8647a resolves an issue where new mysqli would fail if the database wasn't online; it would return the error JSON if the DB was available but the credentials were incorrect, but an offline databse host would throw before if($mysqli->connect_errno) got evaluated; I just wrapped the whole thing in a try/catch block and returned any and all errors as 503 with {"error":"Database unavailable"}
• As Docker uses .dev.yml for most files, I applied that convention in adab9b3175; I also removed all redundant entries, so we share most of the configuration with production and development environments only override what they need in docker-compose.dev.yml
• 34339b8021 adds these instructions (and its consequences) to README.MD
• 907c20bad2 adds APP_ENV=development when running .dev.yml and makes the app stop silencing errors while developing by default (tracing the SQL issue without was a bit difficult : D), and it's a one-line change to re-enable error silencing that's documented in the README

- 729af9ced9520f20c58bb45e5e02faea02212ab5 applys the changes from #25 to the production healthcheck in `docker-compose.yml`. I noticed the health check never failed, as curl just asserts the `200` status code and doesn't care about the JSON content; I made it return 503 as well - 590fb8647ab1b806a1598e020d81d4718332cd01 resolves an issue where `new mysqli` would fail if the database wasn't online; it would return the error JSON if the DB was available but the credentials were incorrect, but an offline databse host would throw before `if($mysqli->connect_errno)` got evaluated; I just wrapped the whole thing in a try/catch block and returned any and all errors as `503` with `{"error":"Database unavailable"}` - As Docker uses `.dev.yml` for most files, I applied that convention in adab9b31756a78180d209f7fd4ef86e2a0eaeb71; I also removed all redundant entries, so we share most of the configuration with production and development environments only override what they need in `docker-compose.dev.yml` - 34339b8021b14ee91cc744a59130e40210f46688 adds these instructions (and its consequences) to `README.MD` - 907c20bad29414d8d41f51503cebe9454d561c5c adds `APP_ENV=development` when running `.dev.yml` and makes the app stop silencing errors while developing by default (tracing the SQL issue without was a bit difficult : D), and it's a one-line change to re-enable error silencing that's documented in the README

ViMaSter added 1 commit 2025-11-01 21:49:49 +01:00

fix: Adds missing documentation hint for secret replacement c2a8412c4b

ViMaSter added 1 commit 2025-11-02 12:47:36 +01:00

revert: Uses previous port mapping e601c6df58

Some checks failed

Hide all checks

images.yml / revert: Uses previous port mapping (push) Failing after 0s

Details

images.yml / revert: Uses previous port mapping (pull_request) Failing after 0s

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin healthcheck:healthcheck

git switch healthcheck

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

CCCHH/sunders!24

No description provided.