forked from kamba4/sunders
Adds health checks to web and data_handler container; restarting on failure #24
stb
Loading…
Reference in a new issue
No description provided.
Delete branch "healthcheck"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Increases the resilience of most containers a bit:
fd6b1d680c:webhealth check verifies the camera endpoint returns at least one position when zoomed all the way out (shows most of Europe and parts of Asia)58f048adba: Adding a health check todata_handlerrequired more verbose output; before this PR, all logs generated bydcrondand its jobs were silenced; they're now stored in/var/logf94080f45f: Previously,update_camera.shwas only started every hour atxx:05; now it's also run immediately after the container starts73e9c279df: Adds a health check todata_handler, which remainshealthyunless the newestupdate_camera_log file contains the worderror82b41342fc: Restarts any container if the associated health check fails (shutdowns by users still allowed)update_cameraon container startup f94080f45fdata_handler, when any error is printed to log files 73e9c279dfwebhealthcheck returns 503 status code if database is unavailable 729af9ced9devoverride adab9b3175development907c20bad2729af9ced9applys the changes from #25 to the production healthcheck indocker-compose.yml. I noticed the health check never failed, as curl just asserts the200status code and doesn't care about the JSON content; I made it return 503 as well590fb8647aresolves an issue wherenew mysqliwould fail if the database wasn't online; it would return the error JSON if the DB was available but the credentials were incorrect, but an offline databse host would throw beforeif($mysqli->connect_errno)got evaluated; I just wrapped the whole thing in a try/catch block and returned any and all errors as503with{"error":"Database unavailable"}.dev.ymlfor most files, I applied that convention inadab9b3175; I also removed all redundant entries, so we share most of the configuration with production and development environments only override what they need indocker-compose.dev.yml34339b8021adds these instructions (and its consequences) toREADME.MD907c20bad2addsAPP_ENV=developmentwhen running.dev.ymland makes the app stop silencing errors while developing by default (tracing the SQL issue without was a bit difficult : D), and it's a one-line change to re-enable error silencing that's documented in the README@ -9,3 +9,3 @@cat<<EOF | dcrontab -# min hour day month weekday command5 * * * * sh /opt/update_camera.sh*/15 * * * * sh /opt/update_camera.sh >> /var/log/update_camera_$(date +\%Y\%m\%d\%H\%M\%S).log 2>&1Füllt das nicht die Platte? Warum nicht einfach nach /dev/console, dann landet es im Container-Log?
Der Health-Check liest die. Ich habs
5ce403cec2in ein Status-File umgebogen, dass nicht ganze Log geschrieben wird.@ -24,0 +24,4 @@####################################################################################################echo "Running update once initially..."sh /opt/update_camera.sh >> /var/log/update_camera_$(date +\%Y\%m\%d\%H\%M\%S).log 2>&1s.o.
#24 (comment)
@ -11,3 +12,3 @@- mariadb:/var/lib/mysqlhealthcheck:test: ["CMD", "mariadb-admin", "ping", "-h", "localhost", "-uroot", "-prootpassword"]test: ["CMD", "mariadb-admin", "ping", "-h", "localhost", "-uroot", "-prootpassword"] # ${{secrets.MYSQL_ROOT_PASSWORD}}Warum das Passwort in einen Kommentar schreiben?
Das Repo selbst hatte nichts nach Prod deployt. Jemand aus dem Infra Team hatte uns gebeten hier einfach nur zu kennzeichnen, wo welche Secrets hinsollten, weil das Compose-File quasi übernommen wurde und dann Secrets gesetzt wurden. Je nach dem wie wir jetzt deployen, kann das gern raus.
6cd0d1ea4baa6ef7f362