Basic CI/CD integration #4

Merged
asohh merged 10 commits from workflows into container 2025-10-12 17:07:02 +02:00
15 changed files with 105 additions and 175 deletions

View file

@ -0,0 +1,34 @@
name: Build Hello World Image
on:
push:
branches:
- main
workflow_dispatch:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Container Registry
uses: docker/login-action@v3
with:
registry: git.hamburg.ccc.de
username: ${{ secrets.REGISTRY_USERNAME }}
password: ${{ secrets.REGISTRY_TOKEN }}
- name: Build and push web image to Container Registry
run: |
docker build -f Containerfile -t git.hamburg.ccc.de/ccchh/sunders/web:latest .
docker push git.hamburg.ccc.de/ccchh/sunders/web:latest
- name: Start Docker Compose services
run: |
docker compose up -d --wait
docker compose down

View file

@ -1,11 +0,0 @@
FROM docker.io/library/alpine
RUN apk add mariadb-client
COPY ./data_init/*.sql /var/sunders/init.sql
CMD mariadb --host="$MYSQL_HOST" \
--user="$CAMERA_USER" \
--password="$CAMERA_USER_PASSWD" \
--skip-ssl \
${MYSQL_DB:+--database="$MYSQL_DB"} \
< "/var/sunders/init.sql"

121
README.md
View file

@ -22,124 +22,35 @@ Color | Description
A running instance of this project can be visited at [https://sunders.uber.space/](https://sunders.uber.space/). A running instance of this project can be visited at [https://sunders.uber.space/](https://sunders.uber.space/).
## Prerequisites
- [Docker](https://www.docker.com)
## Installation ## Installation
If you like to run Surveillance under Surveillance on your own LAMP or LNMP server follow these steps: To run Surveillance using docker, decide between development/testing or production deployments:
1. Get the sources ### Development/testing
- Copy the content of **home/sunders/** to your home directory, e.g. to **~/sunders/**. 1. Clone this repository
- Copy the content of **www/sunders/** to your server's www directory, e.g. to **/var/www/sunders/**. 2. Run `docker compose up` and wait for services to start up
2. Set up the database 3. Visit `http://localhost:8080/` in a browser; you should see an interactive OpenStreetMap
- Change to the directory **~/sunders/init_cameras/db/**. _tbd.: Populate camera data_
- Open the file **createDB.sql** and enter a password for the new database user **camera**. ### Production
- Create the database **camera** by executing the file **createDB.sql**. 1. Close this repository
`mysql -h localhost -u root --password=[mysql root password] < createDB.sql` 2. Hardern security by opening `docker-compose.yml` and modifying `MYSQL_PASSWORD` and `MYSQL_ROOT_PASSWORD`
3. Initialize the database 3. Run `docker compose up` and wait for services to start up
Decide whether you like to start with the surveillance entries of the first planet.osm file from September 12, 2012 or if you like to start with the latest planet.osm file or an extract of an individual country or region. 3. Visit `http://localhost:8080/` in a browser to open the web interface
**Start with the surveillance entries of the first planet.osm file from September 12, 2012** _tbd.: Populate camera data_
Pros: You don't have to download the latest +50GB planet.osm file to create a sql import file.
Cons: It could take several days until your database is up-to-date and contains all surveillance entries that have been added between September 12, 2012 and today. Furthermore you start with a data record for the whole planet. Maybe you are only interested in the data of a certain country or region.
- Execute the file **initializeDB_planet_20120912.sql** for database user **camera**.
`mysql camera -h localhost -u camera --password=[camera user password] < initializeDB_planet_20120912.sql`
**Start with the latest planet.osm file or an extract of an individual country or state**
Pros: You start with the latest data records. Furthermore you can choose what country or region you like to map.
Cons: If you like to map the whole planet you have to download the latest +50GB planet.osm file. According to your internet connection this could take a while. At last you have to install the command line Openstreetmap data processor [Osmosis](https://wiki.openstreetmap.org/wiki/Osmosis) on your computer.
- Download the latest osm.bz2 file you like to extract the data from, e.g. from [planet.openstreetmap.org](https://planet.openstreetmap.org/) or from [download.geofabrik.de](http://download.geofabrik.de/). They also offer a MD5 sum to verify the downloaded file.
- Copy the just downloaded osm.bz2 file to the directory **~/sunders/init_cameras/**.
- Open the file **~/sunders/init_cameras/createInitialDataFiles.sh** and enter the name of the osm.bz2 file as **XML_FILE**.
`XML_FILE=[file name].osm.bz2`
- Execute **createInitialDataFiles.sh** to create the files **surveillance.osm** and **initializeDB.sql**.
- Move the new files **surveillance.osm** and **initializeDB.sql** to the directory **~/sunders/init_cameras/db/** and change to that directory.
- Execute the file **initializeDB.sql** for database user **camera**.
`mysql camera -h localhost -u camera --password=[camera user password] < initializeDB.sql`
4. Update the database
- Change to the directory **~/sunders/update_cameras/**.
- Rename the file **config.php.example** to **config.php**.
- Open the file **config.php** and enter the **MYSQL_PASSWD** of the database user **camera**. Furthermore enter the **REPLICATE_URL** that fits to your project, e.g. from [planet.openstreetmap.org](https://planet.openstreetmap.org/replication/) or from [download.openstreetmap.fr](http://download.openstreetmap.fr/replication/). Here are some examples:
`https://planet.openstreetmap.org/replication/minute/`
`http://download.openstreetmap.fr/replication/planet/minute/`
`http://download.openstreetmap.fr/replication/europe/minute/`
`http://download.openstreetmap.fr/replication/europe/netherlands/minute/`
- The update process is based on the sequence number comparison between the current **state.txt** file from the replication server, and the locally stored **lastState.txt**. So if you downloaded a osm.bz2 file, you should modify the **sequenceNumber** in the **lastState.txt** file accordingly.
- Execute the file **update_camera.sh** to import all surveillance entries that have been added between the creation of the osm.bz2 file and today.
5. Schedule automatic database updates
- Add this line to your crontab:
`* * * * * /home/[user]/sunders/update_cameras/update_camera.sh > /dev/null 2>&1`
- Go to the directory **~/sunders/update_cameras/logs** to check if your schedule works. After one minute there should be a new log file.
- Go back to your crontab to change the schedule to the values you prefer, e.g. to `23 * * * *` to run the update at every 23rd minute past every hour.
6. Configure the website
- Change to the directory **/var/www/sunders/**.
- Rename the file **config.php.example** to **config.php**.
- Open the file **config.php** and change the definitions of **DEFAULT_ZOOM**, **DEFAULT_LAT**, and **DEFAULT_LON** to set the initial focus of the map to the location you want. Furthermore you can set the **DEFAULT_LANGUAGE**.
7. Check the website
- Enter the URL of your Surveillance under Surveillance instance (e.g. https://myserver/sunders/) into your browser.
- You should see a map with camera icons now.
8. Optional: Add statistics table to your database
- Go to the directory **~/sunders/init_cameras/db/**.
- Execute the file **addStatistics.sql** for database user **camera**.
`mysql camera -h localhost -u camera --password=[camera user password] < addStatistics.sql`
- Open the file **~/sunders/update_cameras/config.php**. Change the value of **USE_STATISTICS** from **false** to **true** and enter the username of your GeoNames account at **WEBSERVICE_USER**.
`define('USE_STATISTICS', true);`
- Open the file **/var/www/sunders/config.php** and change the value of **USE_STATISTICS** from **false** to **true**.
`define('USE_STATISTICS', true);`
- Schedule automatic statistics updates by adding this new line to your crontab. Enter your preferred schedule values, e.g. `42 * * * *` to run the update at every 42nd minute past every hour.
`42 * * * * /home/[user]/sunders/update_cameras/update_statistics.sh > /dev/null 2>&1`
- Check the statistics by entering the URL of the visualization site (e.g. https://myserver/sunders/stats/) into your browser.
## Surveillance nodes ## Surveillance nodes

32
docker-compose.yml Normal file
View file

@ -0,0 +1,32 @@
version: '3.8'
services:
db:
image: mariadb:12.0.2
environment:
MYSQL_ROOT_PASSWORD: rootpassword # ${{secrets.MYSQL_ROOT_PASSWORD}}
MYSQL_DATABASE: camera # ${{secrets.MYSQL_DATABASE}}
MYSQL_USER: camera # ${{secrets.MYSQL_USER}}
MYSQL_PASSWORD: camerapassword # ${{secrets.MYSQL_PASSWORD}}
ports:
- "3306:3306"
healthcheck:
test: ["CMD", "mariadb-admin", "ping", "-h", "localhost", "-uroot", "-prootpassword"]
interval: 10s
timeout: 5s
retries: 5
web:
image: git.hamburg.ccc.de/ccchh/sunders/web:latest
environment:
MYSQL_HOST: db
MYSQL_DB: camera # ${{secrets.MYSQL_DATABASE}}
MYSQL_USER: camera # ${{secrets.MYSQL_USER}}
MYSQL_PASSWORD: camerapassword # ${{secrets.MYSQL_PASSWORD}}
DEFAULT_ZOOM: 12
DEFAULT_LAT: 0
DEFAULT_LON: 0
DEFAULT_LANGUAGE: en
ports:
- "8080:80"
depends_on:
- db

View file

@ -1,15 +0,0 @@
<?php
define('REPLICATE_URL', 'https://planet.openstreetmap.org/replication/minute');
define('MYSQL_HOST', 'localhost');
define('MYSQL_DB', 'camera');
define('MYSQL_USER', 'camera');
define('MYSQL_PASSWD', 'xxxxxxxx');
define('USE_STATISTICS', false);
define('WEBSERVICE_COUNTRY_URL', 'http://api.geonames.org/countryCode');
define('WEBSERVICE_USER', 'xxxxxxxx');
define('MAX_WEBREQUESTS_PER_HOUR', 1000);
?>

View file

@ -15,7 +15,7 @@ $id = 0;
$latitude = 0; $latitude = 0;
$longitude = 0; $longitude = 0;
$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB);
if($mysqli->connect_errno) { if($mysqli->connect_errno) {
echo "Error while connecting to DB : $mysqli->error \n" ; echo "Error while connecting to DB : $mysqli->error \n" ;

View file

@ -4,7 +4,7 @@ include "config.php";
if (USE_STATISTICS) { if (USE_STATISTICS) {
$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB);
if ($mysqli->connect_errno) { if ($mysqli->connect_errno) {
echo "Error while connecting to DB : ".$mysqli->error." \n" ; echo "Error while connecting to DB : ".$mysqli->error." \n" ;

View file

@ -5,7 +5,7 @@ define('REPLICATE_URL', 'https://planet.openstreetmap.org/replication/minute');
define('MYSQL_HOST', getenv('MYSQL_HOST')); define('MYSQL_HOST', getenv('MYSQL_HOST'));
define('MYSQL_DB', getenv('MYSQL_DB')); define('MYSQL_DB', getenv('MYSQL_DB'));
define('MYSQL_USER', getenv('MYSQL_USER')); define('MYSQL_USER', getenv('MYSQL_USER'));
define('MYSQL_PASSWD', getenv('MYSQL_PASSWD')); define('MYSQL_PASSWORD', getenv('MYSQL_PASSWORD'));
define('USE_STATISTICS', false); define('USE_STATISTICS', false);
define('WEBSERVICE_COUNTRY_URL', 'http://api.geonames.org/countryCode'); define('WEBSERVICE_COUNTRY_URL', 'http://api.geonames.org/countryCode');

View file

@ -2,7 +2,7 @@
include "config.php"; include "config.php";
$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB);
if($mysqli->connect_errno) { if($mysqli->connect_errno) {
echo "Error while connecting to DB : $mysqli->error \n" ; echo "Error while connecting to DB : $mysqli->error \n" ;

View file

@ -2,15 +2,15 @@
// Admin credentials (must have privileges to create DB/users) // Admin credentials (must have privileges to create DB/users)
$dbHost = getenv('MYSQL_HOST'); $dbHost = getenv('MYSQL_HOST');
$dbAdmin = getenv('MYSQL_USER'); $dbAdmin = getenv('MYSQL_USER');
$dbPassword = getenv('MYSQL_PASSWD'); $dbPassword = getenv('MYSQL_PASSWORD');
$dbName = getenv('MYSQL_DB'); $dbName = getenv('MYSQL_DB');
// Variables for new users // Variables for new users
$cameraUser = getenv('CAMERA_USER'); $cameraUser = getenv('CAMERA_USER');
$cameraPassword = getenv('CAMERA_USER_PASSWD'); $cameraPassword = getenv('CAMERA_USER_PASSWORD');
$camSelectUser = getenv('CAMERA_SELECT_USER'); $camSelectUser = getenv('CAMERA_SELECT_USER');
$camSelectPassword = getenv('CAMERA_SELECT_USER_PASSWD'); $camSelectPassword = getenv('CAMERA_SELECT_USER_PASSWORD');
// Connect to MySQL // Connect to MySQL
$conn = new mysqli($dbHost, $dbAdmin, $dbPassword); $conn = new mysqli($dbHost, $dbAdmin, $dbPassword);

View file

@ -15,7 +15,7 @@ $id = 0;
$latitude = 0; $latitude = 0;
$longitude = 0; $longitude = 0;
$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB);
if($mysqli->connect_errno) { if($mysqli->connect_errno) {
echo "Error while connecting to DB : $mysqli->error \n" ; echo "Error while connecting to DB : $mysqli->error \n" ;

View file

@ -266,7 +266,7 @@
$divDiag2 = ($divWidth * $divWidth) + ($divHeight * $divHeight); $divDiag2 = ($divWidth * $divWidth) + ($divHeight * $divHeight);
/* Connect to database */ /* Connect to database */
$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB);
if($mysqli->connect_errno) { if($mysqli->connect_errno) {
header('Content-type: application/json'); header('Content-type: application/json');
$result = '{"error":"error while connecting to db : ' . $mysqli->error . '"}'; $result = '{"error":"error while connecting to db : ' . $mysqli->error . '"}';

View file

@ -1,19 +1,15 @@
<?php <?php
define('DEFAULT_ZOOM', getenv('DEFAULT_ZOOM') ?: 11);
define('DEFAULT_LAT', getenv('DEFAULT_LAT') ?: 53.5550);
define('DEFAULT_LON', getenv('DEFAULT_LON') ?: 10.0099);
define('DEFAULT_LANGUAGE', getenv('DEFAULT_LANGUAGE') ?: 'en');
define('DEFAULT_PIE', 'country');
define('DEFAULT_TIME', 'single');
define('DEFAULT_ZOOM', 15); define('MYSQL_HOST', getenv('MYSQL_HOST') ?: '');
define('DEFAULT_LAT', 52.37672572); define('MYSQL_DB', getenv('MYSQL_DB') ?: '');
define('DEFAULT_LON', 9.73787785); define('MYSQL_USER', getenv('MYSQL_USER') ?: '');
define('DEFAULT_LANGUAGE', 'en'); define('MYSQL_PASSWORD', getenv('MYSQL_PASSWORD') ?: '');
define('DEFAULT_PIE', 'country');
define('DEFAULT_TIME', 'single');
define('MYSQL_HOST', getenv('MYSQL_HOST'));
define('MYSQL_DB', getenv('MYSQL_DB'));
define('MYSQL_USER', getenv('MYSQL_USER'));
define('MYSQL_PASSWD', getenv('MYSQL_PASSWD'));
define('USE_STATISTICS', false);
define('USE_STATISTICS', false);
?> ?>

View file

@ -1,17 +0,0 @@
<?php
define('DEFAULT_ZOOM', 15);
define('DEFAULT_LAT', 52.37672572);
define('DEFAULT_LON', 9.73787785);
define('DEFAULT_LANGUAGE', 'en');
define('DEFAULT_PIE', 'country');
define('DEFAULT_TIME', 'single');
define('MYSQL_HOST', 'localhost');
define('MYSQL_DB', 'camera');
define('MYSQL_USER', 'camselect');
define('MYSQL_PASSWD', null);
define('USE_STATISTICS', false);
?>

View file

@ -138,7 +138,7 @@
$statsQueryObject = $levelObject->{$statsKey}; $statsQueryObject = $levelObject->{$statsKey};
/* Connect to database */ /* Connect to database */
$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB);
if($mysqli->connect_errno) { if($mysqli->connect_errno) {
echo 'Error while connecting to DB : $mysqli->error \n' ; echo 'Error while connecting to DB : $mysqli->error \n' ;
exit(1); exit(1);