From 4c7eeef1bfafa603edd7b7c0d02aff9ebfe944b9 Mon Sep 17 00:00:00 2001 From: Vincent Mahnke Date: Sat, 11 Oct 2025 21:46:54 +0200 Subject: [PATCH 1/8] docs: Replaces on-premise instructions with container instructions --- README.md | 123 ++++++++---------------------------------------------- 1 file changed, 17 insertions(+), 106 deletions(-) diff --git a/README.md b/README.md index d86ca69..d4b38c1 100644 --- a/README.md +++ b/README.md @@ -22,124 +22,35 @@ Color | Description A running instance of this project can be visited at [https://sunders.uber.space/](https://sunders.uber.space/). +## Prerequisites + +- [Docker](https://www.docker.com) + ## Installation -If you like to run Surveillance under Surveillance on your own LAMP or LNMP server follow these steps: +To run Surveillance using docker, decide between development/testing or production deployments: -1. Get the sources +### Development/testing - - Copy the content of **home/sunders/** to your home directory, e.g. to **~/sunders/**. +1. Clone this repository - - Copy the content of **www/sunders/** to your server's www directory, e.g. to **/var/www/sunders/**. +2. Run `docker compose up` and wait for services to start up -2. Set up the database +3. Visit `http://localhost:8080/` in a browser; you should see an interactive OpenStreetMap - - Change to the directory **~/sunders/init_cameras/db/**. +_tbd.: Populate camera data_ - - Open the file **createDB.sql** and enter a password for the new database user **camera**. +### Production - - Create the database **camera** by executing the file **createDB.sql**. +1. Close this repository - `mysql -h localhost -u root --password=[mysql root password] < createDB.sql` +2. Hardern security by opening `docker-compose.yml` and modifying `MYSQL_PASSWORD` and `MYSQL_ROOT_PASSWORD` -3. Initialize the database +3. Run `docker compose up` and wait for services to start up - Decide whether you like to start with the surveillance entries of the first planet.osm file from September 12, 2012 or if you like to start with the latest planet.osm file or an extract of an individual country or region. +3. Visit `http://localhost:8080/` in a browser to open the web interface - **Start with the surveillance entries of the first planet.osm file from September 12, 2012** - - Pros: You don't have to download the latest +50GB planet.osm file to create a sql import file. - - Cons: It could take several days until your database is up-to-date and contains all surveillance entries that have been added between September 12, 2012 and today. Furthermore you start with a data record for the whole planet. Maybe you are only interested in the data of a certain country or region. - - - Execute the file **initializeDB_planet_20120912.sql** for database user **camera**. - - `mysql camera -h localhost -u camera --password=[camera user password] < initializeDB_planet_20120912.sql` - - **Start with the latest planet.osm file or an extract of an individual country or state** - - Pros: You start with the latest data records. Furthermore you can choose what country or region you like to map. - - Cons: If you like to map the whole planet you have to download the latest +50GB planet.osm file. According to your internet connection this could take a while. At last you have to install the command line Openstreetmap data processor [Osmosis](https://wiki.openstreetmap.org/wiki/Osmosis) on your computer. - - - Download the latest osm.bz2 file you like to extract the data from, e.g. from [planet.openstreetmap.org](https://planet.openstreetmap.org/) or from [download.geofabrik.de](http://download.geofabrik.de/). They also offer a MD5 sum to verify the downloaded file. - - - Copy the just downloaded osm.bz2 file to the directory **~/sunders/init_cameras/**. - - - Open the file **~/sunders/init_cameras/createInitialDataFiles.sh** and enter the name of the osm.bz2 file as **XML_FILE**. - - `XML_FILE=[file name].osm.bz2` - - - Execute **createInitialDataFiles.sh** to create the files **surveillance.osm** and **initializeDB.sql**. - - - Move the new files **surveillance.osm** and **initializeDB.sql** to the directory **~/sunders/init_cameras/db/** and change to that directory. - - - Execute the file **initializeDB.sql** for database user **camera**. - - `mysql camera -h localhost -u camera --password=[camera user password] < initializeDB.sql` - -4. Update the database - - - Change to the directory **~/sunders/update_cameras/**. - - - Rename the file **config.php.example** to **config.php**. - - - Open the file **config.php** and enter the **MYSQL_PASSWD** of the database user **camera**. Furthermore enter the **REPLICATE_URL** that fits to your project, e.g. from [planet.openstreetmap.org](https://planet.openstreetmap.org/replication/) or from [download.openstreetmap.fr](http://download.openstreetmap.fr/replication/). Here are some examples: - - `https://planet.openstreetmap.org/replication/minute/` - `http://download.openstreetmap.fr/replication/planet/minute/` - `http://download.openstreetmap.fr/replication/europe/minute/` - `http://download.openstreetmap.fr/replication/europe/netherlands/minute/` - - - The update process is based on the sequence number comparison between the current **state.txt** file from the replication server, and the locally stored **lastState.txt**. So if you downloaded a osm.bz2 file, you should modify the **sequenceNumber** in the **lastState.txt** file accordingly. - - - Execute the file **update_camera.sh** to import all surveillance entries that have been added between the creation of the osm.bz2 file and today. - -5. Schedule automatic database updates - - - Add this line to your crontab: - - `* * * * * /home/[user]/sunders/update_cameras/update_camera.sh > /dev/null 2>&1` - - - Go to the directory **~/sunders/update_cameras/logs** to check if your schedule works. After one minute there should be a new log file. - - - Go back to your crontab to change the schedule to the values you prefer, e.g. to `23 * * * *` to run the update at every 23rd minute past every hour. - -6. Configure the website - - - Change to the directory **/var/www/sunders/**. - - - Rename the file **config.php.example** to **config.php**. - - - Open the file **config.php** and change the definitions of **DEFAULT_ZOOM**, **DEFAULT_LAT**, and **DEFAULT_LON** to set the initial focus of the map to the location you want. Furthermore you can set the **DEFAULT_LANGUAGE**. - -7. Check the website - - - Enter the URL of your Surveillance under Surveillance instance (e.g. https://myserver/sunders/) into your browser. - - - You should see a map with camera icons now. - -8. Optional: Add statistics table to your database - - - Go to the directory **~/sunders/init_cameras/db/**. - - - Execute the file **addStatistics.sql** for database user **camera**. - - `mysql camera -h localhost -u camera --password=[camera user password] < addStatistics.sql` - - - Open the file **~/sunders/update_cameras/config.php**. Change the value of **USE_STATISTICS** from **false** to **true** and enter the username of your GeoNames account at **WEBSERVICE_USER**. - - `define('USE_STATISTICS', true);` - - - Open the file **/var/www/sunders/config.php** and change the value of **USE_STATISTICS** from **false** to **true**. - - `define('USE_STATISTICS', true);` - - - Schedule automatic statistics updates by adding this new line to your crontab. Enter your preferred schedule values, e.g. `42 * * * *` to run the update at every 42nd minute past every hour. - - `42 * * * * /home/[user]/sunders/update_cameras/update_statistics.sh > /dev/null 2>&1` - - - Check the statistics by entering the URL of the visualization site (e.g. https://myserver/sunders/stats/) into your browser. +_tbd.: Populate camera data_ ## Surveillance nodes @@ -232,4 +143,4 @@ Key | Value [img_example1]: ./www/sunders/images/fixed_z-17_d-90_a-15_h-5.png "Field of view (direction: 90°, angle: 15°, height: 5m)" [img_example2]: ./www/sunders/images/fixed_z-17_d-90_a-15_h-10.png "Field of view (direction: 90°, angle: 15°, height: 10m)" -[img_example3]: ./www/sunders/images/fixed_z-17_d-90_a-60_h-10.png "Field of view (direction: 90°, angle: 60°, height: 10m)" +[img_example3]: ./www/sunders/images/fixed_z-17_d-90_a-60_h-10.png "Field of view (direction: 90°, angle: 60°, height: 10m)" \ No newline at end of file From 8555a6d2416a3580a31c4d9957e6f6c7c56c837a Mon Sep 17 00:00:00 2001 From: Vincent Mahnke Date: Sat, 11 Oct 2025 21:47:23 +0200 Subject: [PATCH 2/8] feat: Adds CI/CD steps to build and push `web` container --- .forgejo/workflows/images.yml | 37 +++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) create mode 100644 .forgejo/workflows/images.yml diff --git a/.forgejo/workflows/images.yml b/.forgejo/workflows/images.yml new file mode 100644 index 0000000..68c9a89 --- /dev/null +++ b/.forgejo/workflows/images.yml @@ -0,0 +1,37 @@ +name: Build Hello World Image + +on: + push: + branches: + - main + workflow_dispatch: + +jobs: + build: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Login to Container Registry + uses: docker/login-action@v3 + with: + registry: git.hamburg.ccc.de + username: ${{ secrets.REGISTRY_USERNAME }} + password: ${{ secrets.REGISTRY_TOKEN }} + + - name: Build and push web image to Container Registry + run: | + docker build -t web:latest . + docker tag web:latest git.hamburg.ccc.de/ccchh/sunders/web:latest + docker push git.hamburg.ccc.de/ccchh/sunders/web:latest + working-directory: ./ + + - name: Start Docker Compose services + run: | + docker compose up -d --wait + docker compose down + working-directory: / \ No newline at end of file From 54c3528a76ec34657f607212aacb9b643c17e9c8 Mon Sep 17 00:00:00 2001 From: Vincent Mahnke Date: Sat, 11 Oct 2025 21:47:48 +0200 Subject: [PATCH 3/8] chore: Removes unused example files --- home/sunders/update_cameras/config.php.example | 15 --------------- www/sunders/config.php.example | 17 ----------------- 2 files changed, 32 deletions(-) delete mode 100644 home/sunders/update_cameras/config.php.example delete mode 100644 www/sunders/config.php.example diff --git a/home/sunders/update_cameras/config.php.example b/home/sunders/update_cameras/config.php.example deleted file mode 100644 index 641fc38..0000000 --- a/home/sunders/update_cameras/config.php.example +++ /dev/null @@ -1,15 +0,0 @@ - diff --git a/www/sunders/config.php.example b/www/sunders/config.php.example deleted file mode 100644 index 6df3a3f..0000000 --- a/www/sunders/config.php.example +++ /dev/null @@ -1,17 +0,0 @@ - From 285de74afa83e189ca1e1dfdf90304f99c5d1dd1 Mon Sep 17 00:00:00 2001 From: Vincent Mahnke Date: Sat, 11 Oct 2025 21:48:07 +0200 Subject: [PATCH 4/8] refactor: Replaces `PASSWD` with `PASSWORD` --- Containerfile.data_init | 2 +- home/sunders/update_cameras/update_camera.php | 2 +- .../update_cameras/update_statistics.php | 2 +- utils/config.php | 2 +- utils/create_db.php | 6 ++-- utils/get_sync_state.php | 2 +- utils/update_camera.php | 2 +- www/sunders/camera.php | 2 +- www/sunders/config.php | 28 ++++++++----------- www/sunders/stats/index.php | 2 +- 10 files changed, 23 insertions(+), 27 deletions(-) diff --git a/Containerfile.data_init b/Containerfile.data_init index 2274490..4604db4 100644 --- a/Containerfile.data_init +++ b/Containerfile.data_init @@ -5,7 +5,7 @@ COPY ./data_init/*.sql /var/sunders/init.sql CMD mariadb --host="$MYSQL_HOST" \ --user="$CAMERA_USER" \ - --password="$CAMERA_USER_PASSWD" \ + --password="$CAMERA_USER_PASSWORD" \ --skip-ssl \ ${MYSQL_DB:+--database="$MYSQL_DB"} \ < "/var/sunders/init.sql" diff --git a/home/sunders/update_cameras/update_camera.php b/home/sunders/update_cameras/update_camera.php index 21a2aaa..75bd3b4 100755 --- a/home/sunders/update_cameras/update_camera.php +++ b/home/sunders/update_cameras/update_camera.php @@ -15,7 +15,7 @@ $id = 0; $latitude = 0; $longitude = 0; -$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); +$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB); if($mysqli->connect_errno) { echo "Error while connecting to DB : $mysqli->error \n" ; diff --git a/home/sunders/update_cameras/update_statistics.php b/home/sunders/update_cameras/update_statistics.php index f6b9332..f6bcc7a 100644 --- a/home/sunders/update_cameras/update_statistics.php +++ b/home/sunders/update_cameras/update_statistics.php @@ -4,7 +4,7 @@ include "config.php"; if (USE_STATISTICS) { - $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); + $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB); if ($mysqli->connect_errno) { echo "Error while connecting to DB : ".$mysqli->error." \n" ; diff --git a/utils/config.php b/utils/config.php index cb479d5..62ff728 100644 --- a/utils/config.php +++ b/utils/config.php @@ -5,7 +5,7 @@ define('REPLICATE_URL', 'https://planet.openstreetmap.org/replication/minute'); define('MYSQL_HOST', getenv('MYSQL_HOST')); define('MYSQL_DB', getenv('MYSQL_DB')); define('MYSQL_USER', getenv('MYSQL_USER')); -define('MYSQL_PASSWD', getenv('MYSQL_PASSWD')); +define('MYSQL_PASSWORD', getenv('MYSQL_PASSWORD')); define('USE_STATISTICS', false); define('WEBSERVICE_COUNTRY_URL', 'http://api.geonames.org/countryCode'); diff --git a/utils/create_db.php b/utils/create_db.php index aa95e87..9643699 100644 --- a/utils/create_db.php +++ b/utils/create_db.php @@ -2,15 +2,15 @@ // Admin credentials (must have privileges to create DB/users) $dbHost = getenv('MYSQL_HOST'); $dbAdmin = getenv('MYSQL_USER'); -$dbPassword = getenv('MYSQL_PASSWD'); +$dbPassword = getenv('MYSQL_PASSWORD'); $dbName = getenv('MYSQL_DB'); // Variables for new users $cameraUser = getenv('CAMERA_USER'); -$cameraPassword = getenv('CAMERA_USER_PASSWD'); +$cameraPassword = getenv('CAMERA_USER_PASSWORD'); $camSelectUser = getenv('CAMERA_SELECT_USER'); -$camSelectPassword = getenv('CAMERA_SELECT_USER_PASSWD'); +$camSelectPassword = getenv('CAMERA_SELECT_USER_PASSWORD'); // Connect to MySQL $conn = new mysqli($dbHost, $dbAdmin, $dbPassword); diff --git a/utils/get_sync_state.php b/utils/get_sync_state.php index cdbe28a..a27b7db 100755 --- a/utils/get_sync_state.php +++ b/utils/get_sync_state.php @@ -2,7 +2,7 @@ include "config.php"; -$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); +$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB); if($mysqli->connect_errno) { echo "Error while connecting to DB : $mysqli->error \n" ; diff --git a/utils/update_camera.php b/utils/update_camera.php index bd8e7f6..dc88e48 100755 --- a/utils/update_camera.php +++ b/utils/update_camera.php @@ -15,7 +15,7 @@ $id = 0; $latitude = 0; $longitude = 0; -$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); +$mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB); if($mysqli->connect_errno) { echo "Error while connecting to DB : $mysqli->error \n" ; diff --git a/www/sunders/camera.php b/www/sunders/camera.php index 83ad163..97a77e7 100644 --- a/www/sunders/camera.php +++ b/www/sunders/camera.php @@ -266,7 +266,7 @@ $divDiag2 = ($divWidth * $divWidth) + ($divHeight * $divHeight); /* Connect to database */ - $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); + $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB); if($mysqli->connect_errno) { header('Content-type: application/json'); $result = '{"error":"error while connecting to db : ' . $mysqli->error . '"}'; diff --git a/www/sunders/config.php b/www/sunders/config.php index 4d934ec..c874eca 100644 --- a/www/sunders/config.php +++ b/www/sunders/config.php @@ -1,19 +1,15 @@ + define('USE_STATISTICS', false); +?> \ No newline at end of file diff --git a/www/sunders/stats/index.php b/www/sunders/stats/index.php index 029e42c..7a0dac4 100644 --- a/www/sunders/stats/index.php +++ b/www/sunders/stats/index.php @@ -138,7 +138,7 @@ $statsQueryObject = $levelObject->{$statsKey}; /* Connect to database */ - $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWD, MYSQL_DB); + $mysqli = new mysqli(MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB); if($mysqli->connect_errno) { echo 'Error while connecting to DB : $mysqli->error \n' ; exit(1); From b99799159e6f5f121b7d64d72fbda57f6fd82cfb Mon Sep 17 00:00:00 2001 From: Vincent Mahnke Date: Sat, 11 Oct 2025 21:48:38 +0200 Subject: [PATCH 5/8] feat: Adds compose file --- docker-compose.yml | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 docker-compose.yml diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..a415a74 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,34 @@ +version: '3.8' +services: + db: + image: mariadb:12.0.2 + environment: + MYSQL_ROOT_PASSWORD: rootpassword # ${{secrets.MYSQL_ROOT_PASSWORD}} + MYSQL_DATABASE: camera # ${{secrets.MYSQL_DATABASE}} + MYSQL_USER: camera # ${{secrets.MYSQL_USER}} + MYSQL_PASSWORD: camerapassword # ${{secrets.MYSQL_PASSWORD}} + ports: + - "3306:3306" + healthcheck: + test: ["CMD", "mysqladmin", "ping", "-h", "localhost"] + interval: 10s + timeout: 5s + retries: 5 + + web: + image: git.hamburg.ccc.de/ccchh/sunders/web:latest + environment: + MYSQL_HOST: db + MYSQL_DB: camera # ${{secrets.MYSQL_DATABASE}} + MYSQL_USER: camera # ${{secrets.MYSQL_USER}} + MYSQL_PASSWORD: camerapassword # ${{secrets.MYSQL_PASSWORD}} + DEFAULT_ZOOM: 12 + DEFAULT_LAT: 0 + DEFAULT_LON: 0 + DEFAULT_LANGUAGE: en + ports: + - "8080:80" + depends_on: + - db + volumes: + - ./www/sunders:/var/www/html \ No newline at end of file From 91384ff495bc311f54f9ecc62a7201b92a61eb56 Mon Sep 17 00:00:00 2001 From: ViMaSter Date: Sat, 11 Oct 2025 23:51:54 +0200 Subject: [PATCH 6/8] fix: CI uses `Containerfile` and directly tags package --- .forgejo/workflows/images.yml | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/.forgejo/workflows/images.yml b/.forgejo/workflows/images.yml index 68c9a89..67606e6 100644 --- a/.forgejo/workflows/images.yml +++ b/.forgejo/workflows/images.yml @@ -25,8 +25,7 @@ jobs: - name: Build and push web image to Container Registry run: | - docker build -t web:latest . - docker tag web:latest git.hamburg.ccc.de/ccchh/sunders/web:latest + docker build -f Containerfile -t git.hamburg.ccc.de/ccchh/sunders/web:latest . docker push git.hamburg.ccc.de/ccchh/sunders/web:latest working-directory: ./ From e411bf8b7636ff96ab181d62afcb91ea23a631ca Mon Sep 17 00:00:00 2001 From: ViMaSter Date: Sat, 11 Oct 2025 23:53:55 +0200 Subject: [PATCH 7/8] fix: Resolves issue with working directory --- .forgejo/workflows/images.yml | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/.forgejo/workflows/images.yml b/.forgejo/workflows/images.yml index 67606e6..4b29291 100644 --- a/.forgejo/workflows/images.yml +++ b/.forgejo/workflows/images.yml @@ -27,10 +27,8 @@ jobs: run: | docker build -f Containerfile -t git.hamburg.ccc.de/ccchh/sunders/web:latest . docker push git.hamburg.ccc.de/ccchh/sunders/web:latest - working-directory: ./ - name: Start Docker Compose services run: | docker compose up -d --wait - docker compose down - working-directory: / \ No newline at end of file + docker compose down \ No newline at end of file From c5f5edda4379e7240126cef829a992a9e97d0b9f Mon Sep 17 00:00:00 2001 From: Vincent Mahnke Date: Sun, 12 Oct 2025 00:01:28 +0200 Subject: [PATCH 8/8] fix: Changes healthcheck to MariaDB compatible syntax --- docker-compose.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker-compose.yml b/docker-compose.yml index a415a74..d1ccc8a 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -10,7 +10,7 @@ services: ports: - "3306:3306" healthcheck: - test: ["CMD", "mysqladmin", "ping", "-h", "localhost"] + test: ["CMD", "mariadb-admin", "ping", "-h", "localhost", "-uroot", "-prootpassword"] interval: 10s timeout: 5s retries: 5