docs: move guides into sub-directory

Also start documenting the docs structure in home.

docs/guides/create-new-web-service-vm.md

@@ -0,0 +1,119 @@
+# How to create all necessary entries for new (web service) VM
+
+Let's assume that you want to add a new web service `example.hamburg.ccc.de` which is going to be hosted on the VM `example` on chaosknoten. These are the steps that you need to take to create the VM and add it to the Ansible repo.
+
+## IP, DNS, VM
+
+1. Allocate a fresh [IPv6 in Netbox in the 2a00:14b0:42:102::/64 net](https://netbox.hamburg.ccc.de/ipam/prefixes/47/ip-addresses/). This will be the management address for the VM. 
+2. On `ns-intern`:
+   1. Add an entry `example.hosts.hamburg.ccc.de` as an AAAA pointing to the allocated IP.
+      To do this, edit the zonefile in `/etc/bind/master/hamburg.ccc.de.zone`.
+   2. Add an entry `example.hamburg.ccc.de` as a CNAME for `public-reverse-proxy` to the same zone.
+   3. Commit and reload the zone by running `sudo make` while inside `/etc/bind/master`.
+3. On Chaosknoten:
+   1. Create a new VM, for example by cloning the Debian template 9023.  
+      Give it the name `example`.
+   2. Edit the ethernet interface to be connected to `vmbr0`, VLAN tag `2`.
+      Good $genders also enable the proxmox firewall.
+   3. Configure the IPv6 address in the Cloud-Init section. Leave IPv4 set to DHCP.
+   4. Make sure the VM is started at boot (options).
+   5. Adjust any other VM parameters as needed.
+   6. Boot the VM.
+4. Add the [VM to Netbox](https://netbox.hamburg.ccc.de/virtualization/virtual-machines/).
+    - Make sure to enter the VM ID.
+    - Add an Ethernet interface to the VM; we typically use `eth0` as a name.
+    - Add IP for that interface, then choose "Assign IP" and search for the IP you've created. Make it the primary IP of that interface.
+
+## Ansible Basics
+
+As the first step, we need to make the host known to Ansible.
+
+1. In `.sops.yaml`, add an entry for the host. Follow the other entries there.
+   1. `keys.hosts.chaosknoten.age` needs an age public key (must be generated; the private key gets added later in the host-specific YAML)
+   2. `creation_rules` needs an entry for the host, referencing the age key.
+   3. Re-encrypt existing files with the new key (manly `group_var/all.sops.yaml`): `find inventories -name "*.sops.*" | xargs sops updatekeys --yes`
+2. In `inventories/chaosknoten/hosts.yaml`:
+   1. Configure basic connection info:
+        ```yaml
+        example:
+            ansible_host: example.hosts.hamburg.ccc.de
+            ansible_user: chaos
+            ansible_ssh_common_args: -J ssh://chaos@router.hamburg.ccc.de
+        ```
+        You typically will want to use router as a jump host so that you can run Ansible on an IPv4 only connection.
+    2. Add the host to the desired roles.
+       1. As a minimum, you'll want the following roles:
+          - `base_config_hosts`
+          - `infrastructure_authorized_keys_hosts`
+          - `ansible_pull_hosts`
+       2. For a typical web service based on Docker Compose, you'll also want:
+          - `docker_compose_hosts`
+          - `nginx_hosts`
+          - `certbot_hosts`.
+ 3. In the directory `inventories/chaosknoten/host_var/`:
+    1. A file `inventories/chaosknoten/host_var/example.yaml` with the host/service specific configuration.
+    2. A file `inventories/chaosknoten/host_var/example.sops.yaml` with the encrypted secrets for the host/service. Run `sops inventories/chaosknoten/host_var/example.yaml` to edit/create that file. Entries here should generally be prefixed with `secret__` to make it easier to see where that variable is coming from in templates etc.
+       * Add an entry `ansible_pull__age_private_key` with the age private key you generated above.
+
+## Ansible-Pull Configuration
+
+Since you added your host to the `ansible_pull_hosts`, you also need to follow [Setting Up Secrets Using SOPS for a New Host](/docs/setting_up_secrets_using_sops_for_a_new_host.md) before continuing.
+
+## Service-specific config
+
+From here, we go into the details of the web service that you want to configure. For a typical web service with Docker Compose, you will likely want to configure the following.
+
+Make `inventories/chaosknoten/host_var/example.yaml` look like this:
+```yaml
+certbot__acme_account_email_address: le-admin@hamburg.ccc.de
+certbot__certificate_domains:
+  - "example.hamburg.ccc.de"
+certbot__new_cert_commands:
+  - "systemctl reload nginx.service"
+
+docker_compose__compose_file_content: "{{ lookup('ansible.builtin.template', 'resources/chaosknoten/example/docker_compose/compose.yaml.j2') }}"
+
+nginx__version_spec: ""
+nginx__configurations:
+  - name: example.hamburg.ccc.de
+    content: "{{ lookup('ansible.builtin.file', 'resources/chaosknoten/spaceapiccc/nginx/example.hamburg.ccc.de.conf') }}"
+```
+
+This will create `compose.yaml` from the template `resources/chaosknoten/example/docker_compose/compose.yaml.j2'`, and the nginx config from `resources/chaosknoten/spaceapiccc/nginx/example.hamburg.ccc.de.conf`. Of course, depending on your service, you might need additional entries. See the other hosts and the roles for more info.
+
+## First Ansible run
+
+Before you can run Ansible successfully, you will want to make sure you can connect to the VM, and that the host key has been added to your known hosts:
+* `ssh chaos@example.hosts.hamburg.ccc.de`
+* `ssh -J chaos@router.hamburg.ccc.de chaos@example.hosts.hamburg.ccc.de`
+
+Then run Ansible for `public-reverse-proxy` to add the necessary entries:
+
+```sh
+ansible-playbook playbooks/deploy.yaml --inventory inventories/chaosknoten/hosts.yaml --limit public-reverse-proxy
+```
+
+Finally run Ansible for the new host:
+
+```sh
+ansible-playbook playbooks/deploy.yaml --inventory inventories/chaosknoten/hosts.yaml --limit example
+```
+
+# Commit your changes
+
+Do not forget to commit your changes, whether it's a new host or you are making changes to an existing host.
+
+And always `git pull` before you run Ansible so avoid reverting anything!
+
+# Monitoring
+
+## Gatus (`status.hamburg.ccc.de`)
+
+After you configured a new service or website, add it to our status and uptime monitoring.  
+Take a look at the configuration in `resources/external/status/docker_compose/config` and extend it to cover the newly added service or website. The configuration should probably happen in either `services-chaosknoten.yaml` or `websites.yaml`. Taking the existing configuration as a reference should give guidance on how to configure new checks. Additionally there's also the comprehensive [Gatus Documentation](https://github.com/TwiN/gatus?tab=readme-ov-file#table-of-contents).
+
+After you've added some checks, the configuration can be deployed using:
+
+```sh
+ansible-playbook playbooks/deploy.yaml --inventory inventories/external --limit status 
+```

docs/guides/setting_up_secrets_using_sops_for_a_new_host.md

@@ -0,0 +1,33 @@
+# Setting Up Secrets Using SOPS for a New Host
+
+Because we're using the `community.sops.sops` vars plugin, the SOPS-encrypted secrets get stored in the inventory.
+
+1. Create a new age key for Ansible pull on the host.
+   ```
+   age-keygen
+   ```
+   Then add the public key part under `keys.hosts.chaosknoten.age` in [.sops.yaml](../.sops.yaml).
+2. Add a new creation rule for the hosts `host_vars` file in the sops config at `.sops.yaml`.
+   It should probably hold all admin keys plus the host entry you just added.
+   You can use existing creation rules as a reference.
+3. Re-encrypt existing files with the new key (manly `group_var/all.sops.yaml`): `find inventories -name "*.sops.*" | xargs sops updatekeys --yes`.
+
+   If GPG yells at you, follow the instructions in our [password-store](https://git.hamburg.ccc.de/CCCHH/password-store).
+4. Create a SOPS secrets file in the `host_vars` subdirectory of the relevant inventory.
+   The name of the file should be in the format `[HOSTNAME].sops.yaml` to get picked up by the vars plugin and to match the previously created creation rule.  
+   This can be accomplished with a command similar to this:
+   ```
+   sops inventories/[chaosknoten|z9]/host_vars/[HOSTNAME].sops.yaml
+   ```
+5. With the editor now open, add the secrets you want to store.
+   Because we're using the `community.sops.sops` vars plugin, the stored secrets will be exposed as Ansible variables.  
+   Also note that SOPS only encrypts the values, not the keys.  
+   When now creating entries, try to adhere to the following variable naming convention:
+   - Make sure to put the prive age key in here under `ansible_pull__age_private_key`.
+   - Prefix variable names with `secret__`, if they are intended to be used in a template file or similar. (e.g. `secret__netbox_secret_key: secret_value`)
+   - Otherwise, if the variable is directly consumed by a role or similar, directly set the variable. (e.g. `netbox__db_password: secret_value`)
+6. Now that the secrets are stored, they are exposed as variables and can simply be used like any other variable.
+
+## GPG Keys
+
+In order to edit encrypted files, you need all the GPG public keys imported into your GPG Keychain. You should be able to find the necessary public keys in https://git.hamburg.ccc.de/CCCHH/password-store.