How to create all necessary entries for new (web service) VM

Let's assume that you want to add a new web service example.hamburg.ccc.de which is going to be hosted on the VM example on chaosknoten. These are the steps that you need to take to create the VM and add it to the Ansible repo.

IP, DNS, VM

Allocate a fresh IPv6 in Netbox in the 2a00:14b0:42:102::/64 net. This will be the management address for the VM.
On ns-intern:
1. Add an entry example.hosts.hamburg.ccc.de as an AAAA pointing to the allocated IP.
2. Add an entry example.hamburg.ccc.de as a CNAME for public-reverse-proxy to the same zone.
3. Commit and reload the zone.
On Chaosknoten:
1. Create a new VM, for example by cloning the Debian template 9023.
  Give it the name example.
2. Edit the ethernet interface to be connected to vmbr0, VLAN tag 2.
3. Configure the IPv6 address in the Cloud-Init section. Leave IPv4 set to DHCP.
4. Make sure the VM is started at boot (options).
5. Adjust any other VM parameters as needed.
6. Boot the VM.
Add the VM to Netbox.
- Make sure to enter the VM ID.
- Add an Ethernet interface to the VM; we typically use eth0 as a name.
- Add IP for that interface, then choose "Assign IP" and search for the IP you've created. Make it the primary IP of that interface.

Ansible Basics

As the first step, we need to make the host known to Ansible.

In .sops.yaml, add an entry for the host. Follow the other entries there.
1. keys.hosts.chaosknoten.age needs an age public key (must be generated; the private key gets added later in the host-specific YAML)
2. creation_rules needs an entry for the host, referencing the age key.
In inventories/chaosknoten/hosts.yaml:
1. Configure basic connection info:
```
example:
    ansible_host: example.hosts.hamburg.ccc.de
    ansible_user: chaos
    ansible_ssh_common_args: -J ssh://chaos@router.hamburg.ccc.de
```
  You typically will want to use router as a jump host so that you can run Ansible on an IPv4 only connection.
2. Add the host to the desired roles.
  1. As a minimum, you'll want the following roles:
    - base_config_hosts
    - infrastructure_authorized_keys_hosts
    - ansible_pull_hosts
  2. For a typical web service based on Docker Compose, you'll also want:
    - docker_compose_hosts
    - nginx_hosts
    - certbot_hosts.
In the directorry inventories/chaosknoten/host_var/:
1. A file inventories/chaosknoten/host_var/example.yaml with the host/service specific configuration.
2. A file inventories/chaosknoten/host_var/example.sops.yaml with the encrypted secrets for the host/service. Run sops inventories/chaosknoten/host_var/example.yaml to edit/create that file. Entries here should generally be prefixed with secret__ to make it easier to see where that variable is coming from in templates etc.
  - Add an entry ansible_pull__age_private_key with the age private key you generated above.

Service-specific config

From here, we go into the details of the web service that you want to configure. For a typical web service with Docker Compose, you will likely want to configure the following.

Make inventories/chaosknoten/host_var/example.yaml look like this:

certbot__version_spec: ""
certbot__acme_account_email_address: le-admin@hamburg.ccc.de
certbot__certificate_domains:
  - "example.hamburg.ccc.de"
certbot__new_cert_commands:
  - "systemctl reload nginx.service"

docker_compose__compose_file_content: "{{ lookup('ansible.builtin.template', 'resources/chaosknoten/example/docker_compose/compose.yaml.j2') }}"

nginx__version_spec: ""
nginx__configurations:
  - name: example.hamburg.ccc.de
    content: "{{ lookup('ansible.builtin.file', 'resources/chaosknoten/spaceapiccc/nginx/example.hamburg.ccc.de.conf') }}"

This will create compose.yaml from the template resources/chaosknoten/example/docker_compose/compose.yaml.j2', and the nginx config from resources/chaosknoten/spaceapiccc/nginx/example.hamburg.ccc.de.conf. Of course, depending on your service, you might need additional entries. See the other hosts and the roles for more info.

First Ansible run

Before you can run Ansible successfully, you will want to make sure you can connect to the VM, and that the host key has been added to your known hosts:

ssh chaos@example.hosts.hamburg.ccc.de
ssh -J chaos@router.hamburg.ccc.de chaos@example.hosts.hamburg.ccc.de

Then run Ansible for public-reverse-proxy to add the necessary entries:

ansible-playbook playbooks/deploy.yaml --inventory inventories/chaosknoten/hosts.yaml --limit public-reverse-proxy

Finally run Ansible for the new host:

ansible-playbook playbooks/deploy.yaml --inventory inventories/chaosknoten/hosts.yaml --limit example

Commit your changes

Do not forget to commit your changes, whether it's a new host or you are making changes to an existing host.

And always git pull before you run Ansible so avoid reverting anything!

Monitoring

Gatus (`status.hamburg.ccc.de`)

After you configured a new service or website, add it to our status and uptime monitoring.
Take a look at the configuration in resources/external/status/docker_compose/config and extend it to cover the newly added service or website. The configuration should probably happen in either services-chaosknoten.yaml or websites.yaml. Taking the existing configuration as a reference should give guidance on how to configure new checks. Additionally there's also the comprehensive Gatus Documentation.

After you've added some checks, the configuration can be deployed using:

ansible-playbook playbooks/deploy.yaml --inventory inventories/external --limit status

6 KiB Raw Blame History