june/hedgedoc-expire
0
0
Fork 0
forked from CCCHH/hedgedoc-expire
Code Pull requests Activity

Compare commits

base: june:patches
Branches Tags
june:main
june:patches
CCCHH:main
..
compare: CCCHH:main
Branches Tags
CCCHH:main
june:main
june:patches

No commits in common. "patches" and "main" have entirely different histories.
patches ... main

6 changed files with 142 additions and 300 deletions
Show stats Expand all files Collapse all files

4
Dockerfile
View file

@ -1,12 +1,12 @@
FROM docker.io/library/debian:12-slim AS builder FROM docker.io/library/debian:12-slim AS builder
RUN apt-get update && \ RUN apt-get update && \
apt-get install --no-install-suggests --no-install-recommends --yes pipx python-is-python3 apt-get install --no-install-suggests --no-install-recommends --yes pipx
ENV PATH="/root/.local/bin:${PATH}" ENV PATH="/root/.local/bin:${PATH}"
RUN pipx install poetry RUN pipx install poetry
RUN pipx inject poetry poetry-plugin-bundle RUN pipx inject poetry poetry-plugin-bundle
WORKDIR /src WORKDIR /src
COPY . . COPY . .
RUN poetry bundle venv --only=main /venv RUN poetry bundle venv --python=/usr/bin/python3 --only=main /venv
FROM gcr.io/distroless/python3-debian12 FROM gcr.io/distroless/python3-debian12
COPY --from=builder /venv /venv COPY --from=builder /venv /venv

107
README.md
View file

@ -1,69 +1,52 @@
# hedgedoc-expire - remove old notes # hedgedoc-expire - remove old notes
A Python app that can be run regularly against a Postgres Hedgedoc database to remove old notes. A Python app that can be run regularly against a Postgres Hedgedoc database to remove old notes. Notes that are expired
Notes that are expired can be emailed to the author as a backup. can be emailed to the author as a backup.
## Expiring old notes and revisions ## Expiring old notes and revisions
Hedgedoc keeps notes and revisions (versions) of those notes forever. Hedgedoc keeps notes and revisions (versions) of those notes forever. This might not be desirable, for example because
This might not be desirable, for example because of data protection reasons. of data protection reasons. With this utility, you can remove old revisions and old notes from the
With this utility, you can remove old revisions and old notes from the database. database. `hedgedoc-expire` works by talking directly to a Postgres database; no API access to Hedgedoc is required.
`hedgedoc-expire` works by talking directly to a Postgres database; no API access to Hedgedoc is required.
Currently, it only works with Postgres. Currently, it only works with Postgres.
### Expiring old revisions ### Expiring old revisions
Using the `-r` or `--revisions` argument. All revisions that have been created before the specified age will be removed. If all revisions are expired, the note
remains available, it just won't have any revisions to revert to. Once you continue editing it, new revisions will be
All revisions that have been created before the specified age will be removed. added.
If all revisions are expired, the note remains available, it just won't have any revisions to revert to.
Once you continue editing it, new revisions will be added.
### Expiring old notes ### Expiring old notes
Using the `-n` or `--notes` argument. Notes that are being expired will be emailed to the account that initially created the note. This allows that user to
restore the note, if necessary. Expiring a note will also remove all existing revisions for the note.
Notes that are being expired will be emailed to the account that initially created the note. You will need to configure your environment for `hedgedoc-expire` to be able to send mail. If the mail is not accepted
This allows that user to restore the note, if necessary. by the mail server, the note will not be removed. Note however that this utility has no idea if the mail server has
Expiring a note will also remove all existing revisions for the note. successfully delivered that mail to the intended recipient; if the mail gets lost somewhere on the way, the note cannot
be recovered.
You will need to configure your environment for `hedgedoc-expire` to be able to send mail.
If the mail is not accepted by the mail server, the note will not be removed.
Note however that this utility has no idea if the mail server has successfully delivered that mail to the intended recipient; if the mail gets lost somewhere on the way, the note cannot be recovered.
## Running `hedgedoc-expire.py` ## Running `hedgedoc-expire.py`
Locally from the command line expiring revisions and notes older than 7 days: Locally from the command line:
```shell ```shell
poetry run python ./hedgedoc-expire.py expire -n 7 -r 7 poetry run python ./hedgedoc-expire.py ...
``` ```
Locally from the command line only expiring revisions older than 7 days: From Docker Compose:
```shell
poetry run python ./hedgedoc-expire.py expire -r 7
```
Locally from the command line checking which notes older than 7 days would be expired (and only checking notes):
```shell
poetry run python ./hedgedoc-expire.py check -n 7
```
From Docker Compose checking which notes older than 95 and which revisions older than 14 days would be expired:
```yaml ```yaml
hedgedoc-expire: hedgedoc-expire:
image: hedgedoc-expire image: hedgedoc-expire
command: "check -r 14 -n 95" command: "-c -r 14 -n 95"
environment: environment:
- "POSTGRES_HOSTNAME=database" - "POSTGRES_HOSTNAME=database"
depends_on: depends_on:
- database - database
``` ```
Running a check against a local setup with one note, with times set to a fraction of a day: Running against a local setup with one note, with times set to a fraction of a day:
```shell ```shell
$ poetry run python ./hedgedoc-expire.py -n .001 -r .001 $ poetry run python ./hedgedoc-expire.py -n .001 -r .001
@ -75,26 +58,26 @@ Notes to be deleted not changed since 2024-05-20 09:02:46.416294+00:00 (a minute
foo@example.com (a day) http://localhost:3000/foo: hedgedoc-expire - remove old notes foo@example.com (a day) http://localhost:3000/foo: hedgedoc-expire - remove old notes
``` ```
## Commands, Arguments and Environment Variables ## Arguments and Environment Variables
`hedgedoc-expire` provides several different commands with check being the default: There are two main modes to run `hedgedoc-require`: check and expire. With `-c`, a report is generated on standard out.
Without it, the expiry action will be taken.
| Command | Description |
|------------|--------------------------------------------------------------------------------------------------------|
| check | Print a list of revisions and notes that would be expired, based on the given arguments `-n` and `-r`. |
| cron | Run `expire` at 2 am local time each day. Will run until killed. |
| emailcheck | Send an email from the configured sender to themselves with the the check report. |
| expire | Expire old revisions and notes, based on the given arguments `-n` and `-r`. |
Additionally the following arguments are available.
`-n` and `-r` are crucial in determining whether or not and how many notes/revisions should be targeted.
| Option | Default | Description | | Option | Default | Description |
|--------|---------|-------------------------------------------------------------------| |--------|---------|-------------------------------------------------------------------|
| -n | None | target all notes not changed in these many days | | -n | 90 | remove all notes not changed in these many days |
| -r | None | target all revisions created more than these many days ago | | -r | 7 | remove all revisions created more than these many days ago |
| -v | false | Print info on current action during `cron` and `expire` commandds | | -v | false | Print info on current action during `cron` and `expire` commandds |
Command is one of:
| Command | Description |
|------------|------------------------------------------------------------------------------------------------------------|
| check | Print a list of revisions and notes that would be expired, based on the given parameters for `-n` and `-r` |
| cron | Run `expire` at 2 am local time each day. Will run until killed. |
| emailcheck | Send an email from the configured sender to themselves with the the check report. |
| expire | Expire old revisions and notes. |
### Environment variables ### Environment variables
To configure the Postgres database connection and the SMTP parameters to send mail, set these variables. To configure the Postgres database connection and the SMTP parameters to send mail, set these variables.
@ -102,15 +85,19 @@ To configure the Postgres database connection and the SMTP parameters to send ma
For the SMTP connection, the code assumes a standard submission protocol setup with enable StartTLS and authentication, For the SMTP connection, the code assumes a standard submission protocol setup with enable StartTLS and authentication,
so you will need to configure a username and password. so you will need to configure a username and password.
| Variable | Default | Description | | Variable | Default | Description |
|---------------------|------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------| |-------------------|-----------------------|-------------------------------------|
| POSTGRES_CONNSTRING | postgresql://hedgedoc:geheim@localhost:5432/hedgedoc | [PostgreSQL connection string](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING) specifying where and how to connnect to the database | | POSTGRES_DATABASE | hedgedoc | database to connect to |
| SMTP_FROM | | sender address for the expiry mails | | POSTGRES_HOSTNAME | localhost | host of the database server |
| SMTP_HOSTNAME | localhost | mail server hostname | | POSTGRES_PASSWORD | geheim | password for the database |
| SMTP_PASSWORD | | SMTP password | | POSTGRES_PORT | 5432 | port number of the database server |
| SMTP_PORT | 587 | port to connect to | | POSTGRES_USERNAME | hedgedoc | username for the database |
| SMTP_USERNAME | | SMTP username | | SMTP_FROM | | sender address for the expiry mails |
| URL | http://localhost:3000 | base URL for linking to notes | | SMTP_HOSTNAME | localhost | mail server hostname |
| SMTP_PASSWORD | | SMTP password |
| SMTP_PORT | 587 | port to connect to |
| SMTP_USERNAME | | SMTP username |
| URL | http://localhost:3000 | base URL for linking to notes |
## Local Development Setup ## Local Development Setup
@ -123,4 +110,4 @@ You will need to create a user using the command line:
```sh ```sh
docker compose exec -it hedgedoc bin/manage_users --add foo@example.com --pass geheim docker compose exec -it hedgedoc bin/manage_users --add foo@example.com --pass geheim
``` ```

2
docker-compose.yaml
View file

@ -48,7 +48,7 @@ services:
image: hedgedoc-expire image: hedgedoc-expire
command: "-v -r .001 -n .001 check" command: "-v -r .001 -n .001 check"
environment: environment:
- "POSTGRES_CONNSTRING=postgres://hedgedoc:geheim@database:5432/hedgedoc" - "POSTGRES_HOSTNAME=database"
depends_on: depends_on:
- database - database

198
hedgedoc-expire.py
View file

@ -1,7 +1,5 @@
#!/bin/env python #!/bin/env python
import argparse import argparse
import base64
import binascii
import email import email
import json import json
import smtplib import smtplib
@ -16,8 +14,7 @@ from textwrap import dedent
from time import sleep from time import sleep
import humanize import humanize
import psycopg import pgsql
from psycopg.rows import dict_row
class Config: class Config:
@ -30,7 +27,11 @@ class Config:
self.revision_age = timedelta(days=14) self.revision_age = timedelta(days=14)
self.note_age = timedelta(days=95) self.note_age = timedelta(days=95)
self.postgres_connection_string = getenv('POSTGRES_CONNSTRING', 'postgresql://hedgedoc:geheim@localhost:5432/hedgedoc') self.postgres_hostname = getenv('POSTGRES_HOSTNAME', 'localhost')
self.postgres_username = getenv('POSTGRES_USERNAME', 'hedgedoc')
self.postgres_password = getenv('POSTGRES_PASSWORD', 'geheim')
self.postgres_database = getenv('POSTGRES_DATABASE', 'hedgedoc')
self.postgres_port = int(getenv('POSTGRES_PORT', '5432'))
self.smtp_hostname = getenv('SMTP_HOSTNAME', 'localhost') self.smtp_hostname = getenv('SMTP_HOSTNAME', 'localhost')
self.smtp_port = int(getenv('SMTP_PORT', '587')) self.smtp_port = int(getenv('SMTP_PORT', '587'))
@ -63,8 +64,7 @@ class EmailSender:
smtp_server = smtplib.SMTP(self.hostname, port=self.port) smtp_server = smtplib.SMTP(self.hostname, port=self.port)
context = ssl.create_default_context() context = ssl.create_default_context()
smtp_server.starttls(context=context) smtp_server.starttls(context=context)
if self.username != "" and self.password != "": smtp_server.login(self.username, self.password)
smtp_server.login(self.username, self.password)
smtp_server.send_message(message) smtp_server.send_message(message)
except Exception as e: except Exception as e:
print(f'Unable to send mail through {self}: {e}') print(f'Unable to send mail through {self}: {e}')
@ -94,14 +94,14 @@ class HedgedocExpire:
profile = json.loads(row['profile']) profile = json.loads(row['profile'])
return profile['emails'][0] return profile['emails'][0]
def notes_to_be_expired(self, conn) -> list[any]: def notes_to_be_expired(self, db) -> list[any]:
""" """
Get a list of all notes to be expired. Get a list of all notes to be expired.
:return: :return:
""" """
notes = []
cutoff = datetime.now(timezone.utc) - self.config.note_age cutoff = datetime.now(timezone.utc) - self.config.note_age
with conn.cursor(row_factory=dict_row) as cur: with db.prepare('''SELECT
cur.execute('''SELECT
"Notes"."alias", "Notes"."alias",
"Notes"."content", "Notes"."content",
"Notes"."createdAt", "Notes"."createdAt",
@ -113,21 +113,34 @@ class HedgedocExpire:
"Users"."email", "Users"."email",
"Users"."profile" "Users"."profile"
FROM "Notes", "Users" FROM "Notes", "Users"
WHERE "Notes"."updatedAt" < %s WHERE "Notes"."updatedAt" < $1
AND "Notes"."ownerId" = "Users"."id" AND "Notes"."ownerId" = "Users"."id"
ORDER BY "Notes"."updatedAt" ORDER BY "Notes"."updatedAt"
''', [cutoff]) ''') as notes_older_than:
return cur.fetchall() for row in notes_older_than(cutoff):
notes.append({
'alias': row.alias if row.alias is not None else row.shortid,
'content': row.content,
'createdAt': row.createdAt,
'email': row.email,
"id": row.id,
'ownerId': row.ownerId,
'profile': row.profile,
'shortid': row.shortid,
'title': row.title,
'updatedAt': row.updatedAt
})
return notes
def revisions_to_be_expired(self, conn) -> list[any]: def revisions_to_be_expired(self, db) -> list[any]:
""" """
Obtain a list of revisions to be expired. Obtain a list of revisions to be expired.
:param conn: the database connection :param db: the database connection
:return: :return:
""" """
revisions = []
cutoff = datetime.now(timezone.utc) - self.config.revision_age cutoff = datetime.now(timezone.utc) - self.config.revision_age
with conn.cursor(row_factory=dict_row) as cur: with db.prepare('''SELECT
cur.execute('''SELECT
"Notes"."alias", "Notes"."alias",
"Revisions"."createdAt", "Revisions"."createdAt",
"Users"."email", "Users"."email",
@ -137,29 +150,40 @@ class HedgedocExpire:
"Notes"."shortid" as "shortid", "Notes"."shortid" as "shortid",
"Notes"."title" "Notes"."title"
FROM "Revisions", "Notes", "Users" FROM "Revisions", "Notes", "Users"
WHERE "Revisions"."createdAt" < %s WHERE "Revisions"."createdAt" < $1
AND "Revisions"."noteId" = "Notes"."id" AND "Revisions"."noteId" = "Notes"."id"
AND "Notes"."ownerId" = "Users"."id" AND "Notes"."ownerId" = "Users"."id"
ORDER BY "Notes"."createdAt", "Revisions"."createdAt" ORDER BY "Notes"."createdAt", "Revisions"."createdAt"
''', [cutoff]) ''') as revs_older_than:
return cur.fetchall() for row in revs_older_than(cutoff):
revisions.append({
'alias': row.alias,
'createdAt': row.createdAt,
'email': row.email,
'noteId': row.noteId,
'profile': row.profile,
'revisionId': row.revisionId,
'shortid': row.shortid,
'title': row.title
})
return revisions
def check_notes_to_be_expired(self, conn) -> str: def check_notes_to_be_expired(self, db) -> str:
""" """
Return a list of notes that will be expired. Return a list of notes that will be expired.
:param conn: the database connection :param db: the database connection
:return: a multi-line text suitable for humans to read :return: a multi-line text suitable for humans to read
""" """
r = '' r = ''
cutoff = datetime.now(timezone.utc) - self.config.note_age cutoff = datetime.now(timezone.utc) - self.config.note_age
r += f'Notes to be deleted not changed since {cutoff} ({humanize.naturaldelta(self.config.note_age)}):\n' r += f'Notes to be deleted not changed since {cutoff} ({humanize.naturaldelta(self.config.note_age)}):\n'
for note in self.notes_to_be_expired(conn): for note in self.notes_to_be_expired(db):
age = datetime.now(timezone.utc) - note['updatedAt'] age = datetime.now(timezone.utc) - datetime.fromisoformat(note['updatedAt'])
url = self.config.url + '/' + (note["alias"] if note["alias"] is not None else note["shortid"]) url = self.config.url + '/' + (note["alias"] if note["alias"] is not None else note["shortid"])
r += f' {self.email_from_email_or_profile(note)} ({humanize.naturaldelta(age)}) {url}: {note["title"]}\n' r += f' {self.email_from_email_or_profile(note)} ({humanize.naturaldelta(age)}) {url}: {note["title"]}\n'
return r return r
def check_revisions_to_be_expired(self, conn) -> str: def check_revisions_to_be_expired(self, db) -> str:
""" """
Return a list of revisions that will be expired. Return a list of revisions that will be expired.
:return: a multi-line text suitable for humans to read :return: a multi-line text suitable for humans to read
@ -168,8 +192,8 @@ class HedgedocExpire:
cutoff = datetime.now(timezone.utc) - self.config.revision_age cutoff = datetime.now(timezone.utc) - self.config.revision_age
r += f'Revisions to be deleted created before {cutoff} ({humanize.naturaldelta(self.config.revision_age)}):\n' r += f'Revisions to be deleted created before {cutoff} ({humanize.naturaldelta(self.config.revision_age)}):\n'
notes = {} notes = {}
for row in self.revisions_to_be_expired(conn): for row in self.revisions_to_be_expired(db):
row['age'] = datetime.now(timezone.utc) - row['createdAt'] row['age'] = datetime.now(timezone.utc) - datetime.fromisoformat(row['createdAt'])
if row['noteId'] not in notes: if row['noteId'] not in notes:
notes[row['noteId']] = [] notes[row['noteId']] = []
notes[row['noteId']].append(row) notes[row['noteId']].append(row)
@ -182,20 +206,19 @@ class HedgedocExpire:
r += f' {humanize.naturaldelta(rev["age"])}: {rev["revisionId"]}\n' r += f' {humanize.naturaldelta(rev["age"])}: {rev["revisionId"]}\n'
return r return r
def expire_old_notes(self, conn) -> None: def expire_old_notes(self, db) -> None:
""" """
Email old notes to their owners, then delete them. Email old notes to their owners, then delete them.
:param conn: the database connection :param db: the database connection
:return: :return:
""" """
with conn.cursor() as cur: with db.prepare('DELETE FROM "Notes" WHERE "id" = $1') as delete_statement:
for note in self.notes_to_be_expired(conn): for note in self.notes_to_be_expired(db):
try: try:
note_age = datetime.now(timezone.utc) - note['updatedAt'] note_age = datetime.now(timezone.utc) - datetime.fromisoformat(note['updatedAt'])
msg = MIMEMultipart() msg = MIMEMultipart()
msg['From'] = self.email_sender.mail_from msg['From'] = self.email_sender.mail_from
msg['To'] = self.email_from_email_or_profile(note) msg['To'] = self.email_from_email_or_profile(note)
msg['Date'] = email.utils.formatdate()
msg['Subject'] = f'Your HedgeDoc Note "{note["title"]}" has expired' msg['Subject'] = f'Your HedgeDoc Note "{note["title"]}" has expired'
msg.attach(MIMEText(dedent(f'''\ msg.attach(MIMEText(dedent(f'''\
You created the note titled "{note["title"]}" on {note["createdAt"]}. You created the note titled "{note["title"]}" on {note["createdAt"]}.
@ -216,92 +239,39 @@ class HedgedocExpire:
self.email_sender.send(msg) self.email_sender.send(msg)
# email backup of the note sent, now we can delete it # email backup of the note sent, now we can delete it
cur.execute('DELETE FROM "Notes" WHERE "id" = %s', [note["id"]]) delete_statement(note["id"])
conn.commit()
if self.config.verbose: if self.config.verbose:
url = self.config.url + '/' + (note["alias"] if note["alias"] is not None else note["shortid"]) url = self.config.url + '/' + (note["alias"] if note["alias"] is not None else note["shortid"])
print(f'Note "{note["title"]}" ({url}) emailed to {msg["To"]}') print(f'Note "{note["title"]}" ({url}) emailed to {msg["To"]}')
try:
with conn.cursor(row_factory=dict_row) as user_history_cur:
# Calculate the urlid of the note from its id.
# See:
# - https://github.com/hedgedoc/hedgedoc/blob/380587b7fd65bc1eb71eef51a3aab324f9877650/lib/models/note.js#L167-L172
# - https://git.cccv.de/infra/ansible/roles/hedgedoc/-/blob/d69cef4bf6c7fe4e67570363659e4c20b0e102af/files/hedgedoc-util.py#L84
urlid = base64.urlsafe_b64encode(binascii.unhexlify(f"{note['id']}".replace('-', ''))).decode().replace('=', '')
# Get all users with note in history.
user_history_cur.execute('''SELECT
"Users"."id",
"Users"."history",
"Users"."email",
"Users"."profile"
FROM "Users"
WHERE jsonb_path_exists("Users"."history"::jsonb, '$[*] ? (@.id == $urlid)', jsonb_build_object('urlid', %s::text))
''', [urlid])
users_with_note = user_history_cur.fetchall()
for user in users_with_note:
history = json.loads(user["history"])
history_without_note = json.dumps([ entry for entry in history if entry["id"] != urlid ])
user_history_cur.execute('''UPDATE "Users"
SET "history" = %s
WHERE "id" = %s
''', [history_without_note, user["id"]])
conn.commit()
if self.config.verbose:
for user in users_with_note:
print(f' deleted history entry for {self.email_from_email_or_profile(user)}')
except Exception as e:
conn.rollback()
print(f'An error occured while trying to delete {note["id"]} from the users history: {e}', file=sys.stderr)
except Exception as e: except Exception as e:
print(f'Unable to send email to {self.email_from_email_or_profile(note)}: {e}', file=sys.stderr) print(f'Unable to send email to {self.email_from_email_or_profile(note)}: {e}', file=sys.stderr)
def expire_old_revisions(self, conn) -> None: def expire_old_revisions(self, db) -> None:
""" """
Removes all revision on all notes that have been modified earlier than age. Removes all revision on all notes that have been modified earlier than age.
:param conn: the database connection :param db: the database connection
:return: :return:
""" """
cutoff = datetime.now(timezone.utc) - self.config.revision_age cutoff = datetime.now(timezone.utc) - self.config.revision_age
with conn.cursor() as cur: with db.prepare('DELETE FROM "Revisions" WHERE "createdAt" < $1 RETURNING id') as delete:
rows = list(cur.execute('DELETE FROM "Revisions" WHERE "createdAt" < %s RETURNING id', [cutoff])) rows = list(delete(cutoff))
if self.config.verbose: if self.config.verbose:
print(f'Deleted {len(rows)} old revisions') print(f'Deleted {len(rows)} old revisions')
conn.commit()
def cmd_check(self) -> None: def cmd_check(self) -> None:
with psycopg.connect(self.config.postgres_connection_string) as conn: with pgsql.Connection((self.config.postgres_hostname, self.config.postgres_port),
if self.config.revision_age is not None: self.config.postgres_username, self.config.postgres_password) as db:
print(self.check_revisions_to_be_expired(conn)) print(self.check_revisions_to_be_expired(db) +
elif self.config.verbose: self.check_notes_to_be_expired(db))
print("Revisions weren't included in the check, not checking.\n")
if self.config.note_age is not None:
print(self.check_notes_to_be_expired(conn))
elif self.config.verbose:
print("Notes weren't included in the check, not checking.\n")
def cmd_emailcheck(self) -> None: def cmd_emailcheck(self) -> None:
with psycopg.connect(self.config.postgres_connection_string) as conn: with pgsql.Connection((self.config.postgres_hostname, self.config.postgres_port),
report = '' self.config.postgres_username, self.config.postgres_password) as db:
report = self.check_revisions_to_be_expired(db) + self.check_notes_to_be_expired(db)
if self.config.revision_age is not None:
report += self.check_revisions_to_be_expired(conn)
else:
report += "Revisions weren't included in the check.\n"
if self.config.note_age is not None:
report += self.check_notes_to_be_expired(conn)
else:
report += "Notes weren't included in the check.\n"
msg = MIMEMultipart() msg = MIMEMultipart()
msg['From'] = self.email_sender.mail_from msg['From'] = self.email_sender.mail_from
msg['To'] = self.email_sender.mail_from msg['To'] = self.email_sender.mail_from
msg['Date'] = email.utils.formatdate()
msg['Subject'] = f'Hedgedoc Expire: Report' msg['Subject'] = f'Hedgedoc Expire: Report'
msg.attach(MIMEText(dedent(f'''\ msg.attach(MIMEText(dedent(f'''\
This report shows which notes and revisions would be deleted if expire would be run now. This report shows which notes and revisions would be deleted if expire would be run now.
@ -313,16 +283,10 @@ class HedgedocExpire:
self.email_sender.send(msg) self.email_sender.send(msg)
def cmd_expire(self) -> None: def cmd_expire(self) -> None:
with psycopg.connect(self.config.postgres_connection_string) as conn: with pgsql.Connection((self.config.postgres_hostname, self.config.postgres_port),
if self.config.revision_age is not None: self.config.postgres_username, self.config.postgres_password) as db:
self.expire_old_revisions(conn) self.expire_old_revisions(db)
elif self.config.verbose: self.expire_old_notes(db)
print("Revisions weren't included in the expire action, not expiring.\n")
if self.config.note_age is not None:
self.expire_old_notes(conn)
elif self.config.verbose:
print("Notes weren't included in the expire action, not expiring.\n")
def main(): def main():
@ -336,18 +300,18 @@ def main():
Revisions of notes that have been created before the specified time will be deleted. Revisions of notes that have been created before the specified time will be deleted.
'''), '''),
epilog=dedent('''\ epilog=dedent('''\
command is one of, which check being the default: command is one of:
- check: Print a list of revisions and notes that would be expired, based on the given arguments -n and -r. - check: print a list of revisions and notes to be expired
- cron: Run `expire` at 2 am local time each day. Will run until killed. - cron: run expire every 24 hours
- emailcheck: Send an email from the configured sender to themselves with the the check report. - emailcheck: send am email from the configured sender to themselves with the the check report
- expire: Expire old revisions and notes, based on the given arguments -n and -r. - expire: expire old revisions and untouched notes
See https://git.hamburg.ccc.de/CCCHH/hedgedoc-expire See https://git.hamburg.ccc.de/CCCHH/hedgedoc-expire
''') ''')
) )
parser.add_argument('-n', '--notes', metavar='DAYS', type=float, parser.add_argument('-n', '--notes', metavar='DAYS', type=float, default=95,
help='remove all notes not changed in these many days') help='remove all notes not changed in these many days')
parser.add_argument('-r', '--revisions', metavar='DAYS', type=float, parser.add_argument('-r', '--revisions', metavar='DAYS', type=float, default=14,
help='remove all revisions created more than these many days ago') help='remove all revisions created more than these many days ago')
parser.add_argument('command', choices=['check', 'cron', 'emailcheck', 'expire'], default='check', nargs='?', parser.add_argument('command', choices=['check', 'cron', 'emailcheck', 'expire'], default='check', nargs='?',
help='action to perform') help='action to perform')
@ -356,8 +320,8 @@ def main():
args = parser.parse_args() args = parser.parse_args()
config = Config() config = Config()
config.note_age = timedelta(days=args.notes) if args.notes is not None else None config.note_age = timedelta(days=args.notes)
config.revision_age = timedelta(days=args.revisions) if args.revisions is not None else None config.revision_age = timedelta(days=args.revisions)
config.verbose = args.verbose config.verbose = args.verbose
mail = EmailSender(config.smtp_hostname, config.smtp_port, config.smtp_username, config.smtp_password, mail = EmailSender(config.smtp_hostname, config.smtp_port, config.smtp_username, config.smtp_password,
config.smtp_from) config.smtp_from)

125
poetry.lock generated
View file

@ -1,4 +1,4 @@
# This file is automatically @generated by Poetry 1.8.4 and should not be changed by hand. # This file is automatically @generated by Poetry 1.8.2 and should not be changed by hand.
[[package]] [[package]]
name = "humanize" name = "humanize"
@ -15,125 +15,16 @@ files = [
tests = ["freezegun", "pytest", "pytest-cov"] tests = ["freezegun", "pytest", "pytest-cov"]
[[package]] [[package]]
name = "psycopg" name = "pgsql"
version = "3.2.3" version = "2.2"
description = "PostgreSQL database adapter for Python" description = "PostgreSQL client library for Python 3"
optional = false optional = false
python-versions = ">=3.8" python-versions = ">=3.11"
files = [ files = [
{file = "psycopg-3.2.3-py3-none-any.whl", hash = "sha256:644d3973fe26908c73d4be746074f6e5224b03c1101d302d9a53bf565ad64907"}, {file = "pgsql-2.2-py3-none-any.whl", hash = "sha256:12d3360645089d2ee6ab04dd8b9a03aeb7092e3459323f9553252bf4b8bdd2ed"},
{file = "psycopg-3.2.3.tar.gz", hash = "sha256:a5764f67c27bec8bfac85764d23c534af2c27b893550377e37ce59c12aac47a2"},
]
[package.dependencies]
psycopg-binary = {version = "3.2.3", optional = true, markers = "implementation_name != \"pypy\" and extra == \"binary\""}
typing-extensions = {version = ">=4.6", markers = "python_version < \"3.13\""}
tzdata = {version = "*", markers = "sys_platform == \"win32\""}
[package.extras]
binary = ["psycopg-binary (==3.2.3)"]
c = ["psycopg-c (==3.2.3)"]
dev = ["ast-comments (>=1.1.2)", "black (>=24.1.0)", "codespell (>=2.2)", "dnspython (>=2.1)", "flake8 (>=4.0)", "mypy (>=1.11)", "types-setuptools (>=57.4)", "wheel (>=0.37)"]
docs = ["Sphinx (>=5.0)", "furo (==2022.6.21)", "sphinx-autobuild (>=2021.3.14)", "sphinx-autodoc-typehints (>=1.12)"]
pool = ["psycopg-pool"]
test = ["anyio (>=4.0)", "mypy (>=1.11)", "pproxy (>=2.7)", "pytest (>=6.2.5)", "pytest-cov (>=3.0)", "pytest-randomly (>=3.5)"]
[[package]]
name = "psycopg-binary"
version = "3.2.3"
description = "PostgreSQL database adapter for Python -- C optimisation distribution"
optional = false
python-versions = ">=3.8"
files = [
{file = "psycopg_binary-3.2.3-cp310-cp310-macosx_12_0_x86_64.whl", hash = "sha256:965455eac8547f32b3181d5ec9ad8b9be500c10fe06193543efaaebe3e4ce70c"},
{file = "psycopg_binary-3.2.3-cp310-cp310-macosx_14_0_arm64.whl", hash = "sha256:71adcc8bc80a65b776510bc39992edf942ace35b153ed7a9c6c573a6849ce308"},
{file = "psycopg_binary-3.2.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f73adc05452fb85e7a12ed3f69c81540a8875960739082e6ea5e28c373a30774"},
{file = "psycopg_binary-3.2.3-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e8630943143c6d6ca9aefc88bbe5e76c90553f4e1a3b2dc339e67dc34aa86f7e"},
{file = "psycopg_binary-3.2.3-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:3bffb61e198a91f712cc3d7f2d176a697cb05b284b2ad150fb8edb308eba9002"},
{file = "psycopg_binary-3.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:dc4fa2240c9fceddaa815a58f29212826fafe43ce80ff666d38c4a03fb036955"},
{file = "psycopg_binary-3.2.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:192a5f8496e6e1243fdd9ac20e117e667c0712f148c5f9343483b84435854c78"},
{file = "psycopg_binary-3.2.3-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:64dc6e9ec64f592f19dc01a784e87267a64a743d34f68488924251253da3c818"},
{file = "psycopg_binary-3.2.3-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:79498df398970abcee3d326edd1d4655de7d77aa9aecd578154f8af35ce7bbd2"},
{file = "psycopg_binary-3.2.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:949551752930d5e478817e0b49956350d866b26578ced0042a61967e3fcccdea"},
{file = "psycopg_binary-3.2.3-cp310-cp310-win_amd64.whl", hash = "sha256:80a2337e2dfb26950894c8301358961430a0304f7bfe729d34cc036474e9c9b1"},
{file = "psycopg_binary-3.2.3-cp311-cp311-macosx_12_0_x86_64.whl", hash = "sha256:6d8f2144e0d5808c2e2aed40fbebe13869cd00c2ae745aca4b3b16a435edb056"},
{file = "psycopg_binary-3.2.3-cp311-cp311-macosx_14_0_arm64.whl", hash = "sha256:94253be2b57ef2fea7ffe08996067aabf56a1eb9648342c9e3bad9e10c46e045"},
{file = "psycopg_binary-3.2.3-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fda0162b0dbfa5eaed6cdc708179fa27e148cb8490c7d62e5cf30713909658ea"},
{file = "psycopg_binary-3.2.3-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:2c0419cdad8c70eaeb3116bb28e7b42d546f91baf5179d7556f230d40942dc78"},
{file = "psycopg_binary-3.2.3-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:74fbf5dd3ef09beafd3557631e282f00f8af4e7a78fbfce8ab06d9cd5a789aae"},
{file = "psycopg_binary-3.2.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7d784f614e4d53050cbe8abf2ae9d1aaacf8ed31ce57b42ce3bf2a48a66c3a5c"},
{file = "psycopg_binary-3.2.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:4e76ce2475ed4885fe13b8254058be710ec0de74ebd8ef8224cf44a9a3358e5f"},
{file = "psycopg_binary-3.2.3-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:5938b257b04c851c2d1e6cb2f8c18318f06017f35be9a5fe761ee1e2e344dfb7"},
{file = "psycopg_binary-3.2.3-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:257c4aea6f70a9aef39b2a77d0658a41bf05c243e2bf41895eb02220ac6306f3"},
{file = "psycopg_binary-3.2.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:06b5cc915e57621eebf2393f4173793ed7e3387295f07fed93ed3fb6a6ccf585"},
{file = "psycopg_binary-3.2.3-cp311-cp311-win_amd64.whl", hash = "sha256:09baa041856b35598d335b1a74e19a49da8500acedf78164600694c0ba8ce21b"},
{file = "psycopg_binary-3.2.3-cp312-cp312-macosx_12_0_x86_64.whl", hash = "sha256:48f8ca6ee8939bab760225b2ab82934d54330eec10afe4394a92d3f2a0c37dd6"},
{file = "psycopg_binary-3.2.3-cp312-cp312-macosx_14_0_arm64.whl", hash = "sha256:5361ea13c241d4f0ec3f95e0bf976c15e2e451e9cc7ef2e5ccfc9d170b197a40"},
{file = "psycopg_binary-3.2.3-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cb987f14af7da7c24f803111dbc7392f5070fd350146af3345103f76ea82e339"},
{file = "psycopg_binary-3.2.3-cp312-cp312-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0463a11b1cace5a6aeffaf167920707b912b8986a9c7920341c75e3686277920"},
{file = "psycopg_binary-3.2.3-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:8b7be9a6c06518967b641fb15032b1ed682fd3b0443f64078899c61034a0bca6"},
{file = "psycopg_binary-3.2.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:64a607e630d9f4b2797f641884e52b9f8e239d35943f51bef817a384ec1678fe"},
{file = "psycopg_binary-3.2.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:fa33ead69ed133210d96af0c63448b1385df48b9c0247eda735c5896b9e6dbbf"},
{file = "psycopg_binary-3.2.3-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:1f8b0d0e99d8e19923e6e07379fa00570be5182c201a8c0b5aaa9a4d4a4ea20b"},
{file = "psycopg_binary-3.2.3-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:709447bd7203b0b2debab1acec23123eb80b386f6c29e7604a5d4326a11e5bd6"},
{file = "psycopg_binary-3.2.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5e37d5027e297a627da3551a1e962316d0f88ee4ada74c768f6c9234e26346d9"},
{file = "psycopg_binary-3.2.3-cp312-cp312-win_amd64.whl", hash = "sha256:261f0031ee6074765096a19b27ed0f75498a8338c3dcd7f4f0d831e38adf12d1"},
{file = "psycopg_binary-3.2.3-cp313-cp313-macosx_12_0_x86_64.whl", hash = "sha256:41fdec0182efac66b27478ac15ef54c9ebcecf0e26ed467eb7d6f262a913318b"},
{file = "psycopg_binary-3.2.3-cp313-cp313-macosx_14_0_arm64.whl", hash = "sha256:07d019a786eb020c0f984691aa1b994cb79430061065a694cf6f94056c603d26"},
{file = "psycopg_binary-3.2.3-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4c57615791a337378fe5381143259a6c432cdcbb1d3e6428bfb7ce59fff3fb5c"},
{file = "psycopg_binary-3.2.3-cp313-cp313-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e8eb9a4e394926b93ad919cad1b0a918e9b4c846609e8c1cfb6b743683f64da0"},
{file = "psycopg_binary-3.2.3-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:5905729668ef1418bd36fbe876322dcb0f90b46811bba96d505af89e6fbdce2f"},
{file = "psycopg_binary-3.2.3-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd65774ed7d65101b314808b6893e1a75b7664f680c3ef18d2e5c84d570fa393"},
{file = "psycopg_binary-3.2.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:700679c02f9348a0d0a2adcd33a0275717cd0d0aee9d4482b47d935023629505"},
{file = "psycopg_binary-3.2.3-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:96334bb64d054e36fed346c50c4190bad9d7c586376204f50bede21a913bf942"},
{file = "psycopg_binary-3.2.3-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:9099e443d4cc24ac6872e6a05f93205ba1a231b1a8917317b07c9ef2b955f1f4"},
{file = "psycopg_binary-3.2.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:1985ab05e9abebfbdf3163a16ebb37fbc5d49aff2bf5b3d7375ff0920bbb54cd"},
{file = "psycopg_binary-3.2.3-cp313-cp313-win_amd64.whl", hash = "sha256:e90352d7b610b4693fad0feea48549d4315d10f1eba5605421c92bb834e90170"},
{file = "psycopg_binary-3.2.3-cp38-cp38-macosx_12_0_x86_64.whl", hash = "sha256:69320f05de8cdf4077ecd7fefdec223890eea232af0d58f2530cbda2871244a0"},
{file = "psycopg_binary-3.2.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4926ea5c46da30bec4a85907aa3f7e4ea6313145b2aa9469fdb861798daf1502"},
{file = "psycopg_binary-3.2.3-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c64c4cd0d50d5b2288ab1bcb26c7126c772bbdebdfadcd77225a77df01c4a57e"},
{file = "psycopg_binary-3.2.3-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:05a1bdce30356e70a05428928717765f4a9229999421013f41338d9680d03a63"},
{file = "psycopg_binary-3.2.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7ad357e426b0ea5c3043b8ec905546fa44b734bf11d33b3da3959f6e4447d350"},
{file = "psycopg_binary-3.2.3-cp38-cp38-musllinux_1_2_aarch64.whl", hash = "sha256:967b47a0fd237aa17c2748fdb7425015c394a6fb57cdad1562e46a6eb070f96d"},
{file = "psycopg_binary-3.2.3-cp38-cp38-musllinux_1_2_i686.whl", hash = "sha256:71db8896b942770ed7ab4efa59b22eee5203be2dfdee3c5258d60e57605d688c"},
{file = "psycopg_binary-3.2.3-cp38-cp38-musllinux_1_2_ppc64le.whl", hash = "sha256:2773f850a778575dd7158a6dd072f7925b67f3ba305e2003538e8831fec77a1d"},
{file = "psycopg_binary-3.2.3-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:aeddf7b3b3f6e24ccf7d0edfe2d94094ea76b40e831c16eff5230e040ce3b76b"},
{file = "psycopg_binary-3.2.3-cp38-cp38-win_amd64.whl", hash = "sha256:824c867a38521d61d62b60aca7db7ca013a2b479e428a0db47d25d8ca5067410"},
{file = "psycopg_binary-3.2.3-cp39-cp39-macosx_12_0_x86_64.whl", hash = "sha256:9994f7db390c17fc2bd4c09dca722fd792ff8a49bb3bdace0c50a83f22f1767d"},
{file = "psycopg_binary-3.2.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1303bf8347d6be7ad26d1362af2c38b3a90b8293e8d56244296488ee8591058e"},
{file = "psycopg_binary-3.2.3-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:842da42a63ecb32612bb7f5b9e9f8617eab9bc23bd58679a441f4150fcc51c96"},
{file = "psycopg_binary-3.2.3-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:2bb342a01c76f38a12432848e6013c57eb630103e7556cf79b705b53814c3949"},
{file = "psycopg_binary-3.2.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:fd40af959173ea0d087b6b232b855cfeaa6738f47cb2a0fd10a7f4fa8b74293f"},
{file = "psycopg_binary-3.2.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:9b60b465773a52c7d4705b0a751f7f1cdccf81dd12aee3b921b31a6e76b07b0e"},
{file = "psycopg_binary-3.2.3-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:fc6d87a1c44df8d493ef44988a3ded751e284e02cdf785f746c2d357e99782a6"},
{file = "psycopg_binary-3.2.3-cp39-cp39-musllinux_1_2_ppc64le.whl", hash = "sha256:f0b018e37608c3bfc6039a1dc4eb461e89334465a19916be0153c757a78ea426"},
{file = "psycopg_binary-3.2.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:2a29f5294b0b6360bfda69653697eff70aaf2908f58d1073b0acd6f6ab5b5a4f"},
{file = "psycopg_binary-3.2.3-cp39-cp39-win_amd64.whl", hash = "sha256:e56b1fd529e5dde2d1452a7d72907b37ed1b4f07fdced5d8fb1e963acfff6749"},
]
[[package]]
name = "typing-extensions"
version = "4.12.2"
description = "Backported and Experimental Type Hints for Python 3.8+"
optional = false
python-versions = ">=3.8"
files = [
{file = "typing_extensions-4.12.2-py3-none-any.whl", hash = "sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d"},
{file = "typing_extensions-4.12.2.tar.gz", hash = "sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8"},
]
[[package]]
name = "tzdata"
version = "2024.2"
description = "Provider of IANA time zone data"
optional = false
python-versions = ">=2"
files = [
{file = "tzdata-2024.2-py2.py3-none-any.whl", hash = "sha256:a48093786cdcde33cad18c2555e8532f34422074448fbc874186f0abd79565cd"},
{file = "tzdata-2024.2.tar.gz", hash = "sha256:7d85cc416e9382e69095b7bdf4afd9e3880418a2413feec7069d533d6b4e31cc"},
] ]
[metadata] [metadata]
lock-version = "2.0" lock-version = "2.0"
python-versions = "^3.11" python-versions = "^3.12"
content-hash = "6ef91abe2e2acfa0b7355dca99bfd6595e18bf90609c323f1857f13a1df73aaf" content-hash = "0b022231d09549e564a7cec69a9169cbd5a27e550b5a10a8b57ef2bbe251b6fe"

6
pyproject.toml
View file

@ -1,14 +1,14 @@
[tool.poetry] [tool.poetry]
name = "hedgedoc-expire" name = "hedgedoc-expire"
version = "0.2.0" version = "0.1.0"
description = "Remove old Hedgedoc notes" description = "Remove old Hedgedoc notes"
authors = ["Stefan Bethke <stb@lassitu.de>"] authors = ["Stefan Bethke <stb@lassitu.de>"]
readme = "README.md" readme = "README.md"
[tool.poetry.dependencies] [tool.poetry.dependencies]
python = "^3.11" python = "^3.12"
pgsql = "^2.2"
humanize = "^4.9.0" humanize = "^4.9.0"
psycopg = {extras = ["binary"], version = "^3.2.3"}
[build-system] [build-system]