This page documents the backup and recovery procedures for the production infrastructure.

Snapshot date: 2026-02
Environment: Docker Swarm (single-node)
Goal: Ensure data recoverability in case of failure

---

1. Backup Scope

The following components must be backed up:

Component	Type	Critical
PostgreSQL	Database	Yes
Forgejo	Repositories + config	Yes
Mattermost	Files + DB	Yes
MinIO	Object storage	Yes
Certbot	TLS certificates	Yes
Docker secrets	Configuration	Yes

---

2. PostgreSQL Backup

Full cluster backup

sudo docker exec -t data_postgres pg_dumpall -U postgres > postgres-backup.sql

Specific database backup

sudo docker exec -t data_postgres pg_dump -U postgres mattermost > mattermost.sql
sudo docker exec -t data_postgres pg_dump -U postgres forgejo > forgejo.sql

Store backup outside the Docker host.

---

3. Volume Backups

Locate volume path

sudo docker volume inspect <volume_name>

Typical host path:

/var/lib/docker/volumes/<volume_name>/_data

---

Forgejo

sudo tar -czf forgejo-backup.tar.gz \
/var/lib/docker/volumes/<forgejo_volume>/_data

---

Mattermost

sudo tar -czf mattermost-files-backup.tar.gz \
/var/lib/docker/volumes/<mattermost_volume>/_data

---

MinIO

sudo tar -czf minio-backup.tar.gz \
/var/lib/docker/volumes/<minio_volume>/_data

---

Certbot

sudo tar -czf certbot-backup.tar.gz \
/home/ubuntu/docker/swarm/infra/certbot

---

4. Docker Secrets Backup

Docker secrets cannot be extracted once created.

Therefore:

• Keep original secret files under:

  swarm/secrets/
  

• Backup this directory securely
• Store encrypted if possible

Example:

tar -czf secrets-backup.tar.gz swarm/secrets

---

5. Full Server Snapshot (Recommended)

If using a VPS provider:

• Use provider snapshot feature
• Perform snapshot after stopping heavy writes if possible
• Combine snapshot with database dump

Snapshots alone are not sufficient.

---

6. Automated Backup (Recommended)

Example simple cron backup:

sudo crontab -e

Example daily PostgreSQL dump:

0 2 * * * docker exec -t data_postgres pg_dumpall -U postgres > /home/ubuntu/backups/postgres-$(date +\%F).sql

Rotate old backups manually or via script.

---

7. Recovery Procedure

Step 1 — Reinstall System

• Install Ubuntu
• Install Docker
• Initialize Swarm
• Recreate networks
• Recreate secrets

---

Step 2 — Restore Volumes

Stop stacks:

sudo docker stack rm apps
sudo docker stack rm infra
sudo docker stack rm data

Extract volume backups into correct paths:

sudo tar -xzf forgejo-backup.tar.gz -C /var/lib/docker/volumes/<forgejo_volume>/_data

Repeat for each component.

---

Step 3 — Restore Database

Start PostgreSQL stack only:

sudo docker stack deploy -c data/data.yml data

Restore:

cat postgres-backup.sql | sudo docker exec -i data_postgres psql -U postgres

---

Step 4 — Redeploy Stacks

sudo docker stack deploy -c apps/apps.yml apps
sudo docker stack deploy -c infra/infra.yml infra

---

8. Disaster Recovery Rules

• Test restore procedure periodically.
• Store backups offsite.
• Encrypt backups at rest.
• Keep at least 7 days of history.
• Verify backups are readable.

Backups not tested are not backups.

---

9. Critical Notes

• Never rely only on VPS snapshots.
• Always backup database separately.
• Always backup volumes separately.
• Monitor available disk space.

---