--- title: "Backups Overview" description: "Restic backup system for nobara-desktop with systemd timers, backing up /home to TrueNAS NAS. Covers configuration, retention policy, Discord failure alerts, and troubleshooting stale locks." type: context domain: backups tags: [restic, backup, systemd, truenas, discord, retention, nobara-desktop] --- # Backups - Technology Context ## Overview Automated backup system for nobara-desktop using restic with systemd timers, backing up to TrueNAS NAS over local network. ## Restic Backup (nobara-desktop) ### Configuration | Setting | Value | |---------|-------| | Tool | restic 0.18.0 | | Source | /home (~181 GiB) | | Destination | /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo | | Schedule | Daily at 3:00 AM (systemd timer, +-30min randomized) | | Encryption | None (trusted LAN) | | Retention | 7 daily, 4 weekly, 6 monthly, 1 yearly | | Alerting | Discord webhook (Homelab Alerts channel) on failure | ### Key Files | File | Purpose | |------|---------| | /home/cal/.local/bin/restic-backup.sh | Backup script | | /home/cal/.config/restic/excludes.txt | Exclusion patterns | | /etc/systemd/system/restic-backup.service | systemd service unit | | /etc/systemd/system/restic-backup.timer | systemd timer unit | ### Features - **Stale lock cleanup**: Runs `restic unlock --remove-all` before each backup to prevent lock accumulation - **Discord failure alerts**: Red embed sent to Homelab Alerts on any failure (mount, repo access, backup, retention) - **ERR trap**: Catches unexpected script exits and alerts - **Cache cleanup**: `--cleanup-cache` flag prevents cache bloat - **Low system impact**: Nice=19, IOSchedulingClass=idle - **Missed backup recovery**: Persistent=true catches backups missed while system was off - **Mount dependency**: RequiresMountsFor ensures TrueNAS is mounted ### Quick Commands ```bash # Check timer status systemctl list-timers restic-backup.timer # View recent logs journalctl -u restic-backup.service -e # List snapshots restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password snapshots # Check repo size restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password stats --mode raw-data # Manual backup sudo systemctl start restic-backup.service # Remove stale locks (should not be needed - script handles this automatically) restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password unlock ``` ## Troubleshooting ### Backup service failing with exit code 11 **Cause**: Stale lock in the restic repository preventing `forget --prune` from running. **Fix**: `restic -r --insecure-no-password unlock --remove-all` **Prevention**: The backup script now clears stale locks automatically before each run (added 2026-02-19). ### Snapshots accumulating beyond retention policy **Cause**: If `forget --prune` fails (e.g., stale lock), snapshots pile up but deduplication keeps disk usage manageable. **Detection**: Check snapshot count with `restic snapshots` - should be ~18 max with current retention. **Impact**: Disk usage grows slowly due to dedup, but metadata overhead increases. ### TrueNAS mount not available **Cause**: NAS offline, network issue, or mount dropped. **Detection**: Script checks `mountpoint -q /mnt/truenas/cals-files` and alerts on failure. **Fix**: Check NAS status, remount with appropriate fstab entry. ## Architecture Notes - Restic was chosen over borg (simpler CLI, good dedup) and rsync (no dedup) - systemd timers chosen over cron for Persistent=true, RequiresMountsFor, and Nice/IOScheduling support - No encryption because backups stay on trusted local TrueNAS - acceptable tradeoff for this environment - Deduplication ratio is excellent (~65:1 across snapshots of similar data)