From b75a09e86e5be70a9024db1168992962794b0739 Mon Sep 17 00:00:00 2001
From: Cal Corum <calcorum@users.noreply.github.com>
Date: Thu, 19 Feb 2026 07:24:33 -0600
Subject: [PATCH] Add backup docs and context loading for restic setup

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---
 CLAUDE.md          |  1 +
 backups/CONTEXT.md | 78 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 79 insertions(+)
 create mode 100644 backups/CONTEXT.md

diff --git a/CLAUDE.md b/CLAUDE.md
index be463e6..587f098 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -27,6 +27,7 @@ When a topic comes up, load `{tech}/CONTEXT.md` + `{tech}/troubleshooting.md`. F
 | steam, proton, steam tinker launch, gaming | `gaming/` |
 | server config, gitea config, infrastructure | `server-configs/` |
 | database, postgres, redis, sql | `databases/` |
+| backup, restic, snapshot, restore, retention | `backups/` |
 
 **Special loads:**
 - Pi-hole → also `networking/pihole-ha-setup.md`
diff --git a/backups/CONTEXT.md b/backups/CONTEXT.md
new file mode 100644
index 0000000..fdcc8a2
--- /dev/null
+++ b/backups/CONTEXT.md
@@ -0,0 +1,78 @@
+# Backups - Technology Context
+
+## Overview
+Automated backup system for nobara-desktop using restic with systemd timers, backing up to TrueNAS NAS over local network.
+
+## Restic Backup (nobara-desktop)
+
+### Configuration
+| Setting | Value |
+|---------|-------|
+| Tool | restic 0.18.0 |
+| Source | /home (~181 GiB) |
+| Destination | /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo |
+| Schedule | Daily at 3:00 AM (systemd timer, +-30min randomized) |
+| Encryption | None (trusted LAN) |
+| Retention | 7 daily, 4 weekly, 6 monthly, 1 yearly |
+| Alerting | Discord webhook (Homelab Alerts channel) on failure |
+
+### Key Files
+| File | Purpose |
+|------|---------|
+| /home/cal/.local/bin/restic-backup.sh | Backup script |
+| /home/cal/.config/restic/excludes.txt | Exclusion patterns |
+| /etc/systemd/system/restic-backup.service | systemd service unit |
+| /etc/systemd/system/restic-backup.timer | systemd timer unit |
+
+### Features
+- **Stale lock cleanup**: Runs `restic unlock --remove-all` before each backup to prevent lock accumulation
+- **Discord failure alerts**: Red embed sent to Homelab Alerts on any failure (mount, repo access, backup, retention)
+- **ERR trap**: Catches unexpected script exits and alerts
+- **Cache cleanup**: `--cleanup-cache` flag prevents cache bloat
+- **Low system impact**: Nice=19, IOSchedulingClass=idle
+- **Missed backup recovery**: Persistent=true catches backups missed while system was off
+- **Mount dependency**: RequiresMountsFor ensures TrueNAS is mounted
+
+### Quick Commands
+```bash
+# Check timer status
+systemctl list-timers restic-backup.timer
+
+# View recent logs
+journalctl -u restic-backup.service -e
+
+# List snapshots
+restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password snapshots
+
+# Check repo size
+restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password stats --mode raw-data
+
+# Manual backup
+sudo systemctl start restic-backup.service
+
+# Remove stale locks (should not be needed - script handles this automatically)
+restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password unlock
+```
+
+## Troubleshooting
+
+### Backup service failing with exit code 11
+**Cause**: Stale lock in the restic repository preventing `forget --prune` from running.
+**Fix**: `restic -r <repo> --insecure-no-password unlock --remove-all`
+**Prevention**: The backup script now clears stale locks automatically before each run (added 2026-02-19).
+
+### Snapshots accumulating beyond retention policy
+**Cause**: If `forget --prune` fails (e.g., stale lock), snapshots pile up but deduplication keeps disk usage manageable.
+**Detection**: Check snapshot count with `restic snapshots` - should be ~18 max with current retention.
+**Impact**: Disk usage grows slowly due to dedup, but metadata overhead increases.
+
+### TrueNAS mount not available
+**Cause**: NAS offline, network issue, or mount dropped.
+**Detection**: Script checks `mountpoint -q /mnt/truenas/cals-files` and alerts on failure.
+**Fix**: Check NAS status, remount with appropriate fstab entry.
+
+## Architecture Notes
+- Restic was chosen over borg (simpler CLI, good dedup) and rsync (no dedup)
+- systemd timers chosen over cron for Persistent=true, RequiresMountsFor, and Nice/IOScheduling support
+- No encryption because backups stay on trusted local TrueNAS - acceptable tradeoff for this environment
+- Deduplication ratio is excellent (~65:1 across snapshots of similar data)