Reindex Knowledge Base / reindex (push) Successful in 3s

Details

docs: add YAML frontmatter to all 151 markdown files

Adds title, description, type, domain, and tags frontmatter to every
doc for improved KB semantic search. The description field is prepended
to every search chunk, and domain/type/tags enable filtered queries.

Type values: context, guide, runbook, reference, troubleshooting
Domain values match directory structure (networking, docker, etc.)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-03-12 09:00:44 -05:00

3.7 KiB

Raw Permalink Blame History

title

description

type

domain

Backups - Technology Context

Overview

Automated backup system for nobara-desktop using restic with systemd timers, backing up to TrueNAS NAS over local network.

Restic Backup (nobara-desktop)

Configuration

Setting	Value
Tool	restic 0.18.0
Source	/home (~181 GiB)
Destination	/mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo
Schedule	Daily at 3:00 AM (systemd timer, +-30min randomized)
Encryption	None (trusted LAN)
Retention	7 daily, 4 weekly, 6 monthly, 1 yearly
Alerting	Discord webhook (Homelab Alerts channel) on failure

Key Files

File	Purpose
/home/cal/.local/bin/restic-backup.sh	Backup script
/home/cal/.config/restic/excludes.txt	Exclusion patterns
/etc/systemd/system/restic-backup.service	systemd service unit
/etc/systemd/system/restic-backup.timer	systemd timer unit

Features

Stale lock cleanup: Runs restic unlock --remove-all before each backup to prevent lock accumulation
Discord failure alerts: Red embed sent to Homelab Alerts on any failure (mount, repo access, backup, retention)
ERR trap: Catches unexpected script exits and alerts
Cache cleanup: --cleanup-cache flag prevents cache bloat
Low system impact: Nice=19, IOSchedulingClass=idle
Missed backup recovery: Persistent=true catches backups missed while system was off
Mount dependency: RequiresMountsFor ensures TrueNAS is mounted

Quick Commands

# Check timer status
systemctl list-timers restic-backup.timer

# View recent logs
journalctl -u restic-backup.service -e

# List snapshots
restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password snapshots

# Check repo size
restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password stats --mode raw-data

# Manual backup
sudo systemctl start restic-backup.service

# Remove stale locks (should not be needed - script handles this automatically)
restic -r /mnt/truenas/cals-files/Backups/nobara-desktop/restic-repo --insecure-no-password unlock

Troubleshooting

Backup service failing with exit code 11

Cause: Stale lock in the restic repository preventing forget --prune from running. Fix: restic -r <repo> --insecure-no-password unlock --remove-all Prevention: The backup script now clears stale locks automatically before each run (added 2026-02-19).

Snapshots accumulating beyond retention policy

Cause: If forget --prune fails (e.g., stale lock), snapshots pile up but deduplication keeps disk usage manageable. Detection: Check snapshot count with restic snapshots - should be ~18 max with current retention. Impact: Disk usage grows slowly due to dedup, but metadata overhead increases.

TrueNAS mount not available

Cause: NAS offline, network issue, or mount dropped. Detection: Script checks mountpoint -q /mnt/truenas/cals-files and alerts on failure. Fix: Check NAS status, remount with appropriate fstab entry.

Architecture Notes

Restic was chosen over borg (simpler CLI, good dedup) and rsync (no dedup)
systemd timers chosen over cron for Persistent=true, RequiresMountsFor, and Nice/IOScheduling support
No encryption because backups stay on trusted local TrueNAS - acceptable tradeoff for this environment
Deduplication ratio is excellent (~65:1 across snapshots of similar data)

3.7 KiB Raw Permalink Blame History