All checks were successful
Reindex Knowledge Base / reindex (push) Successful in 3s
Adds title, description, type, domain, and tags frontmatter to every doc for improved KB semantic search. The description field is prepended to every search chunk, and domain/type/tags enable filtered queries. Type values: context, guide, runbook, reference, troubleshooting Domain values match directory structure (networking, docker, etc.) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
266 lines
6.9 KiB
Markdown
266 lines
6.9 KiB
Markdown
---
|
|
title: "NPM to Caddy Migration Runbook"
|
|
description: "Step-by-step operational runbook for migrating from Nginx Proxy Manager to Caddy on 10.10.0.16. Four phases: prepare (test on alternate ports), cutover (<2 min downtime), validate (24-48h monitoring), and cleanup. Includes rollback plan."
|
|
type: runbook
|
|
domain: server-configs
|
|
tags: [caddy, nginx-proxy-manager, migration, reverse-proxy, dns, pihole, cloudflare]
|
|
---
|
|
|
|
# NPM to Caddy Migration Plan
|
|
|
|
Step-by-step guide to migrate from Nginx Proxy Manager to Caddy on `10.10.0.16`.
|
|
|
|
## Prerequisites
|
|
|
|
- [ ] Cloudflare API token with `Zone:DNS:Edit` for manticorum.com
|
|
- [ ] SSH access to `pihole` (10.10.0.16) and `ubuntu-manticore` (10.10.0.226)
|
|
- [ ] Docker and docker compose installed on 10.10.0.16
|
|
- [ ] Familiarity with current NPM proxy hosts (see README.md)
|
|
|
|
## Phase 1: Prepare (no downtime, no changes to production)
|
|
|
|
### 1.1 Create Cloudflare API Token
|
|
|
|
1. Go to https://dash.cloudflare.com/profile/api-tokens
|
|
2. Create token with permissions:
|
|
- **Zone - DNS - Edit** (scoped to manticorum.com)
|
|
3. Save the token securely
|
|
|
|
### 1.2 Deploy Caddy Config to Host
|
|
|
|
```bash
|
|
# From workstation
|
|
scp -r server-configs/caddy-migration/ pihole:/home/cal/caddy/
|
|
ssh pihole "cp /home/cal/caddy/.env.example /home/cal/caddy/.env"
|
|
```
|
|
|
|
Edit `.env` on the host:
|
|
```bash
|
|
ssh pihole "nano /home/cal/caddy/.env"
|
|
# Set CF_API_TOKEN=<your token>
|
|
```
|
|
|
|
### 1.3 Build and Test Caddy (on alternate ports)
|
|
|
|
Temporarily modify `docker-compose.yml` to use non-conflicting ports:
|
|
```yaml
|
|
ports:
|
|
- "8080:80"
|
|
- "8443:443"
|
|
- "8443:443/udp"
|
|
```
|
|
|
|
```bash
|
|
ssh pihole "cd /home/cal/caddy && docker compose up -d --build"
|
|
```
|
|
|
|
Verify the container starts and the Cloudflare module is loaded:
|
|
```bash
|
|
ssh pihole "docker logs caddy 2>&1 | head -30"
|
|
ssh pihole "docker exec caddy caddy list-modules | grep cloudflare"
|
|
```
|
|
|
|
Verify config is valid:
|
|
```bash
|
|
ssh pihole "docker exec caddy caddy validate --config /etc/caddy/Caddyfile"
|
|
```
|
|
|
|
Test a proxy host directly (bypass DNS):
|
|
```bash
|
|
curl -k --resolve sbadev.manticorum.com:8443:10.10.0.16 https://sbadev.manticorum.com:8443/
|
|
```
|
|
|
|
### 1.4 Verify Cert Issuance
|
|
|
|
Check that Caddy successfully obtains a wildcard cert:
|
|
```bash
|
|
ssh pihole "docker logs caddy 2>&1 | grep -i 'certificate\|tls\|acme'"
|
|
```
|
|
|
|
You should see successful ACME DNS-01 challenge completion.
|
|
|
|
### 1.5 Stop Test Caddy
|
|
|
|
```bash
|
|
ssh pihole "cd /home/cal/caddy && docker compose down"
|
|
```
|
|
|
|
Revert `docker-compose.yml` ports back to 80/443.
|
|
|
|
## Phase 2: Cutover (brief downtime)
|
|
|
|
Expected downtime: **< 2 minutes** (stop NPM, start Caddy, sync DNS).
|
|
|
|
### 2.1 Backup NPM
|
|
|
|
```bash
|
|
ssh pihole "cd ~/nginx-proxy-manager && tar czf ~/npm-backup-$(date +%Y%m%d).tar.gz data/ letsencrypt/"
|
|
```
|
|
|
|
### 2.2 Take a Snapshot
|
|
|
|
If 10.10.0.16 is a VM/LXC on Proxmox, take a snapshot first:
|
|
```bash
|
|
# From proxmox host (adjust VMID)
|
|
pct snapshot <VMID> pre-caddy-migration
|
|
```
|
|
|
|
### 2.3 Stop NPM
|
|
|
|
```bash
|
|
ssh pihole "cd ~/nginx-proxy-manager && docker compose down"
|
|
```
|
|
|
|
Ports 80, 443, and 81 are now free.
|
|
|
|
### 2.4 Start Caddy
|
|
|
|
```bash
|
|
ssh pihole "cd /home/cal/caddy && docker compose up -d"
|
|
```
|
|
|
|
### 2.5 Verify Services
|
|
|
|
Quick smoke test of key services:
|
|
```bash
|
|
# Test from workstation (DNS should already point to 10.10.0.16 via Pi-hole)
|
|
curl -sI https://git.manticorum.com | head -5
|
|
curl -sI https://n8n.manticorum.com | head -5
|
|
curl -sI https://jellyfin.manticorum.com | head -5
|
|
curl -sI https://foundry.manticorum.com | head -5
|
|
curl -sI https://status.manticorum.com | head -5
|
|
|
|
# Test internal-only access
|
|
curl -sI https://radarr.manticorum.com | head -5 # should work from local
|
|
curl -sI https://sonarr.manticorum.com | head -5
|
|
```
|
|
|
|
### 2.6 Update Pi-hole Sync
|
|
|
|
Deploy the new sync script:
|
|
```bash
|
|
ssh pihole "cp /home/cal/caddy/scripts/caddy-pihole-sync.sh /home/cal/scripts/"
|
|
ssh pihole "chmod +x /home/cal/scripts/caddy-pihole-sync.sh"
|
|
```
|
|
|
|
Test dry run:
|
|
```bash
|
|
ssh pihole "/home/cal/scripts/caddy-pihole-sync.sh --dry-run"
|
|
```
|
|
|
|
Run sync:
|
|
```bash
|
|
ssh pihole "/home/cal/scripts/caddy-pihole-sync.sh"
|
|
```
|
|
|
|
Update cron to use the new script:
|
|
```bash
|
|
ssh pihole "crontab -l | sed 's|npm-pihole-sync.sh|caddy-pihole-sync.sh|g' | crontab -"
|
|
```
|
|
|
|
Also update the `CADDYFILE` path variable in the script if the deployment path differs from `/home/cal/caddy/Caddyfile`.
|
|
|
|
## Phase 3: Validate (next 24-48 hours)
|
|
|
|
### 3.1 Monitor Caddy Logs
|
|
|
|
```bash
|
|
ssh pihole "docker logs caddy -f"
|
|
```
|
|
|
|
Look for:
|
|
- Successful TLS handshakes
|
|
- No upstream connection errors
|
|
- Cert renewal events (if timing aligns)
|
|
|
|
### 3.2 Check Uptime Kuma
|
|
|
|
Verify all monitored services at https://status.manticorum.com show UP.
|
|
|
|
### 3.3 Test WebSocket Services
|
|
|
|
These services use WebSockets and should be tested interactively:
|
|
- **Foundry VTT** (foundry.manticorum.com) - open a game session
|
|
- **n8n** (n8n.manticorum.com) - open workflow editor
|
|
- **Memos** (memos.manticorum.com) - create/edit a memo
|
|
- **Termix** (termix.manticorum.com) - open a terminal session
|
|
|
|
### 3.4 Test External Access
|
|
|
|
If any services are accessed via Cloudflare from outside:
|
|
1. From a phone on cellular (not on home WiFi)
|
|
2. Access public services and verify they load
|
|
3. Access internal-only services and verify 403 response
|
|
|
|
### 3.5 Verify Access Restrictions
|
|
|
|
```bash
|
|
# From a machine NOT on 10.0.0.0/23 or 10.10.0.0/24:
|
|
curl -sI https://radarr.manticorum.com # Should return 403
|
|
curl -sI https://sonarr.manticorum.com # Should return 403
|
|
```
|
|
|
|
## Phase 4: Cleanup
|
|
|
|
### 4.1 Remove NPM (after validation period)
|
|
|
|
```bash
|
|
# Keep backup, remove containers and images
|
|
ssh pihole "cd ~/nginx-proxy-manager && docker compose rm -f"
|
|
ssh pihole "docker image rm jc21/nginx-proxy-manager:latest"
|
|
```
|
|
|
|
### 4.2 Update Documentation
|
|
|
|
- Update `server-configs/networking/nginx-proxy-manager-pihole.md` to reference Caddy
|
|
- Update any Uptime Kuma monitors that check port 81 (NPM admin)
|
|
- Update `CONTEXT.md` networking section
|
|
|
|
### 4.3 Free Port 81
|
|
|
|
Port 81 (NPM admin UI) is no longer needed. Caddy's admin API runs on localhost:2019 inside the container by default (not exposed).
|
|
|
|
## Rollback Plan
|
|
|
|
If something goes wrong, rollback takes < 1 minute:
|
|
|
|
```bash
|
|
# Stop Caddy
|
|
ssh pihole "cd /home/cal/caddy && docker compose down"
|
|
|
|
# Restart NPM
|
|
ssh pihole "cd ~/nginx-proxy-manager && docker compose up -d"
|
|
|
|
# Revert cron to old sync script
|
|
ssh pihole "crontab -l | sed 's|caddy-pihole-sync.sh|npm-pihole-sync.sh|g' | crontab -"
|
|
```
|
|
|
|
Or restore from Proxmox snapshot:
|
|
```bash
|
|
pct rollback <VMID> pre-caddy-migration
|
|
```
|
|
|
|
## Adding New Services After Migration
|
|
|
|
Edit the Caddyfile and reload - no web UI needed:
|
|
|
|
```bash
|
|
ssh pihole "nano /home/cal/caddy/Caddyfile"
|
|
|
|
# Add a new block:
|
|
# newservice.manticorum.com {
|
|
# reverse_proxy 10.10.0.xxx:port {
|
|
# import proxy_headers
|
|
# }
|
|
# }
|
|
|
|
# Validate
|
|
ssh pihole "docker exec caddy caddy validate --config /etc/caddy/Caddyfile"
|
|
|
|
# Apply (zero downtime)
|
|
ssh pihole "docker exec caddy caddy reload --config /etc/caddy/Caddyfile"
|
|
|
|
# Sync DNS to Pi-holes
|
|
ssh pihole "/home/cal/scripts/caddy-pihole-sync.sh"
|
|
```
|