cal/claude-home
1
0
Fork 0
Code Issues 6 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Add Gitea Actions workflow templates and automation #1

Merged
cal merged 2 commits from gitea-workflow-templates into main 2026-02-05 20:00:46 +00:00
Conversation 0 Commits 2 Files Changed 12 +2653
12 changed files with 2653 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

221
server-configs/gitea/INDEX.md Normal file
View File

@ -0,0 +1,221 @@
# Gitea Configuration & Templates Index
Quick reference for all Gitea-related documentation and templates.
## 📁 Directory Structure
```
/server-configs/gitea/
├── README.md # Gitea LXC setup and configuration
├── INDEX.md # This file - quick reference
├── deployment-strategies.md # When/how to deploy (manual vs auto)
├── harbor-registry-setup.md # Self-hosted Docker registry guide
├── apply_branch_protection.py # Script for branch protection rules
└── workflow-templates/ # Reusable CI/CD templates
├── README.md # Template usage guide
├── docker-build-template.yml # Complete Docker CI/CD workflow
├── deploy-script-template.sh # Safe manual deployment script
└── snippets/ # Workflow code snippets
└── auto-deploy-with-rollback.yml
```
## 🚀 Quick Links
### Getting Started
- **New to Gitea Actions?** → Start with `workflow-templates/README.md`
- **Need a workflow?** → Copy `docker-build-template.yml`
- **Want to deploy?** → Read `deployment-strategies.md`
### Templates
#### Docker Build Workflow
**File:** `workflow-templates/docker-build-template.yml`
**Features:**
- ✅ Semantic version validation
- ✅ Docker build + push to Docker Hub
- ✅ Discord notifications
- ✅ Build caching
- ✅ Multi-tag strategy
**Use for:**
- Discord bots
- Web applications
- API services
- Any Dockerized project
#### Manual Deploy Script
**File:** `workflow-templates/deploy-script-template.sh`
**Features:**
- ✅ Pre-deployment checks
- ✅ Confirmation prompts
- ✅ Status verification
- ✅ Log viewing
- ✅ Rollback instructions
**Use when:**
- You want manual control
- Deploying to production
- Testing new deployments
#### Auto-Deploy with Rollback
**File:** `workflow-templates/snippets/auto-deploy-with-rollback.yml`
**Features:**
- ✅ SSH deployment
- ✅ Health checks
- ✅ Automatic rollback
- ✅ Deployment notifications
**Use when:**
- You have good test coverage
- Health checks are implemented
- Downtime is acceptable
### Guides
#### Deployment Strategies
**File:** `deployment-strategies.md`
**Covers:**
- Decision framework (when to auto-deploy)
- 5 levels of deployment automation
- Recommendations by project type
- Safety best practices
- Rollback procedures
**Read this before:** Adding auto-deploy to any project
#### Harbor Registry Setup
**File:** `harbor-registry-setup.md`
**Covers:**
- Self-hosted Docker registry
- Complete Harbor installation
- Integration with Gitea Actions
- Vulnerability scanning
- Backup strategies
**Use when:**
- You want private registries
- You hit Docker Hub rate limits
- You want full control
- Learning opportunity
## 📝 Reference Implementations
### Paper Dynasty Discord Bot
**Status:** ✅ Production
**Date:** 2026-02-04
**Setup:**
- Gitea Actions on LXC 225
- Docker build + push to Docker Hub
- Discord notifications working
- Manual deployment to sba-bots (10.10.0.88)
**Workflow:** Based on `docker-build-template.yml`
**What worked:**
- Semantic version validation
- Multi-tag strategy (latest, version, version+commit)
- Discord webhooks with ISO 8601 timestamps
- GitHub Actions cache for faster builds
**Lessons learned:**
- Timestamp format critical for Discord (ISO 8601)
- Health checks needed before auto-deploy
- Manual deploy preferred for production Discord bots
## 🔧 Common Tasks
### Create New Project Workflow
```bash
cd /path/to/your/repo
mkdir -p .gitea/workflows
cp /path/to/docker-build-template.yml .gitea/workflows/docker-build.yml
# Customize:
# - Replace "yourusername/yourrepo" with your Docker Hub repo
# - Replace "Your Project" in notifications
# - Replace Discord webhook URLs
# - Add secrets: DOCKERHUB_USERNAME, DOCKERHUB_TOKEN
# - Create VERSION file: echo "1.0.0" > VERSION
```
### Add Auto-Deploy (When Ready)
```bash
# Copy auto-deploy snippet
cat workflow-templates/snippets/auto-deploy-with-rollback.yml
# Add to your workflow after build step
# Configure secrets: DEPLOY_SSH_KEY, PRODUCTION_HOST, DEPLOY_USER
```
### Manual Deploy
```bash
# Copy deploy script to your repo
cp workflow-templates/deploy-script-template.sh /path/to/repo/deploy.sh
chmod +x /path/to/repo/deploy.sh
# Customize server details in script
# Then use: ./deploy.sh v1.2.3
```
## 🎓 Learning Path
1. **Start:** Read `workflow-templates/README.md`
2. **Setup:** Deploy Gitea Actions runner (see `README.md`)
3. **First workflow:** Copy `docker-build-template.yml`
4. **Deploy:** Use `deploy-script-template.sh`
5. **Advanced:** Read `deployment-strategies.md`
6. **Optional:** Set up Harbor with `harbor-registry-setup.md`
## 🆘 Troubleshooting
### Version validation failing
- Check VERSION file format: just `1.2.3`
- Ensure semver rules followed
- See template comments for valid bumps
### Docker Hub push failing
- Verify secrets set in Gitea
- Check Docker Hub token permissions
- Ensure repo name matches exactly
### Discord notifications not appearing
- Verify webhook URL still valid
- Check timestamp format (ISO 8601)
- Test webhook manually with curl
### More help
- Check specific template comments
- Read troubleshooting sections in guides
- Review Paper Dynasty as reference implementation
## 📚 External Resources
- [Gitea Actions Docs](https://docs.gitea.io/en-us/usage/actions/overview/)
- [Docker Build Push Action](https://github.com/docker/build-push-action)
- [Semantic Versioning](https://semver.org/)
- [Discord Webhooks](https://discord.com/developers/docs/resources/webhook)
- [Harbor Docs](https://goharbor.io/docs/)
## 🔄 Maintenance
**Update this index when:**
- Adding new templates
- Creating new guides
- Changing directory structure
- Adding reference implementations
**Last updated:** 2026-02-04
---
**Quick search tips:**
- Need deployment guide? → `deployment-strategies.md`
- Need workflow template? → `workflow-templates/`
- Need self-hosted registry? → `harbor-registry-setup.md`
- Need deploy script? → `deploy-script-template.sh`

144
server-configs/gitea/README.md
View File

@ -304,3 +304,147 @@ This repository is mirrored on both GitHub and Gitea for redundancy:
- **GitHub**: https://github.com/calcorum/claude-home
- **Gitea**: https://git.manticorum.com/cal/claude-home
---
## Branch Protection Manager
Automated script to apply consistent branch protection rules across all your Gitea repositories.
### Features
- Applies branch protection rules to all repositories for a user/organization
- Supports dry-run mode to preview changes
- Configurable protection rules via environment variables
- Handles updating existing protection rules
- Clear success/error reporting
### Requirements
```bash
pip install requests
```
### Configuration
The script (`apply_branch_protection.py`) uses these settings:
- ✅ Disable direct pushes (force PR workflow)
- ✅ Require 1 approval before merge
- ✅ Restrict approvals to whitelisted users
- ✅ Dismiss stale approvals when new commits are pushed
- ✅ Enable status checks (for CI/CD)
- ✅ Restrict merging to whitelisted users
- ✅ Block merge on rejected reviews
- ✅ Block merge if pull request is outdated (critical for DB migrations)
### Usage
#### 1. Create a Gitea API Token
1. Go to https://git.manticorum.com/user/settings/applications
2. Click "Generate New Token"
3. Give it a name (e.g., "Branch Protection Script")
4. Select permissions: `repo` (full control)
5. Click "Generate Token"
6. Copy the token (you won't see it again!)
#### 2. Set Environment Variables
```bash
export GITEA_TOKEN='your-api-token-here'
export GITEA_URL='https://git.manticorum.com' # Optional, defaults to this
export GITEA_OWNER='cal' # Optional, defaults to cal
```
#### 3. Run the Script
**Dry run (preview changes without applying):**
```bash
cd /mnt/NV2/Development/claude-home/server-configs/gitea
python apply_branch_protection.py --dry-run
```
**Apply to all repositories:**
```bash
python apply_branch_protection.py
```
### Customizing Protection Rules
Edit the `BranchProtectionConfig` section in `main()` to customize the rules:
```python
config = BranchProtectionConfig(
branch_name="main", # Branch to protect
enable_push=False, # Disable direct pushes
required_approvals=1, # Number of required approvals
enable_approvals_whitelist=True, # Restrict who can approve
approvals_whitelist_usernames=[GITEA_OWNER],
dismiss_stale_approvals=True, # Dismiss approvals on new commits
enable_status_check=True, # Require status checks to pass
enable_merge_whitelist=True, # Restrict who can merge
merge_whitelist_usernames=[GITEA_OWNER],
block_on_rejected_reviews=True, # Block merge if reviews are rejected
block_on_outdated_branch=True, # Block merge if branch is outdated
require_signed_commits=False, # Require GPG signatures
)
```
### Example Output
```
============================================================
Gitea Branch Protection Configuration
============================================================
Gitea URL: https://git.manticorum.com
Owner: cal
Branch: main
============================================================
Protection Rules:
• Direct pushes: Disabled
• Required approvals: 1
• Approvals whitelist: cal
• Dismiss stale approvals: True
• Status checks enabled: True
• Merge whitelist: cal
• Block on rejected reviews: True
• Block on outdated branch: True
• Require signed commits: False
============================================================
Applying branch protection to 5 repositories...
📦 major-domo-database
🔄 Updating existing protection...
✅ Successfully applied branch protection
📦 major-domo-bot
✅ Successfully applied branch protection
...
```
### Troubleshooting Branch Protection Script
**"GITEA_TOKEN environment variable is required"**
- You need to set your API token. See usage step 2 above.
**"Error fetching repositories"**
- Check that your `GITEA_URL` is correct
- Verify your API token has `repo` permissions
- Ensure the `GITEA_OWNER` username is correct
**"Could not delete existing protection"**
- The script will still try to create the new protection
- If this fails, manually delete the old protection rule from Gitea web UI
**"Error: 422 Unprocessable Entity"**
This usually means:
- The branch doesn't exist in the repository
- Invalid username in whitelist
- Conflicting protection rule settings
### API References
This script uses the Gitea API branch protection endpoints:
- [Gitea API Documentation](https://docs.gitea.com/api/)
- [Protected Branches](https://docs.gitea.com/usage/access-control/protected-branches)
- [Create Branch Protection Endpoint](https://share.apidog.com/apidoc/docs-site/346218/api-3521477)

299
server-configs/gitea/apply_branch_protection.py Executable file
View File

@ -0,0 +1,299 @@
#!/usr/bin/env python3
"""
Apply branch protection rules to all Gitea repositories.
This script uses the Gitea API to apply consistent branch protection rules
across all repositories owned by a user or organization.
"""
import os
import sys
import requests
from typing import Optional
from dataclasses import dataclass
@dataclass
class BranchProtectionConfig:
"""Configuration for branch protection rules."""
# Branch to protect
branch_name: str = "main"
# Push settings
enable_push: bool = False # Disable direct pushes (force PR workflow)
enable_push_whitelist: bool = False
push_whitelist_usernames: list[str] = None
# Pull request approval settings
required_approvals: int = 1
enable_approvals_whitelist: bool = True
approvals_whitelist_usernames: list[str] = None
dismiss_stale_approvals: bool = True
# Status check settings
enable_status_check: bool = True
status_check_contexts: list[str] = None # Empty for now, add when CI/CD is set up
# Merge settings
enable_merge_whitelist: bool = True
merge_whitelist_usernames: list[str] = None
block_on_rejected_reviews: bool = True
block_on_outdated_branch: bool = True # Critical for database migrations
# Other settings
require_signed_commits: bool = False
protected_file_patterns: str = ""
unprotected_file_patterns: str = ""
def __post_init__(self):
"""Initialize mutable default values."""
if self.push_whitelist_usernames is None:
self.push_whitelist_usernames = []
if self.approvals_whitelist_usernames is None:
self.approvals_whitelist_usernames = []
if self.merge_whitelist_usernames is None:
self.merge_whitelist_usernames = []
if self.status_check_contexts is None:
self.status_check_contexts = []
class GiteaBranchProtectionManager:
"""Manages branch protection rules across Gitea repositories."""
def __init__(self, gitea_url: str, api_token: str, owner: str):
"""
Initialize the manager.
Args:
gitea_url: Base URL of your Gitea instance (e.g., https://git.example.com)
api_token: Gitea API token with repo access
owner: Username or organization name
"""
self.gitea_url = gitea_url.rstrip('/')
self.api_token = api_token
self.owner = owner
self.headers = {
'Authorization': f'token {api_token}',
'Content-Type': 'application/json'
}
def get_repositories(self) -> list[dict]:
"""Get all repositories for the authenticated user."""
# Use /user/repos for authenticated requests (works with API token)
url = f'{self.gitea_url}/api/v1/user/repos'
params = {'limit': 100} # Adjust if you have more than 100 repos
try:
response = requests.get(url, headers=self.headers, params=params)
response.raise_for_status()
repos = response.json()
# Filter to only repos owned by the specified owner if needed
if self.owner:
repos = [r for r in repos if r.get('owner', {}).get('login') == self.owner]
return repos
except requests.exceptions.RequestException as e:
print(f"❌ Error fetching repositories: {e}")
sys.exit(1)
def get_existing_branch_protections(self, repo_name: str) -> list[dict]:
"""Get existing branch protections for a repository."""
url = f'{self.gitea_url}/api/v1/repos/{self.owner}/{repo_name}/branch_protections'
try:
response = requests.get(url, headers=self.headers)
response.raise_for_status()
return response.json()
except requests.exceptions.RequestException as e:
print(f" ⚠️ Could not fetch existing protections: {e}")
return []
def delete_branch_protection(self, repo_name: str, protection_name: str) -> bool:
"""Delete an existing branch protection rule."""
url = f'{self.gitea_url}/api/v1/repos/{self.owner}/{repo_name}/branch_protections/{protection_name}'
try:
response = requests.delete(url, headers=self.headers)
response.raise_for_status()
return True
except requests.exceptions.RequestException as e:
print(f" ⚠️ Could not delete existing protection: {e}")
return False
def apply_branch_protection(self, repo_name: str, config: BranchProtectionConfig) -> bool:
"""
Apply branch protection rules to a repository.
Args:
repo_name: Name of the repository
config: Branch protection configuration
Returns:
True if successful, False otherwise
"""
url = f'{self.gitea_url}/api/v1/repos/{self.owner}/{repo_name}/branch_protections'
payload = {
'branch_name': config.branch_name,
'enable_push': config.enable_push,
'enable_push_whitelist': config.enable_push_whitelist,
'push_whitelist_usernames': config.push_whitelist_usernames,
'required_approvals': config.required_approvals,
'enable_approvals_whitelist': config.enable_approvals_whitelist,
'approvals_whitelist_usernames': config.approvals_whitelist_usernames,
'dismiss_stale_approvals': config.dismiss_stale_approvals,
'enable_status_check': config.enable_status_check,
'status_check_contexts': config.status_check_contexts,
'enable_merge_whitelist': config.enable_merge_whitelist,
'merge_whitelist_usernames': config.merge_whitelist_usernames,
'block_on_rejected_reviews': config.block_on_rejected_reviews,
'block_on_outdated_branch': config.block_on_outdated_branch,
'require_signed_commits': config.require_signed_commits,
'protected_file_patterns': config.protected_file_patterns,
'unprotected_file_patterns': config.unprotected_file_patterns,
}
try:
response = requests.post(url, headers=self.headers, json=payload)
response.raise_for_status()
return True
except requests.exceptions.RequestException as e:
print(f" ❌ Error: {e}")
if hasattr(e.response, 'text'):
print(f" Response: {e.response.text}")
return False
def apply_to_all_repos(self, config: BranchProtectionConfig, dry_run: bool = False):
"""
Apply branch protection rules to all repositories.
Args:
config: Branch protection configuration
dry_run: If True, only show what would be done without making changes
"""
repos = self.get_repositories()
if not repos:
print("No repositories found.")
return
print(f"\n{'DRY RUN - ' if dry_run else ''}Applying branch protection to {len(repos)} repositories...\n")
success_count = 0
skip_count = 0
error_count = 0
for repo in repos:
repo_name = repo['name']
print(f"📦 {repo_name}")
# Check if branch exists
default_branch = repo.get('default_branch', 'main')
if config.branch_name != default_branch:
print(f" ℹ️ Default branch is '{default_branch}', protecting '{config.branch_name}' instead")
if dry_run:
print(f" ✓ Would apply protection to '{config.branch_name}' branch")
skip_count += 1
continue
# Check for existing protection
existing_protections = self.get_existing_branch_protections(repo_name)
existing_main_protection = None
for protection in existing_protections:
if protection.get('branch_name') == config.branch_name:
existing_main_protection = protection.get('id') or protection.get('branch_name')
break
# Delete existing protection if it exists
if existing_main_protection:
print(f" 🔄 Updating existing protection...")
self.delete_branch_protection(repo_name, existing_main_protection)
# Apply new protection
if self.apply_branch_protection(repo_name, config):
print(f" ✅ Successfully applied branch protection")
success_count += 1
else:
print(f" ❌ Failed to apply branch protection")
error_count += 1
print(f"\n{'=' * 60}")
print(f"Summary:")
print(f" ✅ Successful: {success_count}")
if dry_run:
print(f" ⏭️ Skipped (dry run): {skip_count}")
if error_count > 0:
print(f" ❌ Errors: {error_count}")
print(f"{'=' * 60}\n")
def main():
"""Main entry point."""
# Configuration - adjust these values or use environment variables
GITEA_URL = os.getenv('GITEA_URL', 'https://git.manticorum.com')
GITEA_TOKEN = os.getenv('GITEA_TOKEN')
GITEA_OWNER = os.getenv('GITEA_OWNER', 'cal')
# Check for required configuration
if not GITEA_TOKEN:
print("❌ Error: GITEA_TOKEN environment variable is required")
print("\nUsage:")
print(" export GITEA_TOKEN='your-api-token-here'")
print(" python apply_branch_protection.py [--dry-run]")
print("\nOptional environment variables:")
print(" GITEA_URL (default: https://git.manticorum.com)")
print(" GITEA_OWNER (default: cal)")
sys.exit(1)
# Parse command line arguments
dry_run = '--dry-run' in sys.argv
# Configure branch protection rules
# Adjust these settings as needed
config = BranchProtectionConfig(
branch_name="main",
enable_push=False, # Disable direct pushes
required_approvals=1,
enable_approvals_whitelist=True,
approvals_whitelist_usernames=[GITEA_OWNER],
dismiss_stale_approvals=True,
enable_status_check=True,
enable_merge_whitelist=True,
merge_whitelist_usernames=[GITEA_OWNER],
block_on_rejected_reviews=True,
block_on_outdated_branch=True, # Critical for DB repos
require_signed_commits=False,
)
# Display configuration
print(f"\n{'=' * 60}")
print(f"Gitea Branch Protection Configuration")
print(f"{'=' * 60}")
print(f"Gitea URL: {GITEA_URL}")
print(f"Owner: {GITEA_OWNER}")
print(f"Branch: {config.branch_name}")
print(f"{'=' * 60}")
print(f"Protection Rules:")
print(f" • Direct pushes: {'Disabled' if not config.enable_push else 'Enabled'}")
print(f" • Required approvals: {config.required_approvals}")
print(f" • Approvals whitelist: {', '.join(config.approvals_whitelist_usernames)}")
print(f" • Dismiss stale approvals: {config.dismiss_stale_approvals}")
print(f" • Status checks enabled: {config.enable_status_check}")
print(f" • Merge whitelist: {', '.join(config.merge_whitelist_usernames)}")
print(f" • Block on rejected reviews: {config.block_on_rejected_reviews}")
print(f" • Block on outdated branch: {config.block_on_outdated_branch}")
print(f" • Require signed commits: {config.require_signed_commits}")
print(f"{'=' * 60}\n")
if dry_run:
print("🔍 DRY RUN MODE - No changes will be made\n")
# Create manager and apply rules
manager = GiteaBranchProtectionManager(GITEA_URL, GITEA_TOKEN, GITEA_OWNER)
manager.apply_to_all_repos(config, dry_run=dry_run)
if __name__ == '__main__':
main()

356
server-configs/gitea/deployment-strategies.md Normal file
View File

@ -0,0 +1,356 @@
# Deployment Strategies for Gitea CI/CD
Guide to choosing and implementing deployment automation for your projects.
## 🎯 Decision Framework
### Consider These Factors
1. **Criticality**: How important is uptime?
- Discord bot for active league: HIGH
- Personal project: LOW
- Public API: VERY HIGH
2. **Test Coverage**: How confident are you in your tests?
- Comprehensive unit + integration tests: AUTO-DEPLOY OK
- Some tests: SEMI-AUTO
- No tests: MANUAL ONLY
3. **Rollback Complexity**: Can you easily revert?
- Docker with versioned images: EASY
- Database migrations: HARD
- Stateful services: MEDIUM
4. **Traffic Patterns**: When can you deploy?
- 24/7 users: Need zero-downtime
- Business hours only: Deploy off-hours
- Low traffic: Restart acceptable
## 📊 Deployment Options Comparison
| Strategy | Speed | Safety | Complexity | Best For |
|----------|-------|--------|------------|----------|
| **Manual** | Slow | High | Low | Critical systems, low test coverage |
| **Semi-Auto** | Medium | High | Low | Most projects (recommended) |
| **Auto + Health** | Fast | Medium | Medium | Good test coverage, health endpoints |
| **Blue-Green** | Fast | High | High | Zero-downtime required |
| **Canary** | Fast | Very High | Very High | Large user base, gradual rollout |
## 🚀 Recommended Progression
### Level 1: Manual Deploy (Start Here)
```bash
# Simple, safe, controllable
ssh production
docker compose pull
docker compose up -d
```
**Pros:**
- ✅ Full control over timing
- ✅ Can test/verify first
- ✅ Simple to understand
- ✅ Easy to abort
**Cons:**
- ❌ Requires manual SSH
- ❌ Inconsistent process
- ❌ Slower deployments
### Level 2: Deploy Script (Easy Win)
```bash
# One command from local machine
./deploy.sh v1.8.1
```
**Pros:**
- ✅ Consistent process
- ✅ Pre-deployment checks
- ✅ Still manual control
- ✅ Rollback instructions
**Cons:**
- ❌ Still requires human
- ❌ Can't deploy from CI
**Setup:** Copy `/tmp/deploy.sh` to your repo root
### Level 3: CI Notifies, Manual Deploy
```yaml
# Workflow builds, then notifies:
- Discord: "v1.8.1 ready to deploy"
- You run: ./deploy.sh v1.8.1
```
**Pros:**
- ✅ Fast build process
- ✅ Clear deployment signal
- ✅ Choose deployment timing
- ✅ Audit trail
**Cons:**
- ❌ Two-step process
- ❌ Can forget to deploy
**Current State:** This is where Paper Dynasty is now ✅
### Level 4: Auto-Deploy with Safeguards
```yaml
# Workflow builds, tests, deploys, health checks
- Build image
- Push to Docker Hub
- SSH to production
- Pull and restart
- Health check
- Rollback on failure
- Notify Discord
```
**Pros:**
- ✅ Fully automated
- ✅ Fast deployment
- ✅ Automatic rollback
- ✅ Consistent process
**Cons:**
- ❌ Complexity
- ❌ Brief downtime
- ❌ Requires good health checks
**Setup:** Use `/tmp/safe-auto-deploy-step.yml`
### Level 5: Blue-Green Deployment
```yaml
# Run two environments, switch traffic
- Deploy to "green" environment
- Health check green
- Switch load balancer to green
- Keep "blue" as instant rollback
```
**Pros:**
- ✅ Zero downtime
- ✅ Instant rollback
- ✅ Test before switching
- ✅ Very safe
**Cons:**
- ❌ Requires two environments
- ❌ Needs load balancer
- ❌ Database complexity
- ❌ 2x resources
**Best For:** High-traffic web services
## 🎯 Recommendations by Project Type
### Discord Bot (Paper Dynasty)
**Recommended:** Level 3 (CI Notify + Manual Deploy)
**Why?**
- Active users (league members)
- Brief downtime acceptable (1-2 min)
- Manual timing is valuable
- Can deploy during off-hours
**Next Step:** Add deploy script for consistency
### Internal Tool / Dashboard
**Recommended:** Level 4 (Auto-Deploy with Health Check)
**Why?**
- Limited users
- Downtime less critical
- Faster iteration valuable
- Easy rollback available
### Public API / High Traffic
**Recommended:** Level 5 (Blue-Green) or Canary
**Why?**
- Zero downtime required
- Large user base
- Complex rollback scenarios
- Worth the complexity
### Personal Project / Portfolio
**Recommended:** Level 2 or 3
**Why?**
- Simple is better
- Automation overhead not worth it
- Infrequent deploys
- Learning opportunity
## 🛠️ Implementation Guide
### Adding Auto-Deploy to Paper Dynasty
If you decide to try auto-deploy:
1. **Add deploy SSH key to secrets:**
```bash
# Generate deploy-only SSH key
ssh-keygen -t ed25519 -f ~/.ssh/paper_dynasty_deploy -C "gitea-deploy"
# Add public key to sba-bots
ssh-copy-id -i ~/.ssh/paper_dynasty_deploy.pub cal@10.10.0.88
# Add private key to Gitea secrets
cat ~/.ssh/paper_dynasty_deploy
# Copy to: Repo → Settings → Secrets → DEPLOY_SSH_KEY
```
2. **Add production host secret:**
- Secret name: `PRODUCTION_HOST`
- Value: `10.10.0.88`
3. **Add deploy user secret:**
- Secret name: `DEPLOY_USER`
- Value: `cal`
4. **Add deploy step to workflow:**
- Copy from `/tmp/safe-auto-deploy-step.yml`
- Paste after "Build Docker image" step
- Update webhook URL
5. **Test with a minor change:**
- Make a small PR (comment change)
- Merge and watch deployment
- Verify bot restarts successfully
- Check Discord for notifications
6. **Monitor first few deployments:**
- Watch for issues
- Check logs: `ssh sba-bots 'docker compose logs'`
- Verify no errors
### Adding Health Checks
For better auto-deploy safety, add a health endpoint to your bot:
```javascript
// Express health check endpoint
app.get('/health', (req, res) => {
// Check Discord connection
if (!client.ws.ping) {
return res.status(503).json({ status: 'unhealthy', reason: 'Discord disconnected' });
}
// Check database connection
if (!db.isConnected()) {
return res.status(503).json({ status: 'unhealthy', reason: 'Database disconnected' });
}
res.json({
status: 'healthy',
version: process.env.VERSION,
uptime: process.uptime(),
discord: {
guilds: client.guilds.cache.size,
ping: client.ws.ping
}
});
});
```
Then health check in deploy script:
```bash
curl -f http://localhost:3000/health || rollback
```
## ⚠️ Safety Best Practices
### Before Auto-Deploying
1. **✅ Add Comprehensive Tests**
- Unit tests for core logic
- Integration tests for Discord API
- Smoke tests for critical paths
2. **✅ Implement Health Checks**
- HTTP endpoint for status
- Check all critical services
- Return proper status codes
3. **✅ Set Up Monitoring**
- Error tracking (Sentry)
- Performance monitoring (Datadog)
- Discord webhook for errors
4. **✅ Plan Rollback Process**
- Keep last 3 images cached
- Document rollback steps
- Test rollback procedure
5. **✅ Document Deployment**
- What gets deployed
- How to abort
- How to rollback
- Who to contact
### During Deployment
1. **Monitor Logs**
```bash
docker compose logs -f
```
2. **Check Metrics**
- Response times
- Error rates
- Resource usage
3. **Verify Functionality**
- Test critical commands
- Check database connections
- Verify integrations
### After Deployment
1. **Watch for Issues**
- First 5 minutes are critical
- Check error channels
- Monitor user reports
2. **Validate Success**
- All services healthy
- No error spike
- Performance normal
3. **Document Issues**
- What went wrong
- How you fixed it
- How to prevent
## 🔄 Rollback Procedures
### Quick Rollback (Docker)
```bash
ssh production
cd /path/to/app
docker compose down
docker tag yourusername/repo:current yourusername/repo:rollback-backup
docker pull yourusername/repo:v1.7.0 # Last known good
docker compose up -d
```
### Emergency Rollback (CI)
```yaml
# Manually trigger workflow with old version
git tag -f v1.7.0-redeploy
git push -f origin v1.7.0-redeploy
# Watch actions deploy old version
```
## 📚 Further Reading
- [Deployment Strategies](https://www.redhat.com/en/topics/devops/deployment-strategies)
- [Blue-Green Deployments](https://martinfowler.com/bliki/BlueGreenDeployment.html)
- [Canary Releases](https://martinfowler.com/bliki/CanaryRelease.html)
- [Docker Health Checks](https://docs.docker.com/engine/reference/builder/#healthcheck)
---
**Last Updated:** 2026-02-04
**Applies To:** Paper Dynasty Discord bot, future Gitea CI/CD projects

377
server-configs/gitea/harbor-registry-setup.md Normal file
View File

@ -0,0 +1,377 @@
# Harbor Docker Registry Setup Guide
Complete guide to setting up Harbor on a Proxmox LXC for self-hosted Docker registry.
## Prerequisites
- Proxmox LXC with Ubuntu 22.04
- 2 CPU cores, 4GB RAM, 50GB disk
- Docker and docker-compose installed
- Domain name (e.g., registry.manticorum.com)
## Quick Setup
### 1. Create LXC Container
```bash
# On Proxmox host
pct create 227 local:vztmpl/ubuntu-22.04-standard_22.04-1_amd64.tar.zst \
--hostname harbor \
--cores 2 \
--memory 4096 \
--swap 512 \
--net0 name=eth0,bridge=vmbr0,ip=10.10.0.227/24,gw=10.10.0.1 \
--rootfs local-lvm:50 \
--unprivileged 1 \
--features nesting=1 \
--onboot 1 \
--start 1
```
### 2. Install Docker
```bash
ssh root@10.10.0.227
apt update && apt install -y curl
curl -fsSL https://get.docker.com | sh
systemctl enable docker
```
### 3. Download Harbor
```bash
cd /opt
wget https://github.com/goharbor/harbor/releases/download/v2.10.0/harbor-offline-installer-v2.10.0.tgz
tar xzvf harbor-offline-installer-v2.10.0.tgz
cd harbor
```
### 4. Configure Harbor
```bash
cp harbor.yml.tmpl harbor.yml
# Edit harbor.yml
nano harbor.yml
```
**Key settings to change:**
```yaml
hostname: registry.manticorum.com # Your domain
# HTTPS (configure after NPM setup, start with HTTP for now)
# https:
# port: 443
# certificate: /path/to/cert
# private_key: /path/to/key
# Or disable HTTPS initially
# Comment out entire https section
harbor_admin_password: YourSecurePassword123
database:
password: YourDBPassword123
data_volume: /mnt/harbor-data
```
### 5. Install Harbor
```bash
./install.sh
```
### 6. Access Harbor
Open: `http://10.10.0.227` (or `http://registry.manticorum.com` if DNS configured)
**Default login:**
- Username: `admin`
- Password: `YourSecurePassword123` (what you set)
### 7. Configure NPM Reverse Proxy
In Nginx Proxy Manager (10.10.0.16):
**Proxy Host:**
- Domain: `registry.manticorum.com`
- Scheme: `http`
- Forward Hostname: `10.10.0.227`
- Forward Port: `80`
- Websockets: ✅ Enabled
- Block Common Exploits: ✅ Enabled
- SSL: Let's Encrypt
**Custom Nginx Configuration:**
```nginx
# Increase timeouts for large image uploads
proxy_read_timeout 900;
proxy_send_timeout 900;
client_max_body_size 0; # No upload limit
# Required for Docker registry
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
```
## Using Your Registry
### 1. Login from Dev Machine
```bash
docker login registry.manticorum.com
# Username: admin
# Password: YourSecurePassword123
```
### 2. Tag and Push Image
```bash
# Tag existing image
docker tag manticorum67/paper-dynasty:latest registry.manticorum.com/paper-dynasty/bot:latest
# Push to your registry
docker push registry.manticorum.com/paper-dynasty/bot:latest
```
### 3. Pull from Production
```bash
# On sba-bots
docker login registry.manticorum.com
docker pull registry.manticorum.com/paper-dynasty/bot:latest
```
### 4. Update docker-compose
```yaml
services:
paper-dynasty:
# Old: image: manticorum67/paper-dynasty:latest
# New:
image: registry.manticorum.com/paper-dynasty/bot:latest
```
## Integrating with Gitea Actions
Update your workflow to push to both registries:
```yaml
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Harbor
uses: docker/login-action@v3
with:
registry: registry.manticorum.com
username: ${{ secrets.HARBOR_USERNAME }}
password: ${{ secrets.HARBOR_PASSWORD }}
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
push: ${{ github.ref == 'refs/heads/main' }}
tags: |
manticorum67/paper-dynasty:latest
manticorum67/paper-dynasty:v${{ steps.meta.outputs.version }}
registry.manticorum.com/paper-dynasty/bot:latest
registry.manticorum.com/paper-dynasty/bot:v${{ steps.meta.outputs.version }}
```
## Harbor Features
### Create Projects
1. Login to Harbor UI
2. Click **New Project**
3. Name: `paper-dynasty`
4. Access Level: Private or Public
### Enable Vulnerability Scanning
1. Go to **Administration** → **Interrogation Services**
2. Enable **Trivy** scanner
3. Set scan on push: ✅ Enabled
Now images are auto-scanned for CVEs!
### Set Up Replication
Replicate between Harbor and Docker Hub:
1. **Administration** → **Replications**
2. **New Replication Rule**
- Name: `sync-to-dockerhub`
- Source: Local
- Destination: Docker Hub (add endpoint first)
- Trigger: Event Based
### Garbage Collection
Free up disk space from deleted images:
1. **Administration** → **Garbage Collection**
2. Schedule: Daily at 2 AM
3. Dry run first to see what would be deleted
## Backup Strategy
### What to Backup
1. **Harbor database** (PostgreSQL)
2. **Image storage** (`/mnt/harbor-data`)
3. **Configuration** (`/opt/harbor/harbor.yml`)
### Backup Script
```bash
#!/bin/bash
BACKUP_DIR="/mnt/backups/harbor"
DATE=$(date +%Y%m%d)
# Stop Harbor
cd /opt/harbor
docker-compose down
# Backup database
docker exec harbor-db pg_dumpall -U postgres > $BACKUP_DIR/harbor-db-$DATE.sql
# Backup data (incremental)
rsync -av /mnt/harbor-data/ $BACKUP_DIR/harbor-data/
# Backup config
cp /opt/harbor/harbor.yml $BACKUP_DIR/harbor-config-$DATE.yml
# Start Harbor
docker-compose up -d
# Keep last 7 days
find $BACKUP_DIR -name "harbor-db-*.sql" -mtime +7 -delete
```
## Monitoring
### Check Harbor Status
```bash
cd /opt/harbor
docker-compose ps
# View logs
docker-compose logs -f
```
### Disk Usage
```bash
du -sh /mnt/harbor-data
# By project
du -sh /mnt/harbor-data/docker/registry/v2/repositories/*
```
### API Health Check
```bash
curl -k https://registry.manticorum.com/api/v2.0/health
```
## Troubleshooting
### "401 Unauthorized" on push
**Problem:** Docker login not working
**Solution:**
```bash
# Clear old credentials
rm ~/.docker/config.json
# Login again
docker login registry.manticorum.com
```
### "413 Request Entity Too Large"
**Problem:** Nginx upload limit
**Solution:** Add to NPM custom config:
```nginx
client_max_body_size 0;
```
### Disk space full
**Problem:** Old images filling disk
**Solution:**
```bash
# Run garbage collection
cd /opt/harbor
docker-compose exec core /harbor/garbage-collection.sh
# Or via UI: Administration → Garbage Collection → Run Now
```
### Can't pull from registry
**Problem:** Firewall or network issue
**Solution:**
```bash
# Test connection
telnet 10.10.0.227 80
# Check Harbor logs
docker-compose logs registry
```
## Advanced: High Availability
For production-critical registries, set up HA:
1. Multiple Harbor instances
2. Shared storage (NFS, S3, Minio)
3. Load balancer in front
4. Database replication
## Cost Analysis
**LXC Resources:**
- CPU: 2 cores = $0 (spare capacity)
- RAM: 4GB = $0 (spare capacity)
- Disk: 50GB = $0 (local storage)
- Bandwidth: Internal = $0
**Total ongoing cost: $0/month**
**Docker Hub Pro alternative: $5/month**
**Time investment:**
- Setup: 2-3 hours
- Maintenance: 30 min/month
- Break-even: 3 months of learning value
## Next Steps
1. ✅ Set up Harbor on LXC 227
2. ✅ Configure NPM reverse proxy
3. ✅ Test push/pull from dev machine
4. ✅ Update one project to use Harbor
5. ✅ Set up Gitea Actions to push to both registries
6. ✅ Configure vulnerability scanning
7. ✅ Set up automated backups
---
**Created:** 2026-02-04
**For:** Manticorum Home Lab
**Reference:** Paper Dynasty as first use case

221
server-configs/gitea/workflow-templates/README.md Normal file
View File

@ -0,0 +1,221 @@
# Gitea Actions Workflow Templates
Reusable CI/CD workflow templates for Gitea Actions (GitHub Actions compatible).
## Templates
### `docker-build-template.yml`
Complete Docker build pipeline with semantic versioning validation, Docker Hub push, and Discord notifications.
**Features:**
- ✅ Semantic version validation on PRs
- ✅ Docker build on every push/PR
- ✅ Push to Docker Hub on main branch
- ✅ Discord notifications (success/failure)
- ✅ Build caching for faster builds
- ✅ Multi-tag strategy (latest, version, version+commit)
**Reference Implementation:**
Paper Dynasty Discord bot - First production use (2026-02-04)
## Quick Start
1. **Copy template to your repo:**
```bash
mkdir -p .gitea/workflows
cp docker-build-template.yml .gitea/workflows/docker-build.yml
```
2. **Customize placeholders:**
- Replace `yourusername/yourrepo` with your Docker Hub repository
- Replace `Your Project` in notification titles
- Replace `YOUR_DISCORD_WEBHOOK_URL_HERE` with your webhook URLs
3. **Add Gitea secrets:**
- Go to your repo → Settings → Secrets → Actions
- Add `DOCKERHUB_USERNAME` (your Docker Hub username)
- Add `DOCKERHUB_TOKEN` (access token from hub.docker.com)
4. **Create VERSION file:**
```bash
echo "1.0.0" > VERSION
git add VERSION
git commit -m "Add initial VERSION file"
```
5. **Push and test:**
- Create a PR to test version validation
- Merge to main to test Docker push and notifications
## Customization Guide
### Disable Features
**Don't want version validation?**
- Delete the "Check VERSION was bumped" step
**Don't want Discord notifications?**
- Delete both "Discord Notification" steps
**Don't want Docker Hub push?**
- Remove "Login to Docker Hub" step
- Change `push: ${{ github.ref == 'refs/heads/main' }}` to `push: false`
### Customize Version Validation
The template enforces strict semantic versioning. To modify:
**Allow any version bump:**
```bash
# Remove the validation logic, just check if changed:
if [ "$PR_VERSION" = "$MAIN_VERSION" ]; then
echo "❌ VERSION unchanged"
exit 1
fi
echo "✅ VERSION changed: $MAIN_VERSION → $PR_VERSION"
```
**Allow pre-release versions:**
```bash
# Modify parsing to handle versions like "1.2.3-beta"
IFS='-' read -r VERSION_NUMBER PRERELEASE <<< "$PR_VERSION"
```
### Add More Notifications
**Slack webhook:**
```yaml
- name: Slack Notification
if: success() && github.ref == 'refs/heads/main'
run: |
curl -X POST YOUR_SLACK_WEBHOOK_URL \
-H 'Content-Type: application/json' \
-d '{"text": "Build succeeded: v${{ steps.meta.outputs.version }}"}'
```
**Email notification:**
```yaml
- name: Email Notification
if: failure()
uses: dawidd6/action-send-mail@v3
with:
server_address: smtp.gmail.com
server_port: 465
username: ${{ secrets.EMAIL_USERNAME }}
password: ${{ secrets.EMAIL_PASSWORD }}
subject: Build Failed - ${{ github.repository }}
body: Build failed on commit ${{ github.sha }}
to: you@example.com
```
## Troubleshooting
### Version Validation Issues
**Problem:** PR fails validation but VERSION was bumped
**Solution:** Check VERSION file format - should be just `1.2.3` with no prefix, suffix, or extra text
**Problem:** Validation allows invalid bumps
**Solution:** Version parsing may be failing - check for special characters in VERSION file
### Docker Hub Push Issues
**Problem:** Login fails with "unauthorized"
**Solution:** Regenerate Docker Hub access token at hub.docker.com → Settings → Security
**Problem:** Push fails with "denied"
**Solution:** Check repository name matches exactly (case-sensitive)
**Problem:** Tags not appearing on Docker Hub
**Solution:** Wait a few seconds and refresh - tags may take a moment to appear
### Discord Notification Issues
**Problem:** Webhook succeeds but no message appears
**Solution:**
1. Check timestamp format is ISO 8601: `YYYY-MM-DDTHH:MM:SSZ`
2. Test webhook manually with curl
3. Verify webhook hasn't been deleted in Discord
**Problem:** Message appears malformed
**Solution:** Check for unescaped quotes or special characters in message content
**Problem:** Rate limited
**Solution:** Discord limits webhooks to ~5 messages per second - add delays if sending multiple
## Advanced Usage
### Multi-Stage Builds
Add test/lint steps before build:
```yaml
- name: Run tests
run: |
npm install
npm test
- name: Lint code
run: npm run lint
- name: Build Docker image
# ... existing build step
```
### Deploy After Build
Add deployment to production:
```yaml
- name: Deploy to production
if: success() && github.ref == 'refs/heads/main'
run: |
ssh production "docker pull yourusername/yourrepo:latest && docker-compose up -d"
```
### Multiple Docker Registries
Push to multiple registries:
```yaml
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build Docker image
uses: docker/build-push-action@v5
with:
push: ${{ github.ref == 'refs/heads/main' }}
tags: |
yourusername/yourrepo:latest
ghcr.io/yourusername/yourrepo:latest
```
## Template Updates
This template was created based on the Paper Dynasty Discord bot workflow and represents battle-tested CI/CD practices. Future improvements might include:
- [ ] Automatic changelog generation from commits
- [ ] Security scanning (Trivy, Snyk)
- [ ] Multi-architecture builds (ARM, AMD64)
- [ ] Deployment strategies (blue-green, canary)
- [ ] Integration testing with docker-compose
- [ ] Performance benchmarking
## Contributing
Found a bug or improvement? Update this template and document the change in this README.
## License
Free to use and modify for any project.
---
**Template Version:** 1.0.0
**Last Updated:** 2026-02-04
**Maintained By:** Manticorum Home Lab

82
server-configs/gitea/workflow-templates/deploy-script-template.sh Normal file
View File

@ -0,0 +1,82 @@
#!/bin/bash
# Paper Dynasty - Manual Deployment Script
#
# Usage: ./deploy.sh [version]
# Example: ./deploy.sh v1.8.1
#
# This script provides a safe, manual way to deploy Paper Dynasty
# with proper checks and rollback capability.
set -e
VERSION=${1:-latest}
SERVER="sba-bots"
SERVER_IP="10.10.0.88"
DEPLOY_PATH="/home/cal/container-data/paper-dynasty"
echo "🚀 Deploying Paper Dynasty ${VERSION} to production"
echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━"
# Pre-deployment checks
echo ""
echo "📋 Pre-deployment checks..."
# Check SSH connection
if ! ssh cal@${SERVER_IP} "echo 'SSH OK'" > /dev/null 2>&1; then
echo "❌ Cannot connect to ${SERVER}"
exit 1
fi
echo "✅ SSH connection OK"
# Check if container exists
if ! ssh cal@${SERVER_IP} "cd ${DEPLOY_PATH} && docker compose ps" > /dev/null 2>&1; then
echo "❌ Cannot find Paper Dynasty container on ${SERVER}"
exit 1
fi
echo "✅ Container found"
# Confirm deployment
echo ""
echo "⚠️ This will restart the Paper Dynasty bot (brief downtime)"
read -p "Continue with deployment? (y/N) " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "❌ Deployment cancelled"
exit 1
fi
# Deploy
echo ""
echo "📥 Pulling image manticorum67/paper-dynasty:${VERSION}..."
ssh cal@${SERVER_IP} << EOF
cd ${DEPLOY_PATH}
# Pull new image
docker compose pull
# Stop old container
echo "🛑 Stopping container..."
docker compose down
# Start new container
echo "▶️ Starting container..."
docker compose up -d
# Wait for startup
sleep 5
# Check status
echo ""
echo "📊 Container status:"
docker compose ps
echo ""
echo "📝 Recent logs:"
docker compose logs --tail 20
EOF
echo ""
echo "✅ Deployment complete!"
echo ""
echo "To check logs: ssh ${SERVER} 'cd ${DEPLOY_PATH} && docker compose logs -f'"
echo "To rollback: ssh ${SERVER} 'cd ${DEPLOY_PATH} && docker compose down && docker compose up -d'"

392
server-configs/gitea/workflow-templates/docker-build-template.yml Normal file
View File

@ -0,0 +1,392 @@
# Gitea Actions: Docker Build, Push, and Notify
#
# This workflow provides a complete CI/CD pipeline for Docker-based projects:
# - Validates semantic versioning on PRs
# - Builds Docker images on every push/PR
# - Pushes to Docker Hub on main branch merges
# - Sends Discord notifications on success/failure
#
# Template created: 2026-02-04
# For: Paper Dynasty Discord bot (reference implementation)
name: Build Docker Image
on:
push:
branches:
- main
pull_request:
branches:
- main
jobs:
build:
runs-on: ubuntu-latest
steps:
# ==============================================
# 1. CHECKOUT CODE
# ==============================================
- name: Checkout code
uses: actions/checkout@v4
# ==============================================
# 2. SEMANTIC VERSION VALIDATION (PRs only)
# ==============================================
# Enforces proper semantic versioning:
# - Blocks PRs that don't bump VERSION file
# - Validates version changes follow semver rules
# - Prevents skipping versions or going backwards
#
# Valid bumps:
# - Patch: 1.2.3 → 1.2.4 (bug fixes)
# - Minor: 1.2.3 → 1.3.0 (new features)
# - Major: 1.2.3 → 2.0.0 (breaking changes)
#
# Invalid bumps:
# - 1.2.3 → 1.4.0 (skipped minor version)
# - 1.2.3 → 1.2.0 (went backwards)
# - 1.2.3 → 1.3.1 (didn't reset patch)
#
- name: Check VERSION was bumped (semantic versioning)
if: github.event_name == 'pull_request'
run: |
# Get VERSION from this PR branch
PR_VERSION=$(cat VERSION 2>/dev/null || echo "0.0.0")
# Get VERSION from main branch
git fetch origin main:main
MAIN_VERSION=$(git show main:VERSION 2>/dev/null || echo "0.0.0")
echo "📋 Semantic Version Check"
echo "Main branch version: $MAIN_VERSION"
echo "PR branch version: $PR_VERSION"
echo ""
# Parse versions into components
IFS='.' read -r MAIN_MAJOR MAIN_MINOR MAIN_PATCH <<< "$MAIN_VERSION"
IFS='.' read -r PR_MAJOR PR_MINOR PR_PATCH <<< "$PR_VERSION"
# Remove any non-numeric characters
MAIN_MAJOR=${MAIN_MAJOR//[!0-9]/}
MAIN_MINOR=${MAIN_MINOR//[!0-9]/}
MAIN_PATCH=${MAIN_PATCH//[!0-9]/}
PR_MAJOR=${PR_MAJOR//[!0-9]/}
PR_MINOR=${PR_MINOR//[!0-9]/}
PR_PATCH=${PR_PATCH//[!0-9]/}
# Check if VERSION unchanged
if [ "$PR_VERSION" = "$MAIN_VERSION" ]; then
echo "❌ ERROR: VERSION file has not been updated!"
echo ""
echo "Please update the VERSION file in your PR."
echo "Current version: $MAIN_VERSION"
exit 1
fi
# Validate semantic version bump
VALID=false
BUMP_TYPE=""
# Check for major version bump (X.0.0)
if [ "$PR_MAJOR" -eq $((MAIN_MAJOR + 1)) ] && [ "$PR_MINOR" -eq 0 ] && [ "$PR_PATCH" -eq 0 ]; then
VALID=true
BUMP_TYPE="major"
# Check for minor version bump (x.X.0)
elif [ "$PR_MAJOR" -eq "$MAIN_MAJOR" ] && [ "$PR_MINOR" -eq $((MAIN_MINOR + 1)) ] && [ "$PR_PATCH" -eq 0 ]; then
VALID=true
BUMP_TYPE="minor"
# Check for patch version bump (x.x.X)
elif [ "$PR_MAJOR" -eq "$MAIN_MAJOR" ] && [ "$PR_MINOR" -eq "$MAIN_MINOR" ] && [ "$PR_PATCH" -eq $((MAIN_PATCH + 1)) ]; then
VALID=true
BUMP_TYPE="patch"
fi
if [ "$VALID" = true ]; then
echo "✅ Valid $BUMP_TYPE version bump: $MAIN_VERSION → $PR_VERSION"
else
echo "❌ ERROR: Invalid semantic version change!"
echo ""
echo "Current version: $MAIN_VERSION"
echo "PR version: $PR_VERSION"
echo ""
echo "Valid version bumps:"
echo " - Patch: $MAIN_MAJOR.$MAIN_MINOR.$((MAIN_PATCH + 1))"
echo " - Minor: $MAIN_MAJOR.$((MAIN_MINOR + 1)).0"
echo " - Major: $((MAIN_MAJOR + 1)).0.0"
echo ""
echo "Common issues:"
echo " ❌ Skipping versions (e.g., 2.5.0 → 2.7.0)"
echo " ❌ Going backwards (e.g., 2.5.0 → 2.4.0)"
echo " ❌ Not resetting lower components (e.g., 2.5.0 → 2.6.1)"
exit 1
fi
# ==============================================
# 3. DOCKER BUILDX SETUP
# ==============================================
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# ==============================================
# 4. DOCKER HUB LOGIN (main branch only)
# ==============================================
# Requires secrets in Gitea:
# - DOCKERHUB_USERNAME: Your Docker Hub username
# - DOCKERHUB_TOKEN: Docker Hub access token (not password!)
#
# To create token:
# 1. Go to hub.docker.com
# 2. Account Settings → Security → New Access Token
# 3. Copy token to Gitea repo → Settings → Secrets → Actions
#
- name: Login to Docker Hub
if: github.ref == 'refs/heads/main'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
# ==============================================
# 5. EXTRACT METADATA
# ==============================================
# Reads VERSION file and generates image tags:
# - version: From VERSION file (e.g., "1.2.3")
# - sha_short: First 7 chars of commit SHA
# - version_sha: Combined version+commit (e.g., "v1.2.3-a1b2c3d")
# - branch: Current branch name
# - timestamp: ISO 8601 format for Discord
#
- name: Extract metadata
id: meta
run: |
VERSION=$(cat VERSION 2>/dev/null || echo "0.0.0")
SHA_SHORT=$(echo ${{ github.sha }} | cut -c1-7)
echo "version=${VERSION}" >> $GITHUB_OUTPUT
echo "sha_short=${SHA_SHORT}" >> $GITHUB_OUTPUT
echo "version_sha=v${VERSION}-${SHA_SHORT}" >> $GITHUB_OUTPUT
echo "branch=${{ github.ref_name }}" >> $GITHUB_OUTPUT
echo "timestamp=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_OUTPUT
# ==============================================
# 6. BUILD AND PUSH DOCKER IMAGE
# ==============================================
# Creates 3 tags for each build:
# - latest: Always points to newest build
# - v{VERSION}: Semantic version from VERSION file
# - v{VERSION}-{COMMIT}: Version + commit hash for traceability
#
# Example tags:
# - yourusername/yourrepo:latest
# - yourusername/yourrepo:v1.2.3
# - yourusername/yourrepo:v1.2.3-a1b2c3d
#
# Push behavior:
# - PRs: Build only (test), don't push
# - Main: Build and push to Docker Hub
#
# CUSTOMIZE: Replace "yourusername/yourrepo" with your Docker Hub repo
#
- name: Build Docker image
uses: docker/build-push-action@v5
with:
context: .
push: ${{ github.ref == 'refs/heads/main' }}
tags: |
yourusername/yourrepo:latest
yourusername/yourrepo:v${{ steps.meta.outputs.version }}
yourusername/yourrepo:${{ steps.meta.outputs.version_sha }}
cache-from: type=gha
cache-to: type=gha,mode=max
# ==============================================
# 7. BUILD SUMMARY
# ==============================================
# Creates a formatted summary visible in Actions UI
# Shows: image tags, build details, push status
#
- name: Build Summary
run: |
echo "## 🐳 Docker Build Successful! ✅" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Image Tags:**" >> $GITHUB_STEP_SUMMARY
echo "- \`yourusername/yourrepo:latest\`" >> $GITHUB_STEP_SUMMARY
echo "- \`yourusername/yourrepo:v${{ steps.meta.outputs.version }}\`" >> $GITHUB_STEP_SUMMARY
echo "- \`yourusername/yourrepo:${{ steps.meta.outputs.version_sha }}\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "**Build Details:**" >> $GITHUB_STEP_SUMMARY
echo "- Branch: \`${{ steps.meta.outputs.branch }}\`" >> $GITHUB_STEP_SUMMARY
echo "- Commit: \`${{ github.sha }}\`" >> $GITHUB_STEP_SUMMARY
echo "- Timestamp: \`${{ steps.meta.outputs.timestamp }}\`" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
if [ "${{ github.ref }}" == "refs/heads/main" ]; then
echo "🚀 **Pushed to Docker Hub!**" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
echo "Pull with: \`docker pull yourusername/yourrepo:latest\`" >> $GITHUB_STEP_SUMMARY
else
echo "_PR build - image not pushed to Docker Hub_" >> $GITHUB_STEP_SUMMARY
fi
# ==============================================
# 8. DISCORD NOTIFICATION - SUCCESS
# ==============================================
# Sends green embed to Discord on successful builds
#
# Only fires on main branch pushes (not PRs)
#
# Setup:
# 1. Create webhook in Discord channel:
# Right-click channel → Edit → Integrations → Webhooks → New
# 2. Copy webhook URL
# 3. Replace the URL below
#
# CUSTOMIZE:
# - Replace webhook URL with yours
# - Replace "Your Project" in title
# - Replace Docker Hub URLs with your repo
#
- name: Discord Notification - Success
if: success() && github.ref == 'refs/heads/main'
run: |
curl -H "Content-Type: application/json" \
-d '{
"embeds": [{
"title": "✅ Your Project Build Successful",
"description": "Docker image built and pushed to Docker Hub!",
"color": 3066993,
"fields": [
{
"name": "Version",
"value": "`v${{ steps.meta.outputs.version }}`",
"inline": true
},
{
"name": "Image Tag",
"value": "`${{ steps.meta.outputs.version_sha }}`",
"inline": true
},
{
"name": "Branch",
"value": "`${{ steps.meta.outputs.branch }}`",
"inline": true
},
{
"name": "Commit",
"value": "`${{ steps.meta.outputs.sha_short }}`",
"inline": true
},
{
"name": "Author",
"value": "${{ github.actor }}",
"inline": true
},
{
"name": "Docker Hub",
"value": "[yourusername/yourrepo](https://hub.docker.com/r/yourusername/yourrepo)",
"inline": false
},
{
"name": "View Run",
"value": "[Click here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})",
"inline": false
}
],
"timestamp": "${{ steps.meta.outputs.timestamp }}"
}]
}' \
YOUR_DISCORD_WEBHOOK_URL_HERE
# ==============================================
# 9. DISCORD NOTIFICATION - FAILURE
# ==============================================
# Sends red embed to Discord on build failures
#
# Only fires on main branch pushes (not PRs)
#
# CUSTOMIZE:
# - Replace webhook URL with yours
# - Replace "Your Project" in title
#
- name: Discord Notification - Failure
if: failure() && github.ref == 'refs/heads/main'
run: |
TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
curl -H "Content-Type: application/json" \
-d '{
"embeds": [{
"title": "❌ Your Project Build Failed",
"description": "Docker build encountered an error.",
"color": 15158332,
"fields": [
{
"name": "Branch",
"value": "`${{ github.ref_name }}`",
"inline": true
},
{
"name": "Commit",
"value": "`${{ github.sha }}`",
"inline": true
},
{
"name": "Author",
"value": "${{ github.actor }}",
"inline": true
},
{
"name": "View Logs",
"value": "[Click here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})",
"inline": false
}
],
"timestamp": "'"$TIMESTAMP"'"
}]
}' \
YOUR_DISCORD_WEBHOOK_URL_HERE
# ==============================================
# CUSTOMIZATION CHECKLIST
# ==============================================
# Before using this template in a new project:
#
# ✅ Replace "yourusername/yourrepo" with your Docker Hub repository
# ✅ Replace "Your Project" in Discord notification titles
# ✅ Replace Discord webhook URLs (both success and failure)
# ✅ Add secrets to Gitea repo: DOCKERHUB_USERNAME, DOCKERHUB_TOKEN
# ✅ Create VERSION file in repo root with initial version (e.g., "1.0.0")
# ✅ Update branch name if not using "main"
#
# Optional customizations:
# - Adjust runner labels (runs-on) if using self-hosted runners
# - Modify version validation rules if you don't want strict semver
# - Add additional notification channels (Slack, email, etc.)
# - Add deployment steps after Docker push
# - Customize Discord embed colors, fields, or formatting
#
# ==============================================
# TROUBLESHOOTING
# ==============================================
# Common issues and solutions:
#
# 1. VERSION validation failing unexpectedly
# - Ensure VERSION file exists in repo root
# - Check file contains only version number (no 'v' prefix or extra text)
# - Verify version follows semver: MAJOR.MINOR.PATCH
#
# 2. Docker Hub push failing
# - Verify DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets are set
# - Check Docker Hub token has push permissions
# - Ensure repository name matches your Docker Hub repo exactly
#
# 3. Discord notifications not appearing
# - Test webhook URL manually with curl
# - Check webhook still exists in Discord channel settings
# - Verify timestamp format is ISO 8601 (YYYY-MM-DDTHH:MM:SSZ)
# - Look for HTTP error codes in Actions logs
#
# 4. Build cache not working
# - GitHub Actions cache is stored per repository
# - Cache is shared across branches
# - May need to clear cache if corrupted
#
# ==============================================

167
server-configs/gitea/workflow-templates/snippets/auto-deploy-with-rollback.yml Normal file
View File

@ -0,0 +1,167 @@
# ==============================================
# SAFE AUTO-DEPLOY WITH HEALTH CHECK & ROLLBACK
# ==============================================
# Enhanced deployment with safety features:
# - Health check after deployment
# - Automatic rollback on failure
# - Deployment notifications
# - Downtime tracking
#
- name: Deploy to Production (Safe)
if: success() && github.ref == 'refs/heads/main'
run: |
# Set up SSH
mkdir -p ~/.ssh
echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/deploy_key
chmod 600 ~/.ssh/deploy_key
ssh-keyscan -H ${{ secrets.PRODUCTION_HOST }} >> ~/.ssh/known_hosts
echo "🚀 Deploying Paper Dynasty v${{ steps.meta.outputs.version }} to production..."
DEPLOY_START=$(date +%s)
# Deploy with health check and rollback
ssh -i ~/.ssh/deploy_key ${{ secrets.DEPLOY_USER }}@${{ secrets.PRODUCTION_HOST }} bash << 'EOF'
set -e
cd /home/cal/container-data/paper-dynasty
# Save current image tag for rollback
CURRENT_IMAGE=$(docker compose images -q paper-dynasty 2>/dev/null || echo "none")
echo "Current image: $CURRENT_IMAGE"
# Pull new image
echo "📥 Pulling new image..."
docker compose pull
# Stop old container
echo "🛑 Stopping old container..."
docker compose down
# Start new container
echo "▶️ Starting new container..."
docker compose up -d
# Health check with retry
echo "🏥 Running health check..."
for i in {1..10}; do
sleep 3
if docker compose ps | grep -q "Up"; then
echo "✅ Container is up!"
# Additional health check: check bot is responding
# Adjust this based on your bot's health endpoint
# if curl -f http://localhost:YOUR_PORT/health; then
# echo "✅ Health check passed!"
# exit 0
# fi
exit 0
fi
echo "Waiting for container... ($i/10)"
done
# If we get here, deployment failed
echo "❌ Health check failed! Rolling back..."
# Rollback to previous image
if [ "$CURRENT_IMAGE" != "none" ]; then
docker compose down
# This assumes you have the old image still cached
# In production, you might want to keep the last N images
docker compose up -d
echo "⏪ Rolled back to previous version"
exit 1
else
echo "⚠️ No previous image to rollback to!"
exit 1
fi
EOF
DEPLOY_STATUS=$?
DEPLOY_END=$(date +%s)
DEPLOY_TIME=$((DEPLOY_END - DEPLOY_START))
if [ $DEPLOY_STATUS -eq 0 ]; then
echo "✅ Deployment successful! (${DEPLOY_TIME}s)"
else
echo "❌ Deployment failed after ${DEPLOY_TIME}s"
exit 1
fi
# ==============================================
# DEPLOYMENT NOTIFICATION - SUCCESS
# ==============================================
- name: Discord Notification - Deployed Successfully
if: success() && github.ref == 'refs/heads/main'
run: |
curl -H "Content-Type: application/json" \
-d '{
"embeds": [{
"title": "🚀 Paper Dynasty Deployed to Production",
"description": "New version is live and healthy!",
"color": 5793266,
"fields": [
{
"name": "Version",
"value": "`v${{ steps.meta.outputs.version }}`",
"inline": true
},
{
"name": "Deployed By",
"value": "${{ github.actor }}",
"inline": true
},
{
"name": "Server",
"value": "sba-bots (10.10.0.88)",
"inline": true
},
{
"name": "Status",
"value": "✅ Health check passed",
"inline": false
}
],
"timestamp": "${{ steps.meta.outputs.timestamp }}"
}]
}' \
https://discord.com/api/webhooks/YOUR_WEBHOOK_URL
# ==============================================
# DEPLOYMENT NOTIFICATION - FAILED/ROLLED BACK
# ==============================================
- name: Discord Notification - Deployment Failed
if: failure() && github.ref == 'refs/heads/main'
run: |
TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ)
curl -H "Content-Type: application/json" \
-d '{
"embeds": [{
"title": "⚠️ Paper Dynasty Deployment Failed",
"description": "Deployment failed and was rolled back to previous version.",
"color": 16776960,
"fields": [
{
"name": "Attempted Version",
"value": "`v${{ steps.meta.outputs.version }}`",
"inline": true
},
{
"name": "Author",
"value": "${{ github.actor }}",
"inline": true
},
{
"name": "Action",
"value": "🔄 Rolled back to previous version",
"inline": false
},
{
"name": "View Logs",
"value": "[Click here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})",
"inline": false
}
],
"timestamp": "'"$TIMESTAMP"'"
}]
}' \
https://discord.com/api/webhooks/YOUR_WEBHOOK_URL

18
server-configs/hosts.yml
View File

@ -110,6 +110,7 @@ hosts:
services:
- n8n
- omni-tools
- termix
# Foundry VTT LXC (Proxmox)
foundry-lxc:
@ -187,6 +188,23 @@ hosts:
- major-domo-dev
- strat-gameplay-webapp
# NPM + Pi-hole Host
npm-pihole:
type: docker
ssh_alias: npm-pihole
ip: 10.10.0.16
user: cal
description: "Nginx Proxy Manager + Pi-hole DNS server"
config_paths:
scripts: /home/cal/scripts
services:
- nginx-proxy-manager
- pihole
documentation: server-configs/networking/nginx-proxy-manager-pihole.md
automation:
dns_sync: /home/cal/scripts/npm-pihole-sync.sh
cron: "0 * * * *" # Hourly
# Akamai Cloud Server
akamai:
type: docker

302
server-configs/networking/nginx-proxy-manager-pihole.md Normal file
View File

@ -0,0 +1,302 @@
# Nginx Proxy Manager + Pi-hole Setup
**Host**: 10.10.0.16
**Services**: Nginx Proxy Manager, Pi-hole DNS
This host runs both NPM (reverse proxy) and Pi-hole (DNS server) as Docker containers.
## Quick Info
| Property | Value |
|----------|-------|
| **IP** | 10.10.0.16 |
| **User** | cal |
| **NPM Container** | nginx-proxy-manager_app_1 |
| **Pi-hole Container** | pihole |
| **Pi-hole Web** | http://10.10.0.16/admin |
## Services
### Nginx Proxy Manager
- **Container**: nginx-proxy-manager_app_1
- **Web UI**: http://10.10.0.16:81
- **Ports**: 80, 443, 81
- **Database**: SQLite at `/data/database.sqlite`
- **Config**: `/data/nginx/`
- **Logs**: `/data/logs/`
### Pi-hole
- **Container**: pihole
- **Web UI**: http://10.10.0.16/admin
- **Ports**: 53 (DNS), 80 (Web)
- **Config**: `/etc/pihole/`
- **Custom DNS**: `/etc/pihole/custom.list`
## Architecture
```
┌─────────────┐
│ Clients │
│ (Homelab) │
└──────┬──────┘
│ DNS Query: termix.manticorum.com?
▼
┌─────────────────────┐
│ Pi-hole (10.10.0.16)│
│ Returns: 10.10.0.16 │ ← Local DNS override
└──────┬──────────────┘
│ HTTPS Request
▼
┌──────────────────────────┐
│ NPM (10.10.0.16) │
│ - Checks access list │ ← Sees real client IP (10.0.0.206)
│ - Terminates SSL │
│ - Proxies to backend │
└──────┬───────────────────┘
│ HTTP Proxy
▼
┌──────────────────────┐
│ Backend Service │
│ (e.g., Termix) │
│ 10.10.0.210:8180 │
└──────────────────────┘
```
## DNS Sync Automation
### Problem Solved
When accessing homelab services by domain name (e.g., `termix.manticorum.com`), DNS would resolve to Cloudflare IPs, causing traffic to hairpin through the internet. This broke IP-based access lists since NPM would see your public IP instead of your internal IP.
### Solution
Automatically sync all NPM proxy hosts to Pi-hole's local DNS, pointing them to NPM's internal IP (10.10.0.16).
### Sync Script
**Location**: `/home/cal/scripts/npm-pihole-sync.sh` (on 10.10.0.16)
**Repository**: `server-configs/networking/scripts/npm-pihole-sync.sh`
The script:
1. Queries NPM's SQLite database for all enabled proxy hosts
2. Extracts domain names
3. Creates Pi-hole custom DNS entries pointing to NPM (10.10.0.16)
4. Reloads Pi-hole DNS
**Usage:**
```bash
# Dry run (preview changes)
ssh cal@10.10.0.16 /home/cal/scripts/npm-pihole-sync.sh --dry-run
# Apply changes
ssh cal@10.10.0.16 /home/cal/scripts/npm-pihole-sync.sh
```
**Automation:**
- Runs hourly via cron: `0 * * * * /home/cal/scripts/npm-pihole-sync.sh >> /home/cal/logs/npm-pihole-sync.log 2>&1`
- Logs: `/home/cal/logs/npm-pihole-sync.log`
**Deployment:**
```bash
# Copy script to server
scp server-configs/networking/scripts/npm-pihole-sync.sh cal@10.10.0.16:/home/cal/scripts/
# Make executable
ssh cal@10.10.0.16 "chmod +x /home/cal/scripts/npm-pihole-sync.sh"
# Test
ssh cal@10.10.0.16 "/home/cal/scripts/npm-pihole-sync.sh --dry-run"
# Apply
ssh cal@10.10.0.16 "/home/cal/scripts/npm-pihole-sync.sh"
```
## Cloudflare Real IP Configuration
To support external access through Cloudflare while maintaining proper IP detection for access lists, NPM is configured to trust Cloudflare's IP ranges and read the real client IP from the `CF-Connecting-IP` header.
**Configuration** (in `/etc/nginx/nginx.conf` inside NPM container):
```nginx
# Trust internal networks
set_real_ip_from 10.0.0.0/8;
set_real_ip_from 172.16.0.0/12;
set_real_ip_from 192.168.0.0/16;
# Trust Cloudflare IPs
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 104.16.0.0/13;
set_real_ip_from 104.24.0.0/14;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 131.0.72.0/22;
# Read real IP from Cloudflare header
real_ip_header CF-Connecting-IP;
real_ip_recursive on;
```
This ensures:
- **Local access**: Client IP seen correctly (e.g., 10.0.0.206)
- **External access via Cloudflare**: Real public IP seen (not Cloudflare's proxy IP)
## Access Lists
NPM supports IP-based access restrictions. Common patterns:
### Internal Only
Allows access only from local networks:
```
allow 10.0.0.0/24;
allow 10.10.0.0/24;
deny all;
```
Use this for:
- Internal services (Termix, n8n, OmniTools)
- Admin interfaces
- Development environments
### Cloudflare Only
Trusts Cloudflare's authenticated access (Cloudflare Access/Zero Trust):
- No IP restrictions needed
- Cloudflare handles authentication
- Use for public services behind Cloudflare Access
## Management
### Access Containers
```bash
# SSH to host
ssh cal@10.10.0.16
# Access NPM container
docker exec -it nginx-proxy-manager_app_1 bash
# Access Pi-hole container
docker exec -it pihole bash
```
### NPM Commands
```bash
# View proxy host configs
docker exec nginx-proxy-manager_app_1 ls /data/nginx/proxy_host/
# View specific proxy host
docker exec nginx-proxy-manager_app_1 cat /data/nginx/proxy_host/21.conf
# Test nginx config
docker exec nginx-proxy-manager_app_1 nginx -t
# Reload nginx
docker exec nginx-proxy-manager_app_1 nginx -s reload
# View access logs
docker exec nginx-proxy-manager_app_1 tail -f /data/logs/proxy-host-21_access.log
```
### Pi-hole Commands
```bash
# View custom DNS
docker exec pihole cat /etc/pihole/custom.list
# Restart DNS
docker exec pihole pihole restartdns reload
# View logs
docker logs pihole -f
# Pi-hole status
docker exec pihole pihole status
```
### Query NPM Database
```bash
# List all proxy hosts
docker exec nginx-proxy-manager_app_1 python3 -c "
import sqlite3
conn = sqlite3.connect('/data/database.sqlite')
cursor = conn.cursor()
cursor.execute('SELECT id, domain_names, forward_host, forward_port FROM proxy_host')
for row in cursor.fetchall():
print(row)
conn.close()
"
```
## Troubleshooting
### 403 Forbidden Errors
**Symptom**: Getting 403 errors when accessing services with "Internal Only" access list.
**Causes**:
1. **Hairpinning through Cloudflare**: DNS resolving to Cloudflare IPs instead of local
- **Fix**: Ensure Pi-hole has local DNS override (run sync script)
- **Check**: `dig +short service.manticorum.com @10.10.0.16` should return `10.10.0.16`
2. **Access list doesn't include your subnet**: Your IP not in allowed ranges
- **Fix**: Add your subnet to access list (e.g., `10.0.0.0/24`)
- **Check**: View access logs to see what IP NPM is seeing
3. **Real IP not detected**: NPM seeing proxy/Cloudflare IP instead of client IP
- **Fix**: Ensure Cloudflare IPs are in `set_real_ip_from` and `real_ip_header CF-Connecting-IP` is set
- **Check**: Access logs should show your real IP, not Cloudflare's
**Debug Steps**:
```bash
# Check what IP NPM sees
docker exec nginx-proxy-manager_app_1 tail -20 /data/logs/proxy-host-21_access.log
# Check DNS resolution
dig +short termix.manticorum.com @10.10.0.16
# Check access list in proxy host config
docker exec nginx-proxy-manager_app_1 cat /data/nginx/proxy_host/21.conf | grep -A 10 "Access Rules"
```
### DNS Not Resolving Locally
**Symptom**: Domains still resolve to Cloudflare IPs.
**Causes**:
1. DNS sync not run or failed
2. Desktop not using Pi-hole for DNS
3. DNS cache on desktop
**Fix**:
```bash
# Re-run sync
ssh cal@10.10.0.16 /home/cal/scripts/npm-pihole-sync.sh
# Check Pi-hole has the record
ssh cal@10.10.0.16 "docker exec pihole cat /etc/pihole/custom.list | grep termix"
# Verify DNS server on desktop
cat /etc/resolv.conf # Should show 10.10.0.16
# Flush DNS cache on desktop
sudo resolvectl flush-caches # SystemD
# or
sudo systemd-resolve --flush-caches
```
## Related Documentation
- [Gitea Server](../gitea/README.md) - Uses NPM proxy
- [n8n Automation](../../productivity/n8n/CONTEXT.md) - Uses NPM proxy
- [Akamai NPM](../akamai/docker-compose/nginx-proxy-manager/) - Public-facing NPM instance
## Deployment Date
**Created**: 2026-02-04
**By**: Claude Code
**NPM Version**: (check with `docker exec nginx-proxy-manager_app_1 cat /app/package.json | grep version`)
**Pi-hole Version**: (check with `docker exec pihole pihole -v`)

74
server-configs/networking/scripts/npm-pihole-sync.sh Executable file
View File

@ -0,0 +1,74 @@
#!/bin/bash
# NPM to Pi-hole DNS Sync
# Syncs Nginx Proxy Manager proxy hosts to Pi-hole local DNS
# All domains point to NPM's IP, not the forward destination
set -e
DRY_RUN=false
if [[ "$1" == "--dry-run" ]]; then
DRY_RUN=true
fi
# NPM's IP address (where all domains should point)
NPM_IP="10.10.0.16"
echo "NPM → Pi-hole DNS Sync"
echo "============================================================"
# Query NPM database for all enabled proxy hosts
DOMAINS=$(docker exec nginx-proxy-manager_app_1 python3 -c '
import sqlite3
import json
conn = sqlite3.connect("/data/database.sqlite")
cursor = conn.cursor()
cursor.execute("SELECT domain_names FROM proxy_host WHERE enabled = 1")
domains = []
for (domain_names,) in cursor.fetchall():
for domain in json.loads(domain_names or "[]"):
domains.append(domain)
for domain in sorted(domains):
print(domain)
conn.close()
')
# Count records
RECORD_COUNT=$(echo "$DOMAINS" | wc -l)
echo "Found $RECORD_COUNT enabled proxy hosts"
echo ""
echo "All domains will point to NPM at: $NPM_IP"
echo ""
echo "Domains to sync:"
echo "$DOMAINS" | awk -v ip="$NPM_IP" '{printf " %-15s %s\n", ip, $0}'
if [ "$DRY_RUN" = true ]; then
echo ""
echo "[DRY RUN] Not applying changes"
exit 0
fi
# Build new custom.list
NEW_DNS="# Pi-hole Local DNS Records
# Auto-synced from Nginx Proxy Manager
# All domains point to NPM at $NPM_IP
"
while IFS= read -r domain; do
NEW_DNS+="$NPM_IP $domain"$'\n'
done <<< "$DOMAINS"
# Write to Pi-hole
echo "$NEW_DNS" | docker exec -i pihole tee /etc/pihole/custom.list > /dev/null
# Reload Pi-hole DNS
docker exec pihole pihole restartdns reload > /dev/null
echo ""
echo "✓ Updated $RECORD_COUNT DNS records in Pi-hole"
echo "✓ All domains now point to NPM at $NPM_IP"
echo "✓ Reloaded Pi-hole DNS"