diff --git a/server-configs/gitea/INDEX.md b/server-configs/gitea/INDEX.md new file mode 100644 index 0000000..ce02385 --- /dev/null +++ b/server-configs/gitea/INDEX.md @@ -0,0 +1,221 @@ +# Gitea Configuration & Templates Index + +Quick reference for all Gitea-related documentation and templates. + +## 📁 Directory Structure + +``` +/server-configs/gitea/ +├── README.md # Gitea LXC setup and configuration +├── INDEX.md # This file - quick reference +├── deployment-strategies.md # When/how to deploy (manual vs auto) +├── harbor-registry-setup.md # Self-hosted Docker registry guide +├── apply_branch_protection.py # Script for branch protection rules +└── workflow-templates/ # Reusable CI/CD templates + ├── README.md # Template usage guide + ├── docker-build-template.yml # Complete Docker CI/CD workflow + ├── deploy-script-template.sh # Safe manual deployment script + └── snippets/ # Workflow code snippets + └── auto-deploy-with-rollback.yml +``` + +## 🚀 Quick Links + +### Getting Started +- **New to Gitea Actions?** → Start with `workflow-templates/README.md` +- **Need a workflow?** → Copy `docker-build-template.yml` +- **Want to deploy?** → Read `deployment-strategies.md` + +### Templates + +#### Docker Build Workflow +**File:** `workflow-templates/docker-build-template.yml` + +**Features:** +- ✅ Semantic version validation +- ✅ Docker build + push to Docker Hub +- ✅ Discord notifications +- ✅ Build caching +- ✅ Multi-tag strategy + +**Use for:** +- Discord bots +- Web applications +- API services +- Any Dockerized project + +#### Manual Deploy Script +**File:** `workflow-templates/deploy-script-template.sh` + +**Features:** +- ✅ Pre-deployment checks +- ✅ Confirmation prompts +- ✅ Status verification +- ✅ Log viewing +- ✅ Rollback instructions + +**Use when:** +- You want manual control +- Deploying to production +- Testing new deployments + +#### Auto-Deploy with Rollback +**File:** `workflow-templates/snippets/auto-deploy-with-rollback.yml` + +**Features:** +- ✅ SSH deployment +- ✅ Health checks +- ✅ Automatic rollback +- ✅ Deployment notifications + +**Use when:** +- You have good test coverage +- Health checks are implemented +- Downtime is acceptable + +### Guides + +#### Deployment Strategies +**File:** `deployment-strategies.md` + +**Covers:** +- Decision framework (when to auto-deploy) +- 5 levels of deployment automation +- Recommendations by project type +- Safety best practices +- Rollback procedures + +**Read this before:** Adding auto-deploy to any project + +#### Harbor Registry Setup +**File:** `harbor-registry-setup.md` + +**Covers:** +- Self-hosted Docker registry +- Complete Harbor installation +- Integration with Gitea Actions +- Vulnerability scanning +- Backup strategies + +**Use when:** +- You want private registries +- You hit Docker Hub rate limits +- You want full control +- Learning opportunity + +## 📝 Reference Implementations + +### Paper Dynasty Discord Bot +**Status:** ✅ Production +**Date:** 2026-02-04 + +**Setup:** +- Gitea Actions on LXC 225 +- Docker build + push to Docker Hub +- Discord notifications working +- Manual deployment to sba-bots (10.10.0.88) + +**Workflow:** Based on `docker-build-template.yml` + +**What worked:** +- Semantic version validation +- Multi-tag strategy (latest, version, version+commit) +- Discord webhooks with ISO 8601 timestamps +- GitHub Actions cache for faster builds + +**Lessons learned:** +- Timestamp format critical for Discord (ISO 8601) +- Health checks needed before auto-deploy +- Manual deploy preferred for production Discord bots + +## 🔧 Common Tasks + +### Create New Project Workflow +```bash +cd /path/to/your/repo +mkdir -p .gitea/workflows +cp /path/to/docker-build-template.yml .gitea/workflows/docker-build.yml + +# Customize: +# - Replace "yourusername/yourrepo" with your Docker Hub repo +# - Replace "Your Project" in notifications +# - Replace Discord webhook URLs +# - Add secrets: DOCKERHUB_USERNAME, DOCKERHUB_TOKEN +# - Create VERSION file: echo "1.0.0" > VERSION +``` + +### Add Auto-Deploy (When Ready) +```bash +# Copy auto-deploy snippet +cat workflow-templates/snippets/auto-deploy-with-rollback.yml + +# Add to your workflow after build step +# Configure secrets: DEPLOY_SSH_KEY, PRODUCTION_HOST, DEPLOY_USER +``` + +### Manual Deploy +```bash +# Copy deploy script to your repo +cp workflow-templates/deploy-script-template.sh /path/to/repo/deploy.sh +chmod +x /path/to/repo/deploy.sh + +# Customize server details in script +# Then use: ./deploy.sh v1.2.3 +``` + +## 🎓 Learning Path + +1. **Start:** Read `workflow-templates/README.md` +2. **Setup:** Deploy Gitea Actions runner (see `README.md`) +3. **First workflow:** Copy `docker-build-template.yml` +4. **Deploy:** Use `deploy-script-template.sh` +5. **Advanced:** Read `deployment-strategies.md` +6. **Optional:** Set up Harbor with `harbor-registry-setup.md` + +## 🆘 Troubleshooting + +### Version validation failing +- Check VERSION file format: just `1.2.3` +- Ensure semver rules followed +- See template comments for valid bumps + +### Docker Hub push failing +- Verify secrets set in Gitea +- Check Docker Hub token permissions +- Ensure repo name matches exactly + +### Discord notifications not appearing +- Verify webhook URL still valid +- Check timestamp format (ISO 8601) +- Test webhook manually with curl + +### More help +- Check specific template comments +- Read troubleshooting sections in guides +- Review Paper Dynasty as reference implementation + +## 📚 External Resources + +- [Gitea Actions Docs](https://docs.gitea.io/en-us/usage/actions/overview/) +- [Docker Build Push Action](https://github.com/docker/build-push-action) +- [Semantic Versioning](https://semver.org/) +- [Discord Webhooks](https://discord.com/developers/docs/resources/webhook) +- [Harbor Docs](https://goharbor.io/docs/) + +## 🔄 Maintenance + +**Update this index when:** +- Adding new templates +- Creating new guides +- Changing directory structure +- Adding reference implementations + +**Last updated:** 2026-02-04 + +--- + +**Quick search tips:** +- Need deployment guide? → `deployment-strategies.md` +- Need workflow template? → `workflow-templates/` +- Need self-hosted registry? → `harbor-registry-setup.md` +- Need deploy script? → `deploy-script-template.sh` diff --git a/server-configs/gitea/README.md b/server-configs/gitea/README.md index b072fcd..8cf4de0 100644 --- a/server-configs/gitea/README.md +++ b/server-configs/gitea/README.md @@ -304,3 +304,147 @@ This repository is mirrored on both GitHub and Gitea for redundancy: - **GitHub**: https://github.com/calcorum/claude-home - **Gitea**: https://git.manticorum.com/cal/claude-home +--- + +## Branch Protection Manager + +Automated script to apply consistent branch protection rules across all your Gitea repositories. + +### Features + +- Applies branch protection rules to all repositories for a user/organization +- Supports dry-run mode to preview changes +- Configurable protection rules via environment variables +- Handles updating existing protection rules +- Clear success/error reporting + +### Requirements + +```bash +pip install requests +``` + +### Configuration + +The script (`apply_branch_protection.py`) uses these settings: +- ✅ Disable direct pushes (force PR workflow) +- ✅ Require 1 approval before merge +- ✅ Restrict approvals to whitelisted users +- ✅ Dismiss stale approvals when new commits are pushed +- ✅ Enable status checks (for CI/CD) +- ✅ Restrict merging to whitelisted users +- ✅ Block merge on rejected reviews +- ✅ Block merge if pull request is outdated (critical for DB migrations) + +### Usage + +#### 1. Create a Gitea API Token + +1. Go to https://git.manticorum.com/user/settings/applications +2. Click "Generate New Token" +3. Give it a name (e.g., "Branch Protection Script") +4. Select permissions: `repo` (full control) +5. Click "Generate Token" +6. Copy the token (you won't see it again!) + +#### 2. Set Environment Variables + +```bash +export GITEA_TOKEN='your-api-token-here' +export GITEA_URL='https://git.manticorum.com' # Optional, defaults to this +export GITEA_OWNER='cal' # Optional, defaults to cal +``` + +#### 3. Run the Script + +**Dry run (preview changes without applying):** +```bash +cd /mnt/NV2/Development/claude-home/server-configs/gitea +python apply_branch_protection.py --dry-run +``` + +**Apply to all repositories:** +```bash +python apply_branch_protection.py +``` + +### Customizing Protection Rules + +Edit the `BranchProtectionConfig` section in `main()` to customize the rules: + +```python +config = BranchProtectionConfig( + branch_name="main", # Branch to protect + enable_push=False, # Disable direct pushes + required_approvals=1, # Number of required approvals + enable_approvals_whitelist=True, # Restrict who can approve + approvals_whitelist_usernames=[GITEA_OWNER], + dismiss_stale_approvals=True, # Dismiss approvals on new commits + enable_status_check=True, # Require status checks to pass + enable_merge_whitelist=True, # Restrict who can merge + merge_whitelist_usernames=[GITEA_OWNER], + block_on_rejected_reviews=True, # Block merge if reviews are rejected + block_on_outdated_branch=True, # Block merge if branch is outdated + require_signed_commits=False, # Require GPG signatures +) +``` + +### Example Output + +``` +============================================================ +Gitea Branch Protection Configuration +============================================================ +Gitea URL: https://git.manticorum.com +Owner: cal +Branch: main +============================================================ +Protection Rules: + â€ĸ Direct pushes: Disabled + â€ĸ Required approvals: 1 + â€ĸ Approvals whitelist: cal + â€ĸ Dismiss stale approvals: True + â€ĸ Status checks enabled: True + â€ĸ Merge whitelist: cal + â€ĸ Block on rejected reviews: True + â€ĸ Block on outdated branch: True + â€ĸ Require signed commits: False +============================================================ + +Applying branch protection to 5 repositories... + +đŸ“Ļ major-domo-database + 🔄 Updating existing protection... + ✅ Successfully applied branch protection +đŸ“Ļ major-domo-bot + ✅ Successfully applied branch protection +... +``` + +### Troubleshooting Branch Protection Script + +**"GITEA_TOKEN environment variable is required"** +- You need to set your API token. See usage step 2 above. + +**"Error fetching repositories"** +- Check that your `GITEA_URL` is correct +- Verify your API token has `repo` permissions +- Ensure the `GITEA_OWNER` username is correct + +**"Could not delete existing protection"** +- The script will still try to create the new protection +- If this fails, manually delete the old protection rule from Gitea web UI + +**"Error: 422 Unprocessable Entity"** +This usually means: +- The branch doesn't exist in the repository +- Invalid username in whitelist +- Conflicting protection rule settings + +### API References + +This script uses the Gitea API branch protection endpoints: +- [Gitea API Documentation](https://docs.gitea.com/api/) +- [Protected Branches](https://docs.gitea.com/usage/access-control/protected-branches) +- [Create Branch Protection Endpoint](https://share.apidog.com/apidoc/docs-site/346218/api-3521477) + diff --git a/server-configs/gitea/apply_branch_protection.py b/server-configs/gitea/apply_branch_protection.py new file mode 100755 index 0000000..00c8d79 --- /dev/null +++ b/server-configs/gitea/apply_branch_protection.py @@ -0,0 +1,299 @@ +#!/usr/bin/env python3 +""" +Apply branch protection rules to all Gitea repositories. + +This script uses the Gitea API to apply consistent branch protection rules +across all repositories owned by a user or organization. +""" + +import os +import sys +import requests +from typing import Optional +from dataclasses import dataclass + + +@dataclass +class BranchProtectionConfig: + """Configuration for branch protection rules.""" + + # Branch to protect + branch_name: str = "main" + + # Push settings + enable_push: bool = False # Disable direct pushes (force PR workflow) + enable_push_whitelist: bool = False + push_whitelist_usernames: list[str] = None + + # Pull request approval settings + required_approvals: int = 1 + enable_approvals_whitelist: bool = True + approvals_whitelist_usernames: list[str] = None + dismiss_stale_approvals: bool = True + + # Status check settings + enable_status_check: bool = True + status_check_contexts: list[str] = None # Empty for now, add when CI/CD is set up + + # Merge settings + enable_merge_whitelist: bool = True + merge_whitelist_usernames: list[str] = None + block_on_rejected_reviews: bool = True + block_on_outdated_branch: bool = True # Critical for database migrations + + # Other settings + require_signed_commits: bool = False + protected_file_patterns: str = "" + unprotected_file_patterns: str = "" + + def __post_init__(self): + """Initialize mutable default values.""" + if self.push_whitelist_usernames is None: + self.push_whitelist_usernames = [] + if self.approvals_whitelist_usernames is None: + self.approvals_whitelist_usernames = [] + if self.merge_whitelist_usernames is None: + self.merge_whitelist_usernames = [] + if self.status_check_contexts is None: + self.status_check_contexts = [] + + +class GiteaBranchProtectionManager: + """Manages branch protection rules across Gitea repositories.""" + + def __init__(self, gitea_url: str, api_token: str, owner: str): + """ + Initialize the manager. + + Args: + gitea_url: Base URL of your Gitea instance (e.g., https://git.example.com) + api_token: Gitea API token with repo access + owner: Username or organization name + """ + self.gitea_url = gitea_url.rstrip('/') + self.api_token = api_token + self.owner = owner + self.headers = { + 'Authorization': f'token {api_token}', + 'Content-Type': 'application/json' + } + + def get_repositories(self) -> list[dict]: + """Get all repositories for the authenticated user.""" + # Use /user/repos for authenticated requests (works with API token) + url = f'{self.gitea_url}/api/v1/user/repos' + params = {'limit': 100} # Adjust if you have more than 100 repos + + try: + response = requests.get(url, headers=self.headers, params=params) + response.raise_for_status() + repos = response.json() + # Filter to only repos owned by the specified owner if needed + if self.owner: + repos = [r for r in repos if r.get('owner', {}).get('login') == self.owner] + return repos + except requests.exceptions.RequestException as e: + print(f"❌ Error fetching repositories: {e}") + sys.exit(1) + + def get_existing_branch_protections(self, repo_name: str) -> list[dict]: + """Get existing branch protections for a repository.""" + url = f'{self.gitea_url}/api/v1/repos/{self.owner}/{repo_name}/branch_protections' + + try: + response = requests.get(url, headers=self.headers) + response.raise_for_status() + return response.json() + except requests.exceptions.RequestException as e: + print(f" âš ī¸ Could not fetch existing protections: {e}") + return [] + + def delete_branch_protection(self, repo_name: str, protection_name: str) -> bool: + """Delete an existing branch protection rule.""" + url = f'{self.gitea_url}/api/v1/repos/{self.owner}/{repo_name}/branch_protections/{protection_name}' + + try: + response = requests.delete(url, headers=self.headers) + response.raise_for_status() + return True + except requests.exceptions.RequestException as e: + print(f" âš ī¸ Could not delete existing protection: {e}") + return False + + def apply_branch_protection(self, repo_name: str, config: BranchProtectionConfig) -> bool: + """ + Apply branch protection rules to a repository. + + Args: + repo_name: Name of the repository + config: Branch protection configuration + + Returns: + True if successful, False otherwise + """ + url = f'{self.gitea_url}/api/v1/repos/{self.owner}/{repo_name}/branch_protections' + + payload = { + 'branch_name': config.branch_name, + 'enable_push': config.enable_push, + 'enable_push_whitelist': config.enable_push_whitelist, + 'push_whitelist_usernames': config.push_whitelist_usernames, + 'required_approvals': config.required_approvals, + 'enable_approvals_whitelist': config.enable_approvals_whitelist, + 'approvals_whitelist_usernames': config.approvals_whitelist_usernames, + 'dismiss_stale_approvals': config.dismiss_stale_approvals, + 'enable_status_check': config.enable_status_check, + 'status_check_contexts': config.status_check_contexts, + 'enable_merge_whitelist': config.enable_merge_whitelist, + 'merge_whitelist_usernames': config.merge_whitelist_usernames, + 'block_on_rejected_reviews': config.block_on_rejected_reviews, + 'block_on_outdated_branch': config.block_on_outdated_branch, + 'require_signed_commits': config.require_signed_commits, + 'protected_file_patterns': config.protected_file_patterns, + 'unprotected_file_patterns': config.unprotected_file_patterns, + } + + try: + response = requests.post(url, headers=self.headers, json=payload) + response.raise_for_status() + return True + except requests.exceptions.RequestException as e: + print(f" ❌ Error: {e}") + if hasattr(e.response, 'text'): + print(f" Response: {e.response.text}") + return False + + def apply_to_all_repos(self, config: BranchProtectionConfig, dry_run: bool = False): + """ + Apply branch protection rules to all repositories. + + Args: + config: Branch protection configuration + dry_run: If True, only show what would be done without making changes + """ + repos = self.get_repositories() + + if not repos: + print("No repositories found.") + return + + print(f"\n{'DRY RUN - ' if dry_run else ''}Applying branch protection to {len(repos)} repositories...\n") + + success_count = 0 + skip_count = 0 + error_count = 0 + + for repo in repos: + repo_name = repo['name'] + print(f"đŸ“Ļ {repo_name}") + + # Check if branch exists + default_branch = repo.get('default_branch', 'main') + if config.branch_name != default_branch: + print(f" â„šī¸ Default branch is '{default_branch}', protecting '{config.branch_name}' instead") + + if dry_run: + print(f" ✓ Would apply protection to '{config.branch_name}' branch") + skip_count += 1 + continue + + # Check for existing protection + existing_protections = self.get_existing_branch_protections(repo_name) + existing_main_protection = None + + for protection in existing_protections: + if protection.get('branch_name') == config.branch_name: + existing_main_protection = protection.get('id') or protection.get('branch_name') + break + + # Delete existing protection if it exists + if existing_main_protection: + print(f" 🔄 Updating existing protection...") + self.delete_branch_protection(repo_name, existing_main_protection) + + # Apply new protection + if self.apply_branch_protection(repo_name, config): + print(f" ✅ Successfully applied branch protection") + success_count += 1 + else: + print(f" ❌ Failed to apply branch protection") + error_count += 1 + + print(f"\n{'=' * 60}") + print(f"Summary:") + print(f" ✅ Successful: {success_count}") + if dry_run: + print(f" â­ī¸ Skipped (dry run): {skip_count}") + if error_count > 0: + print(f" ❌ Errors: {error_count}") + print(f"{'=' * 60}\n") + + +def main(): + """Main entry point.""" + # Configuration - adjust these values or use environment variables + GITEA_URL = os.getenv('GITEA_URL', 'https://git.manticorum.com') + GITEA_TOKEN = os.getenv('GITEA_TOKEN') + GITEA_OWNER = os.getenv('GITEA_OWNER', 'cal') + + # Check for required configuration + if not GITEA_TOKEN: + print("❌ Error: GITEA_TOKEN environment variable is required") + print("\nUsage:") + print(" export GITEA_TOKEN='your-api-token-here'") + print(" python apply_branch_protection.py [--dry-run]") + print("\nOptional environment variables:") + print(" GITEA_URL (default: https://git.manticorum.com)") + print(" GITEA_OWNER (default: cal)") + sys.exit(1) + + # Parse command line arguments + dry_run = '--dry-run' in sys.argv + + # Configure branch protection rules + # Adjust these settings as needed + config = BranchProtectionConfig( + branch_name="main", + enable_push=False, # Disable direct pushes + required_approvals=1, + enable_approvals_whitelist=True, + approvals_whitelist_usernames=[GITEA_OWNER], + dismiss_stale_approvals=True, + enable_status_check=True, + enable_merge_whitelist=True, + merge_whitelist_usernames=[GITEA_OWNER], + block_on_rejected_reviews=True, + block_on_outdated_branch=True, # Critical for DB repos + require_signed_commits=False, + ) + + # Display configuration + print(f"\n{'=' * 60}") + print(f"Gitea Branch Protection Configuration") + print(f"{'=' * 60}") + print(f"Gitea URL: {GITEA_URL}") + print(f"Owner: {GITEA_OWNER}") + print(f"Branch: {config.branch_name}") + print(f"{'=' * 60}") + print(f"Protection Rules:") + print(f" â€ĸ Direct pushes: {'Disabled' if not config.enable_push else 'Enabled'}") + print(f" â€ĸ Required approvals: {config.required_approvals}") + print(f" â€ĸ Approvals whitelist: {', '.join(config.approvals_whitelist_usernames)}") + print(f" â€ĸ Dismiss stale approvals: {config.dismiss_stale_approvals}") + print(f" â€ĸ Status checks enabled: {config.enable_status_check}") + print(f" â€ĸ Merge whitelist: {', '.join(config.merge_whitelist_usernames)}") + print(f" â€ĸ Block on rejected reviews: {config.block_on_rejected_reviews}") + print(f" â€ĸ Block on outdated branch: {config.block_on_outdated_branch}") + print(f" â€ĸ Require signed commits: {config.require_signed_commits}") + print(f"{'=' * 60}\n") + + if dry_run: + print("🔍 DRY RUN MODE - No changes will be made\n") + + # Create manager and apply rules + manager = GiteaBranchProtectionManager(GITEA_URL, GITEA_TOKEN, GITEA_OWNER) + manager.apply_to_all_repos(config, dry_run=dry_run) + + +if __name__ == '__main__': + main() diff --git a/server-configs/gitea/deployment-strategies.md b/server-configs/gitea/deployment-strategies.md new file mode 100644 index 0000000..025fdda --- /dev/null +++ b/server-configs/gitea/deployment-strategies.md @@ -0,0 +1,356 @@ +# Deployment Strategies for Gitea CI/CD + +Guide to choosing and implementing deployment automation for your projects. + +## đŸŽ¯ Decision Framework + +### Consider These Factors + +1. **Criticality**: How important is uptime? + - Discord bot for active league: HIGH + - Personal project: LOW + - Public API: VERY HIGH + +2. **Test Coverage**: How confident are you in your tests? + - Comprehensive unit + integration tests: AUTO-DEPLOY OK + - Some tests: SEMI-AUTO + - No tests: MANUAL ONLY + +3. **Rollback Complexity**: Can you easily revert? + - Docker with versioned images: EASY + - Database migrations: HARD + - Stateful services: MEDIUM + +4. **Traffic Patterns**: When can you deploy? + - 24/7 users: Need zero-downtime + - Business hours only: Deploy off-hours + - Low traffic: Restart acceptable + +## 📊 Deployment Options Comparison + +| Strategy | Speed | Safety | Complexity | Best For | +|----------|-------|--------|------------|----------| +| **Manual** | Slow | High | Low | Critical systems, low test coverage | +| **Semi-Auto** | Medium | High | Low | Most projects (recommended) | +| **Auto + Health** | Fast | Medium | Medium | Good test coverage, health endpoints | +| **Blue-Green** | Fast | High | High | Zero-downtime required | +| **Canary** | Fast | Very High | Very High | Large user base, gradual rollout | + +## 🚀 Recommended Progression + +### Level 1: Manual Deploy (Start Here) +```bash +# Simple, safe, controllable +ssh production +docker compose pull +docker compose up -d +``` + +**Pros:** +- ✅ Full control over timing +- ✅ Can test/verify first +- ✅ Simple to understand +- ✅ Easy to abort + +**Cons:** +- ❌ Requires manual SSH +- ❌ Inconsistent process +- ❌ Slower deployments + +### Level 2: Deploy Script (Easy Win) +```bash +# One command from local machine +./deploy.sh v1.8.1 +``` + +**Pros:** +- ✅ Consistent process +- ✅ Pre-deployment checks +- ✅ Still manual control +- ✅ Rollback instructions + +**Cons:** +- ❌ Still requires human +- ❌ Can't deploy from CI + +**Setup:** Copy `/tmp/deploy.sh` to your repo root + +### Level 3: CI Notifies, Manual Deploy +```yaml +# Workflow builds, then notifies: +- Discord: "v1.8.1 ready to deploy" +- You run: ./deploy.sh v1.8.1 +``` + +**Pros:** +- ✅ Fast build process +- ✅ Clear deployment signal +- ✅ Choose deployment timing +- ✅ Audit trail + +**Cons:** +- ❌ Two-step process +- ❌ Can forget to deploy + +**Current State:** This is where Paper Dynasty is now ✅ + +### Level 4: Auto-Deploy with Safeguards +```yaml +# Workflow builds, tests, deploys, health checks +- Build image +- Push to Docker Hub +- SSH to production +- Pull and restart +- Health check +- Rollback on failure +- Notify Discord +``` + +**Pros:** +- ✅ Fully automated +- ✅ Fast deployment +- ✅ Automatic rollback +- ✅ Consistent process + +**Cons:** +- ❌ Complexity +- ❌ Brief downtime +- ❌ Requires good health checks + +**Setup:** Use `/tmp/safe-auto-deploy-step.yml` + +### Level 5: Blue-Green Deployment +```yaml +# Run two environments, switch traffic +- Deploy to "green" environment +- Health check green +- Switch load balancer to green +- Keep "blue" as instant rollback +``` + +**Pros:** +- ✅ Zero downtime +- ✅ Instant rollback +- ✅ Test before switching +- ✅ Very safe + +**Cons:** +- ❌ Requires two environments +- ❌ Needs load balancer +- ❌ Database complexity +- ❌ 2x resources + +**Best For:** High-traffic web services + +## đŸŽ¯ Recommendations by Project Type + +### Discord Bot (Paper Dynasty) +**Recommended:** Level 3 (CI Notify + Manual Deploy) + +**Why?** +- Active users (league members) +- Brief downtime acceptable (1-2 min) +- Manual timing is valuable +- Can deploy during off-hours + +**Next Step:** Add deploy script for consistency + +### Internal Tool / Dashboard +**Recommended:** Level 4 (Auto-Deploy with Health Check) + +**Why?** +- Limited users +- Downtime less critical +- Faster iteration valuable +- Easy rollback available + +### Public API / High Traffic +**Recommended:** Level 5 (Blue-Green) or Canary + +**Why?** +- Zero downtime required +- Large user base +- Complex rollback scenarios +- Worth the complexity + +### Personal Project / Portfolio +**Recommended:** Level 2 or 3 + +**Why?** +- Simple is better +- Automation overhead not worth it +- Infrequent deploys +- Learning opportunity + +## đŸ› ī¸ Implementation Guide + +### Adding Auto-Deploy to Paper Dynasty + +If you decide to try auto-deploy: + +1. **Add deploy SSH key to secrets:** + ```bash + # Generate deploy-only SSH key + ssh-keygen -t ed25519 -f ~/.ssh/paper_dynasty_deploy -C "gitea-deploy" + + # Add public key to sba-bots + ssh-copy-id -i ~/.ssh/paper_dynasty_deploy.pub cal@10.10.0.88 + + # Add private key to Gitea secrets + cat ~/.ssh/paper_dynasty_deploy + # Copy to: Repo → Settings → Secrets → DEPLOY_SSH_KEY + ``` + +2. **Add production host secret:** + - Secret name: `PRODUCTION_HOST` + - Value: `10.10.0.88` + +3. **Add deploy user secret:** + - Secret name: `DEPLOY_USER` + - Value: `cal` + +4. **Add deploy step to workflow:** + - Copy from `/tmp/safe-auto-deploy-step.yml` + - Paste after "Build Docker image" step + - Update webhook URL + +5. **Test with a minor change:** + - Make a small PR (comment change) + - Merge and watch deployment + - Verify bot restarts successfully + - Check Discord for notifications + +6. **Monitor first few deployments:** + - Watch for issues + - Check logs: `ssh sba-bots 'docker compose logs'` + - Verify no errors + +### Adding Health Checks + +For better auto-deploy safety, add a health endpoint to your bot: + +```javascript +// Express health check endpoint +app.get('/health', (req, res) => { + // Check Discord connection + if (!client.ws.ping) { + return res.status(503).json({ status: 'unhealthy', reason: 'Discord disconnected' }); + } + + // Check database connection + if (!db.isConnected()) { + return res.status(503).json({ status: 'unhealthy', reason: 'Database disconnected' }); + } + + res.json({ + status: 'healthy', + version: process.env.VERSION, + uptime: process.uptime(), + discord: { + guilds: client.guilds.cache.size, + ping: client.ws.ping + } + }); +}); +``` + +Then health check in deploy script: +```bash +curl -f http://localhost:3000/health || rollback +``` + +## âš ī¸ Safety Best Practices + +### Before Auto-Deploying + +1. **✅ Add Comprehensive Tests** + - Unit tests for core logic + - Integration tests for Discord API + - Smoke tests for critical paths + +2. **✅ Implement Health Checks** + - HTTP endpoint for status + - Check all critical services + - Return proper status codes + +3. **✅ Set Up Monitoring** + - Error tracking (Sentry) + - Performance monitoring (Datadog) + - Discord webhook for errors + +4. **✅ Plan Rollback Process** + - Keep last 3 images cached + - Document rollback steps + - Test rollback procedure + +5. **✅ Document Deployment** + - What gets deployed + - How to abort + - How to rollback + - Who to contact + +### During Deployment + +1. **Monitor Logs** + ```bash + docker compose logs -f + ``` + +2. **Check Metrics** + - Response times + - Error rates + - Resource usage + +3. **Verify Functionality** + - Test critical commands + - Check database connections + - Verify integrations + +### After Deployment + +1. **Watch for Issues** + - First 5 minutes are critical + - Check error channels + - Monitor user reports + +2. **Validate Success** + - All services healthy + - No error spike + - Performance normal + +3. **Document Issues** + - What went wrong + - How you fixed it + - How to prevent + +## 🔄 Rollback Procedures + +### Quick Rollback (Docker) +```bash +ssh production +cd /path/to/app +docker compose down +docker tag yourusername/repo:current yourusername/repo:rollback-backup +docker pull yourusername/repo:v1.7.0 # Last known good +docker compose up -d +``` + +### Emergency Rollback (CI) +```yaml +# Manually trigger workflow with old version +git tag -f v1.7.0-redeploy +git push -f origin v1.7.0-redeploy +# Watch actions deploy old version +``` + +## 📚 Further Reading + +- [Deployment Strategies](https://www.redhat.com/en/topics/devops/deployment-strategies) +- [Blue-Green Deployments](https://martinfowler.com/bliki/BlueGreenDeployment.html) +- [Canary Releases](https://martinfowler.com/bliki/CanaryRelease.html) +- [Docker Health Checks](https://docs.docker.com/engine/reference/builder/#healthcheck) + +--- + +**Last Updated:** 2026-02-04 +**Applies To:** Paper Dynasty Discord bot, future Gitea CI/CD projects diff --git a/server-configs/gitea/harbor-registry-setup.md b/server-configs/gitea/harbor-registry-setup.md new file mode 100644 index 0000000..d2a6f33 --- /dev/null +++ b/server-configs/gitea/harbor-registry-setup.md @@ -0,0 +1,377 @@ +# Harbor Docker Registry Setup Guide + +Complete guide to setting up Harbor on a Proxmox LXC for self-hosted Docker registry. + +## Prerequisites + +- Proxmox LXC with Ubuntu 22.04 +- 2 CPU cores, 4GB RAM, 50GB disk +- Docker and docker-compose installed +- Domain name (e.g., registry.manticorum.com) + +## Quick Setup + +### 1. Create LXC Container + +```bash +# On Proxmox host +pct create 227 local:vztmpl/ubuntu-22.04-standard_22.04-1_amd64.tar.zst \ + --hostname harbor \ + --cores 2 \ + --memory 4096 \ + --swap 512 \ + --net0 name=eth0,bridge=vmbr0,ip=10.10.0.227/24,gw=10.10.0.1 \ + --rootfs local-lvm:50 \ + --unprivileged 1 \ + --features nesting=1 \ + --onboot 1 \ + --start 1 +``` + +### 2. Install Docker + +```bash +ssh root@10.10.0.227 + +apt update && apt install -y curl +curl -fsSL https://get.docker.com | sh +systemctl enable docker +``` + +### 3. Download Harbor + +```bash +cd /opt +wget https://github.com/goharbor/harbor/releases/download/v2.10.0/harbor-offline-installer-v2.10.0.tgz +tar xzvf harbor-offline-installer-v2.10.0.tgz +cd harbor +``` + +### 4. Configure Harbor + +```bash +cp harbor.yml.tmpl harbor.yml + +# Edit harbor.yml +nano harbor.yml +``` + +**Key settings to change:** +```yaml +hostname: registry.manticorum.com # Your domain + +# HTTPS (configure after NPM setup, start with HTTP for now) +# https: +# port: 443 +# certificate: /path/to/cert +# private_key: /path/to/key + +# Or disable HTTPS initially +# Comment out entire https section + +harbor_admin_password: YourSecurePassword123 + +database: + password: YourDBPassword123 + +data_volume: /mnt/harbor-data +``` + +### 5. Install Harbor + +```bash +./install.sh +``` + +### 6. Access Harbor + +Open: `http://10.10.0.227` (or `http://registry.manticorum.com` if DNS configured) + +**Default login:** +- Username: `admin` +- Password: `YourSecurePassword123` (what you set) + +### 7. Configure NPM Reverse Proxy + +In Nginx Proxy Manager (10.10.0.16): + +**Proxy Host:** +- Domain: `registry.manticorum.com` +- Scheme: `http` +- Forward Hostname: `10.10.0.227` +- Forward Port: `80` +- Websockets: ✅ Enabled +- Block Common Exploits: ✅ Enabled +- SSL: Let's Encrypt + +**Custom Nginx Configuration:** +```nginx +# Increase timeouts for large image uploads +proxy_read_timeout 900; +proxy_send_timeout 900; +client_max_body_size 0; # No upload limit + +# Required for Docker registry +proxy_set_header Host $host; +proxy_set_header X-Real-IP $remote_addr; +proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; +proxy_set_header X-Forwarded-Proto $scheme; +``` + +## Using Your Registry + +### 1. Login from Dev Machine + +```bash +docker login registry.manticorum.com +# Username: admin +# Password: YourSecurePassword123 +``` + +### 2. Tag and Push Image + +```bash +# Tag existing image +docker tag manticorum67/paper-dynasty:latest registry.manticorum.com/paper-dynasty/bot:latest + +# Push to your registry +docker push registry.manticorum.com/paper-dynasty/bot:latest +``` + +### 3. Pull from Production + +```bash +# On sba-bots +docker login registry.manticorum.com +docker pull registry.manticorum.com/paper-dynasty/bot:latest +``` + +### 4. Update docker-compose + +```yaml +services: + paper-dynasty: + # Old: image: manticorum67/paper-dynasty:latest + # New: + image: registry.manticorum.com/paper-dynasty/bot:latest +``` + +## Integrating with Gitea Actions + +Update your workflow to push to both registries: + +```yaml +- name: Login to Docker Hub + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + +- name: Login to Harbor + uses: docker/login-action@v3 + with: + registry: registry.manticorum.com + username: ${{ secrets.HARBOR_USERNAME }} + password: ${{ secrets.HARBOR_PASSWORD }} + +- name: Build and push + uses: docker/build-push-action@v5 + with: + context: . + push: ${{ github.ref == 'refs/heads/main' }} + tags: | + manticorum67/paper-dynasty:latest + manticorum67/paper-dynasty:v${{ steps.meta.outputs.version }} + registry.manticorum.com/paper-dynasty/bot:latest + registry.manticorum.com/paper-dynasty/bot:v${{ steps.meta.outputs.version }} +``` + +## Harbor Features + +### Create Projects + +1. Login to Harbor UI +2. Click **New Project** +3. Name: `paper-dynasty` +4. Access Level: Private or Public + +### Enable Vulnerability Scanning + +1. Go to **Administration** → **Interrogation Services** +2. Enable **Trivy** scanner +3. Set scan on push: ✅ Enabled + +Now images are auto-scanned for CVEs! + +### Set Up Replication + +Replicate between Harbor and Docker Hub: + +1. **Administration** → **Replications** +2. **New Replication Rule** + - Name: `sync-to-dockerhub` + - Source: Local + - Destination: Docker Hub (add endpoint first) + - Trigger: Event Based + +### Garbage Collection + +Free up disk space from deleted images: + +1. **Administration** → **Garbage Collection** +2. Schedule: Daily at 2 AM +3. Dry run first to see what would be deleted + +## Backup Strategy + +### What to Backup + +1. **Harbor database** (PostgreSQL) +2. **Image storage** (`/mnt/harbor-data`) +3. **Configuration** (`/opt/harbor/harbor.yml`) + +### Backup Script + +```bash +#!/bin/bash +BACKUP_DIR="/mnt/backups/harbor" +DATE=$(date +%Y%m%d) + +# Stop Harbor +cd /opt/harbor +docker-compose down + +# Backup database +docker exec harbor-db pg_dumpall -U postgres > $BACKUP_DIR/harbor-db-$DATE.sql + +# Backup data (incremental) +rsync -av /mnt/harbor-data/ $BACKUP_DIR/harbor-data/ + +# Backup config +cp /opt/harbor/harbor.yml $BACKUP_DIR/harbor-config-$DATE.yml + +# Start Harbor +docker-compose up -d + +# Keep last 7 days +find $BACKUP_DIR -name "harbor-db-*.sql" -mtime +7 -delete +``` + +## Monitoring + +### Check Harbor Status + +```bash +cd /opt/harbor +docker-compose ps + +# View logs +docker-compose logs -f +``` + +### Disk Usage + +```bash +du -sh /mnt/harbor-data + +# By project +du -sh /mnt/harbor-data/docker/registry/v2/repositories/* +``` + +### API Health Check + +```bash +curl -k https://registry.manticorum.com/api/v2.0/health +``` + +## Troubleshooting + +### "401 Unauthorized" on push + +**Problem:** Docker login not working + +**Solution:** +```bash +# Clear old credentials +rm ~/.docker/config.json + +# Login again +docker login registry.manticorum.com +``` + +### "413 Request Entity Too Large" + +**Problem:** Nginx upload limit + +**Solution:** Add to NPM custom config: +```nginx +client_max_body_size 0; +``` + +### Disk space full + +**Problem:** Old images filling disk + +**Solution:** +```bash +# Run garbage collection +cd /opt/harbor +docker-compose exec core /harbor/garbage-collection.sh + +# Or via UI: Administration → Garbage Collection → Run Now +``` + +### Can't pull from registry + +**Problem:** Firewall or network issue + +**Solution:** +```bash +# Test connection +telnet 10.10.0.227 80 + +# Check Harbor logs +docker-compose logs registry +``` + +## Advanced: High Availability + +For production-critical registries, set up HA: + +1. Multiple Harbor instances +2. Shared storage (NFS, S3, Minio) +3. Load balancer in front +4. Database replication + +## Cost Analysis + +**LXC Resources:** +- CPU: 2 cores = $0 (spare capacity) +- RAM: 4GB = $0 (spare capacity) +- Disk: 50GB = $0 (local storage) +- Bandwidth: Internal = $0 + +**Total ongoing cost: $0/month** + +**Docker Hub Pro alternative: $5/month** + +**Time investment:** +- Setup: 2-3 hours +- Maintenance: 30 min/month +- Break-even: 3 months of learning value + +## Next Steps + +1. ✅ Set up Harbor on LXC 227 +2. ✅ Configure NPM reverse proxy +3. ✅ Test push/pull from dev machine +4. ✅ Update one project to use Harbor +5. ✅ Set up Gitea Actions to push to both registries +6. ✅ Configure vulnerability scanning +7. ✅ Set up automated backups + +--- + +**Created:** 2026-02-04 +**For:** Manticorum Home Lab +**Reference:** Paper Dynasty as first use case diff --git a/server-configs/gitea/workflow-templates/README.md b/server-configs/gitea/workflow-templates/README.md new file mode 100644 index 0000000..e8de8aa --- /dev/null +++ b/server-configs/gitea/workflow-templates/README.md @@ -0,0 +1,221 @@ +# Gitea Actions Workflow Templates + +Reusable CI/CD workflow templates for Gitea Actions (GitHub Actions compatible). + +## Templates + +### `docker-build-template.yml` + +Complete Docker build pipeline with semantic versioning validation, Docker Hub push, and Discord notifications. + +**Features:** +- ✅ Semantic version validation on PRs +- ✅ Docker build on every push/PR +- ✅ Push to Docker Hub on main branch +- ✅ Discord notifications (success/failure) +- ✅ Build caching for faster builds +- ✅ Multi-tag strategy (latest, version, version+commit) + +**Reference Implementation:** +Paper Dynasty Discord bot - First production use (2026-02-04) + +## Quick Start + +1. **Copy template to your repo:** + ```bash + mkdir -p .gitea/workflows + cp docker-build-template.yml .gitea/workflows/docker-build.yml + ``` + +2. **Customize placeholders:** + - Replace `yourusername/yourrepo` with your Docker Hub repository + - Replace `Your Project` in notification titles + - Replace `YOUR_DISCORD_WEBHOOK_URL_HERE` with your webhook URLs + +3. **Add Gitea secrets:** + - Go to your repo → Settings → Secrets → Actions + - Add `DOCKERHUB_USERNAME` (your Docker Hub username) + - Add `DOCKERHUB_TOKEN` (access token from hub.docker.com) + +4. **Create VERSION file:** + ```bash + echo "1.0.0" > VERSION + git add VERSION + git commit -m "Add initial VERSION file" + ``` + +5. **Push and test:** + - Create a PR to test version validation + - Merge to main to test Docker push and notifications + +## Customization Guide + +### Disable Features + +**Don't want version validation?** +- Delete the "Check VERSION was bumped" step + +**Don't want Discord notifications?** +- Delete both "Discord Notification" steps + +**Don't want Docker Hub push?** +- Remove "Login to Docker Hub" step +- Change `push: ${{ github.ref == 'refs/heads/main' }}` to `push: false` + +### Customize Version Validation + +The template enforces strict semantic versioning. To modify: + +**Allow any version bump:** +```bash +# Remove the validation logic, just check if changed: +if [ "$PR_VERSION" = "$MAIN_VERSION" ]; then + echo "❌ VERSION unchanged" + exit 1 +fi +echo "✅ VERSION changed: $MAIN_VERSION → $PR_VERSION" +``` + +**Allow pre-release versions:** +```bash +# Modify parsing to handle versions like "1.2.3-beta" +IFS='-' read -r VERSION_NUMBER PRERELEASE <<< "$PR_VERSION" +``` + +### Add More Notifications + +**Slack webhook:** +```yaml +- name: Slack Notification + if: success() && github.ref == 'refs/heads/main' + run: | + curl -X POST YOUR_SLACK_WEBHOOK_URL \ + -H 'Content-Type: application/json' \ + -d '{"text": "Build succeeded: v${{ steps.meta.outputs.version }}"}' +``` + +**Email notification:** +```yaml +- name: Email Notification + if: failure() + uses: dawidd6/action-send-mail@v3 + with: + server_address: smtp.gmail.com + server_port: 465 + username: ${{ secrets.EMAIL_USERNAME }} + password: ${{ secrets.EMAIL_PASSWORD }} + subject: Build Failed - ${{ github.repository }} + body: Build failed on commit ${{ github.sha }} + to: you@example.com +``` + +## Troubleshooting + +### Version Validation Issues + +**Problem:** PR fails validation but VERSION was bumped +**Solution:** Check VERSION file format - should be just `1.2.3` with no prefix, suffix, or extra text + +**Problem:** Validation allows invalid bumps +**Solution:** Version parsing may be failing - check for special characters in VERSION file + +### Docker Hub Push Issues + +**Problem:** Login fails with "unauthorized" +**Solution:** Regenerate Docker Hub access token at hub.docker.com → Settings → Security + +**Problem:** Push fails with "denied" +**Solution:** Check repository name matches exactly (case-sensitive) + +**Problem:** Tags not appearing on Docker Hub +**Solution:** Wait a few seconds and refresh - tags may take a moment to appear + +### Discord Notification Issues + +**Problem:** Webhook succeeds but no message appears +**Solution:** +1. Check timestamp format is ISO 8601: `YYYY-MM-DDTHH:MM:SSZ` +2. Test webhook manually with curl +3. Verify webhook hasn't been deleted in Discord + +**Problem:** Message appears malformed +**Solution:** Check for unescaped quotes or special characters in message content + +**Problem:** Rate limited +**Solution:** Discord limits webhooks to ~5 messages per second - add delays if sending multiple + +## Advanced Usage + +### Multi-Stage Builds + +Add test/lint steps before build: + +```yaml +- name: Run tests + run: | + npm install + npm test + +- name: Lint code + run: npm run lint + +- name: Build Docker image + # ... existing build step +``` + +### Deploy After Build + +Add deployment to production: + +```yaml +- name: Deploy to production + if: success() && github.ref == 'refs/heads/main' + run: | + ssh production "docker pull yourusername/yourrepo:latest && docker-compose up -d" +``` + +### Multiple Docker Registries + +Push to multiple registries: + +```yaml +- name: Login to GitHub Container Registry + uses: docker/login-action@v3 + with: + registry: ghcr.io + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + +- name: Build Docker image + uses: docker/build-push-action@v5 + with: + push: ${{ github.ref == 'refs/heads/main' }} + tags: | + yourusername/yourrepo:latest + ghcr.io/yourusername/yourrepo:latest +``` + +## Template Updates + +This template was created based on the Paper Dynasty Discord bot workflow and represents battle-tested CI/CD practices. Future improvements might include: + +- [ ] Automatic changelog generation from commits +- [ ] Security scanning (Trivy, Snyk) +- [ ] Multi-architecture builds (ARM, AMD64) +- [ ] Deployment strategies (blue-green, canary) +- [ ] Integration testing with docker-compose +- [ ] Performance benchmarking + +## Contributing + +Found a bug or improvement? Update this template and document the change in this README. + +## License + +Free to use and modify for any project. + +--- + +**Template Version:** 1.0.0 +**Last Updated:** 2026-02-04 +**Maintained By:** Manticorum Home Lab diff --git a/server-configs/gitea/workflow-templates/deploy-script-template.sh b/server-configs/gitea/workflow-templates/deploy-script-template.sh new file mode 100644 index 0000000..a2a7d68 --- /dev/null +++ b/server-configs/gitea/workflow-templates/deploy-script-template.sh @@ -0,0 +1,82 @@ +#!/bin/bash +# Paper Dynasty - Manual Deployment Script +# +# Usage: ./deploy.sh [version] +# Example: ./deploy.sh v1.8.1 +# +# This script provides a safe, manual way to deploy Paper Dynasty +# with proper checks and rollback capability. + +set -e + +VERSION=${1:-latest} +SERVER="sba-bots" +SERVER_IP="10.10.0.88" +DEPLOY_PATH="/home/cal/container-data/paper-dynasty" + +echo "🚀 Deploying Paper Dynasty ${VERSION} to production" +echo "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━" + +# Pre-deployment checks +echo "" +echo "📋 Pre-deployment checks..." + +# Check SSH connection +if ! ssh cal@${SERVER_IP} "echo 'SSH OK'" > /dev/null 2>&1; then + echo "❌ Cannot connect to ${SERVER}" + exit 1 +fi +echo "✅ SSH connection OK" + +# Check if container exists +if ! ssh cal@${SERVER_IP} "cd ${DEPLOY_PATH} && docker compose ps" > /dev/null 2>&1; then + echo "❌ Cannot find Paper Dynasty container on ${SERVER}" + exit 1 +fi +echo "✅ Container found" + +# Confirm deployment +echo "" +echo "âš ī¸ This will restart the Paper Dynasty bot (brief downtime)" +read -p "Continue with deployment? (y/N) " -n 1 -r +echo +if [[ ! $REPLY =~ ^[Yy]$ ]]; then + echo "❌ Deployment cancelled" + exit 1 +fi + +# Deploy +echo "" +echo "đŸ“Ĩ Pulling image manticorum67/paper-dynasty:${VERSION}..." +ssh cal@${SERVER_IP} << EOF + cd ${DEPLOY_PATH} + + # Pull new image + docker compose pull + + # Stop old container + echo "🛑 Stopping container..." + docker compose down + + # Start new container + echo "â–ļī¸ Starting container..." + docker compose up -d + + # Wait for startup + sleep 5 + + # Check status + echo "" + echo "📊 Container status:" + docker compose ps + + echo "" + echo "📝 Recent logs:" + docker compose logs --tail 20 +EOF + +echo "" +echo "✅ Deployment complete!" +echo "" +echo "To check logs: ssh ${SERVER} 'cd ${DEPLOY_PATH} && docker compose logs -f'" +echo "To rollback: ssh ${SERVER} 'cd ${DEPLOY_PATH} && docker compose down && docker compose up -d'" diff --git a/server-configs/gitea/workflow-templates/docker-build-template.yml b/server-configs/gitea/workflow-templates/docker-build-template.yml new file mode 100644 index 0000000..9faf913 --- /dev/null +++ b/server-configs/gitea/workflow-templates/docker-build-template.yml @@ -0,0 +1,392 @@ +# Gitea Actions: Docker Build, Push, and Notify +# +# This workflow provides a complete CI/CD pipeline for Docker-based projects: +# - Validates semantic versioning on PRs +# - Builds Docker images on every push/PR +# - Pushes to Docker Hub on main branch merges +# - Sends Discord notifications on success/failure +# +# Template created: 2026-02-04 +# For: Paper Dynasty Discord bot (reference implementation) + +name: Build Docker Image + +on: + push: + branches: + - main + pull_request: + branches: + - main + +jobs: + build: + runs-on: ubuntu-latest + + steps: + # ============================================== + # 1. CHECKOUT CODE + # ============================================== + - name: Checkout code + uses: actions/checkout@v4 + + # ============================================== + # 2. SEMANTIC VERSION VALIDATION (PRs only) + # ============================================== + # Enforces proper semantic versioning: + # - Blocks PRs that don't bump VERSION file + # - Validates version changes follow semver rules + # - Prevents skipping versions or going backwards + # + # Valid bumps: + # - Patch: 1.2.3 → 1.2.4 (bug fixes) + # - Minor: 1.2.3 → 1.3.0 (new features) + # - Major: 1.2.3 → 2.0.0 (breaking changes) + # + # Invalid bumps: + # - 1.2.3 → 1.4.0 (skipped minor version) + # - 1.2.3 → 1.2.0 (went backwards) + # - 1.2.3 → 1.3.1 (didn't reset patch) + # + - name: Check VERSION was bumped (semantic versioning) + if: github.event_name == 'pull_request' + run: | + # Get VERSION from this PR branch + PR_VERSION=$(cat VERSION 2>/dev/null || echo "0.0.0") + + # Get VERSION from main branch + git fetch origin main:main + MAIN_VERSION=$(git show main:VERSION 2>/dev/null || echo "0.0.0") + + echo "📋 Semantic Version Check" + echo "Main branch version: $MAIN_VERSION" + echo "PR branch version: $PR_VERSION" + echo "" + + # Parse versions into components + IFS='.' read -r MAIN_MAJOR MAIN_MINOR MAIN_PATCH <<< "$MAIN_VERSION" + IFS='.' read -r PR_MAJOR PR_MINOR PR_PATCH <<< "$PR_VERSION" + + # Remove any non-numeric characters + MAIN_MAJOR=${MAIN_MAJOR//[!0-9]/} + MAIN_MINOR=${MAIN_MINOR//[!0-9]/} + MAIN_PATCH=${MAIN_PATCH//[!0-9]/} + PR_MAJOR=${PR_MAJOR//[!0-9]/} + PR_MINOR=${PR_MINOR//[!0-9]/} + PR_PATCH=${PR_PATCH//[!0-9]/} + + # Check if VERSION unchanged + if [ "$PR_VERSION" = "$MAIN_VERSION" ]; then + echo "❌ ERROR: VERSION file has not been updated!" + echo "" + echo "Please update the VERSION file in your PR." + echo "Current version: $MAIN_VERSION" + exit 1 + fi + + # Validate semantic version bump + VALID=false + BUMP_TYPE="" + + # Check for major version bump (X.0.0) + if [ "$PR_MAJOR" -eq $((MAIN_MAJOR + 1)) ] && [ "$PR_MINOR" -eq 0 ] && [ "$PR_PATCH" -eq 0 ]; then + VALID=true + BUMP_TYPE="major" + # Check for minor version bump (x.X.0) + elif [ "$PR_MAJOR" -eq "$MAIN_MAJOR" ] && [ "$PR_MINOR" -eq $((MAIN_MINOR + 1)) ] && [ "$PR_PATCH" -eq 0 ]; then + VALID=true + BUMP_TYPE="minor" + # Check for patch version bump (x.x.X) + elif [ "$PR_MAJOR" -eq "$MAIN_MAJOR" ] && [ "$PR_MINOR" -eq "$MAIN_MINOR" ] && [ "$PR_PATCH" -eq $((MAIN_PATCH + 1)) ]; then + VALID=true + BUMP_TYPE="patch" + fi + + if [ "$VALID" = true ]; then + echo "✅ Valid $BUMP_TYPE version bump: $MAIN_VERSION → $PR_VERSION" + else + echo "❌ ERROR: Invalid semantic version change!" + echo "" + echo "Current version: $MAIN_VERSION" + echo "PR version: $PR_VERSION" + echo "" + echo "Valid version bumps:" + echo " - Patch: $MAIN_MAJOR.$MAIN_MINOR.$((MAIN_PATCH + 1))" + echo " - Minor: $MAIN_MAJOR.$((MAIN_MINOR + 1)).0" + echo " - Major: $((MAIN_MAJOR + 1)).0.0" + echo "" + echo "Common issues:" + echo " ❌ Skipping versions (e.g., 2.5.0 → 2.7.0)" + echo " ❌ Going backwards (e.g., 2.5.0 → 2.4.0)" + echo " ❌ Not resetting lower components (e.g., 2.5.0 → 2.6.1)" + exit 1 + fi + + # ============================================== + # 3. DOCKER BUILDX SETUP + # ============================================== + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + # ============================================== + # 4. DOCKER HUB LOGIN (main branch only) + # ============================================== + # Requires secrets in Gitea: + # - DOCKERHUB_USERNAME: Your Docker Hub username + # - DOCKERHUB_TOKEN: Docker Hub access token (not password!) + # + # To create token: + # 1. Go to hub.docker.com + # 2. Account Settings → Security → New Access Token + # 3. Copy token to Gitea repo → Settings → Secrets → Actions + # + - name: Login to Docker Hub + if: github.ref == 'refs/heads/main' + uses: docker/login-action@v3 + with: + username: ${{ secrets.DOCKERHUB_USERNAME }} + password: ${{ secrets.DOCKERHUB_TOKEN }} + + # ============================================== + # 5. EXTRACT METADATA + # ============================================== + # Reads VERSION file and generates image tags: + # - version: From VERSION file (e.g., "1.2.3") + # - sha_short: First 7 chars of commit SHA + # - version_sha: Combined version+commit (e.g., "v1.2.3-a1b2c3d") + # - branch: Current branch name + # - timestamp: ISO 8601 format for Discord + # + - name: Extract metadata + id: meta + run: | + VERSION=$(cat VERSION 2>/dev/null || echo "0.0.0") + SHA_SHORT=$(echo ${{ github.sha }} | cut -c1-7) + echo "version=${VERSION}" >> $GITHUB_OUTPUT + echo "sha_short=${SHA_SHORT}" >> $GITHUB_OUTPUT + echo "version_sha=v${VERSION}-${SHA_SHORT}" >> $GITHUB_OUTPUT + echo "branch=${{ github.ref_name }}" >> $GITHUB_OUTPUT + echo "timestamp=$(date -u +'%Y-%m-%dT%H:%M:%SZ')" >> $GITHUB_OUTPUT + + # ============================================== + # 6. BUILD AND PUSH DOCKER IMAGE + # ============================================== + # Creates 3 tags for each build: + # - latest: Always points to newest build + # - v{VERSION}: Semantic version from VERSION file + # - v{VERSION}-{COMMIT}: Version + commit hash for traceability + # + # Example tags: + # - yourusername/yourrepo:latest + # - yourusername/yourrepo:v1.2.3 + # - yourusername/yourrepo:v1.2.3-a1b2c3d + # + # Push behavior: + # - PRs: Build only (test), don't push + # - Main: Build and push to Docker Hub + # + # CUSTOMIZE: Replace "yourusername/yourrepo" with your Docker Hub repo + # + - name: Build Docker image + uses: docker/build-push-action@v5 + with: + context: . + push: ${{ github.ref == 'refs/heads/main' }} + tags: | + yourusername/yourrepo:latest + yourusername/yourrepo:v${{ steps.meta.outputs.version }} + yourusername/yourrepo:${{ steps.meta.outputs.version_sha }} + cache-from: type=gha + cache-to: type=gha,mode=max + + # ============================================== + # 7. BUILD SUMMARY + # ============================================== + # Creates a formatted summary visible in Actions UI + # Shows: image tags, build details, push status + # + - name: Build Summary + run: | + echo "## đŸŗ Docker Build Successful! ✅" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "**Image Tags:**" >> $GITHUB_STEP_SUMMARY + echo "- \`yourusername/yourrepo:latest\`" >> $GITHUB_STEP_SUMMARY + echo "- \`yourusername/yourrepo:v${{ steps.meta.outputs.version }}\`" >> $GITHUB_STEP_SUMMARY + echo "- \`yourusername/yourrepo:${{ steps.meta.outputs.version_sha }}\`" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "**Build Details:**" >> $GITHUB_STEP_SUMMARY + echo "- Branch: \`${{ steps.meta.outputs.branch }}\`" >> $GITHUB_STEP_SUMMARY + echo "- Commit: \`${{ github.sha }}\`" >> $GITHUB_STEP_SUMMARY + echo "- Timestamp: \`${{ steps.meta.outputs.timestamp }}\`" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + if [ "${{ github.ref }}" == "refs/heads/main" ]; then + echo "🚀 **Pushed to Docker Hub!**" >> $GITHUB_STEP_SUMMARY + echo "" >> $GITHUB_STEP_SUMMARY + echo "Pull with: \`docker pull yourusername/yourrepo:latest\`" >> $GITHUB_STEP_SUMMARY + else + echo "_PR build - image not pushed to Docker Hub_" >> $GITHUB_STEP_SUMMARY + fi + + # ============================================== + # 8. DISCORD NOTIFICATION - SUCCESS + # ============================================== + # Sends green embed to Discord on successful builds + # + # Only fires on main branch pushes (not PRs) + # + # Setup: + # 1. Create webhook in Discord channel: + # Right-click channel → Edit → Integrations → Webhooks → New + # 2. Copy webhook URL + # 3. Replace the URL below + # + # CUSTOMIZE: + # - Replace webhook URL with yours + # - Replace "Your Project" in title + # - Replace Docker Hub URLs with your repo + # + - name: Discord Notification - Success + if: success() && github.ref == 'refs/heads/main' + run: | + curl -H "Content-Type: application/json" \ + -d '{ + "embeds": [{ + "title": "✅ Your Project Build Successful", + "description": "Docker image built and pushed to Docker Hub!", + "color": 3066993, + "fields": [ + { + "name": "Version", + "value": "`v${{ steps.meta.outputs.version }}`", + "inline": true + }, + { + "name": "Image Tag", + "value": "`${{ steps.meta.outputs.version_sha }}`", + "inline": true + }, + { + "name": "Branch", + "value": "`${{ steps.meta.outputs.branch }}`", + "inline": true + }, + { + "name": "Commit", + "value": "`${{ steps.meta.outputs.sha_short }}`", + "inline": true + }, + { + "name": "Author", + "value": "${{ github.actor }}", + "inline": true + }, + { + "name": "Docker Hub", + "value": "[yourusername/yourrepo](https://hub.docker.com/r/yourusername/yourrepo)", + "inline": false + }, + { + "name": "View Run", + "value": "[Click here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})", + "inline": false + } + ], + "timestamp": "${{ steps.meta.outputs.timestamp }}" + }] + }' \ + YOUR_DISCORD_WEBHOOK_URL_HERE + + # ============================================== + # 9. DISCORD NOTIFICATION - FAILURE + # ============================================== + # Sends red embed to Discord on build failures + # + # Only fires on main branch pushes (not PRs) + # + # CUSTOMIZE: + # - Replace webhook URL with yours + # - Replace "Your Project" in title + # + - name: Discord Notification - Failure + if: failure() && github.ref == 'refs/heads/main' + run: | + TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ) + curl -H "Content-Type: application/json" \ + -d '{ + "embeds": [{ + "title": "❌ Your Project Build Failed", + "description": "Docker build encountered an error.", + "color": 15158332, + "fields": [ + { + "name": "Branch", + "value": "`${{ github.ref_name }}`", + "inline": true + }, + { + "name": "Commit", + "value": "`${{ github.sha }}`", + "inline": true + }, + { + "name": "Author", + "value": "${{ github.actor }}", + "inline": true + }, + { + "name": "View Logs", + "value": "[Click here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})", + "inline": false + } + ], + "timestamp": "'"$TIMESTAMP"'" + }] + }' \ + YOUR_DISCORD_WEBHOOK_URL_HERE + +# ============================================== +# CUSTOMIZATION CHECKLIST +# ============================================== +# Before using this template in a new project: +# +# ✅ Replace "yourusername/yourrepo" with your Docker Hub repository +# ✅ Replace "Your Project" in Discord notification titles +# ✅ Replace Discord webhook URLs (both success and failure) +# ✅ Add secrets to Gitea repo: DOCKERHUB_USERNAME, DOCKERHUB_TOKEN +# ✅ Create VERSION file in repo root with initial version (e.g., "1.0.0") +# ✅ Update branch name if not using "main" +# +# Optional customizations: +# - Adjust runner labels (runs-on) if using self-hosted runners +# - Modify version validation rules if you don't want strict semver +# - Add additional notification channels (Slack, email, etc.) +# - Add deployment steps after Docker push +# - Customize Discord embed colors, fields, or formatting +# +# ============================================== +# TROUBLESHOOTING +# ============================================== +# Common issues and solutions: +# +# 1. VERSION validation failing unexpectedly +# - Ensure VERSION file exists in repo root +# - Check file contains only version number (no 'v' prefix or extra text) +# - Verify version follows semver: MAJOR.MINOR.PATCH +# +# 2. Docker Hub push failing +# - Verify DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secrets are set +# - Check Docker Hub token has push permissions +# - Ensure repository name matches your Docker Hub repo exactly +# +# 3. Discord notifications not appearing +# - Test webhook URL manually with curl +# - Check webhook still exists in Discord channel settings +# - Verify timestamp format is ISO 8601 (YYYY-MM-DDTHH:MM:SSZ) +# - Look for HTTP error codes in Actions logs +# +# 4. Build cache not working +# - GitHub Actions cache is stored per repository +# - Cache is shared across branches +# - May need to clear cache if corrupted +# +# ============================================== diff --git a/server-configs/gitea/workflow-templates/snippets/auto-deploy-with-rollback.yml b/server-configs/gitea/workflow-templates/snippets/auto-deploy-with-rollback.yml new file mode 100644 index 0000000..c1deaa7 --- /dev/null +++ b/server-configs/gitea/workflow-templates/snippets/auto-deploy-with-rollback.yml @@ -0,0 +1,167 @@ + # ============================================== + # SAFE AUTO-DEPLOY WITH HEALTH CHECK & ROLLBACK + # ============================================== + # Enhanced deployment with safety features: + # - Health check after deployment + # - Automatic rollback on failure + # - Deployment notifications + # - Downtime tracking + # + - name: Deploy to Production (Safe) + if: success() && github.ref == 'refs/heads/main' + run: | + # Set up SSH + mkdir -p ~/.ssh + echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/deploy_key + chmod 600 ~/.ssh/deploy_key + ssh-keyscan -H ${{ secrets.PRODUCTION_HOST }} >> ~/.ssh/known_hosts + + echo "🚀 Deploying Paper Dynasty v${{ steps.meta.outputs.version }} to production..." + DEPLOY_START=$(date +%s) + + # Deploy with health check and rollback + ssh -i ~/.ssh/deploy_key ${{ secrets.DEPLOY_USER }}@${{ secrets.PRODUCTION_HOST }} bash << 'EOF' + set -e + cd /home/cal/container-data/paper-dynasty + + # Save current image tag for rollback + CURRENT_IMAGE=$(docker compose images -q paper-dynasty 2>/dev/null || echo "none") + echo "Current image: $CURRENT_IMAGE" + + # Pull new image + echo "đŸ“Ĩ Pulling new image..." + docker compose pull + + # Stop old container + echo "🛑 Stopping old container..." + docker compose down + + # Start new container + echo "â–ļī¸ Starting new container..." + docker compose up -d + + # Health check with retry + echo "đŸĨ Running health check..." + for i in {1..10}; do + sleep 3 + if docker compose ps | grep -q "Up"; then + echo "✅ Container is up!" + + # Additional health check: check bot is responding + # Adjust this based on your bot's health endpoint + # if curl -f http://localhost:YOUR_PORT/health; then + # echo "✅ Health check passed!" + # exit 0 + # fi + + exit 0 + fi + echo "Waiting for container... ($i/10)" + done + + # If we get here, deployment failed + echo "❌ Health check failed! Rolling back..." + + # Rollback to previous image + if [ "$CURRENT_IMAGE" != "none" ]; then + docker compose down + # This assumes you have the old image still cached + # In production, you might want to keep the last N images + docker compose up -d + echo "âĒ Rolled back to previous version" + exit 1 + else + echo "âš ī¸ No previous image to rollback to!" + exit 1 + fi + EOF + + DEPLOY_STATUS=$? + DEPLOY_END=$(date +%s) + DEPLOY_TIME=$((DEPLOY_END - DEPLOY_START)) + + if [ $DEPLOY_STATUS -eq 0 ]; then + echo "✅ Deployment successful! (${DEPLOY_TIME}s)" + else + echo "❌ Deployment failed after ${DEPLOY_TIME}s" + exit 1 + fi + + # ============================================== + # DEPLOYMENT NOTIFICATION - SUCCESS + # ============================================== + - name: Discord Notification - Deployed Successfully + if: success() && github.ref == 'refs/heads/main' + run: | + curl -H "Content-Type: application/json" \ + -d '{ + "embeds": [{ + "title": "🚀 Paper Dynasty Deployed to Production", + "description": "New version is live and healthy!", + "color": 5793266, + "fields": [ + { + "name": "Version", + "value": "`v${{ steps.meta.outputs.version }}`", + "inline": true + }, + { + "name": "Deployed By", + "value": "${{ github.actor }}", + "inline": true + }, + { + "name": "Server", + "value": "sba-bots (10.10.0.88)", + "inline": true + }, + { + "name": "Status", + "value": "✅ Health check passed", + "inline": false + } + ], + "timestamp": "${{ steps.meta.outputs.timestamp }}" + }] + }' \ + https://discord.com/api/webhooks/YOUR_WEBHOOK_URL + + # ============================================== + # DEPLOYMENT NOTIFICATION - FAILED/ROLLED BACK + # ============================================== + - name: Discord Notification - Deployment Failed + if: failure() && github.ref == 'refs/heads/main' + run: | + TIMESTAMP=$(date -u +%Y-%m-%dT%H:%M:%SZ) + curl -H "Content-Type: application/json" \ + -d '{ + "embeds": [{ + "title": "âš ī¸ Paper Dynasty Deployment Failed", + "description": "Deployment failed and was rolled back to previous version.", + "color": 16776960, + "fields": [ + { + "name": "Attempted Version", + "value": "`v${{ steps.meta.outputs.version }}`", + "inline": true + }, + { + "name": "Author", + "value": "${{ github.actor }}", + "inline": true + }, + { + "name": "Action", + "value": "🔄 Rolled back to previous version", + "inline": false + }, + { + "name": "View Logs", + "value": "[Click here](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }})", + "inline": false + } + ], + "timestamp": "'"$TIMESTAMP"'" + }] + }' \ + https://discord.com/api/webhooks/YOUR_WEBHOOK_URL diff --git a/server-configs/hosts.yml b/server-configs/hosts.yml index f00894d..19a5a67 100644 --- a/server-configs/hosts.yml +++ b/server-configs/hosts.yml @@ -110,6 +110,7 @@ hosts: services: - n8n - omni-tools + - termix # Foundry VTT LXC (Proxmox) foundry-lxc: @@ -187,6 +188,23 @@ hosts: - major-domo-dev - strat-gameplay-webapp + # NPM + Pi-hole Host + npm-pihole: + type: docker + ssh_alias: npm-pihole + ip: 10.10.0.16 + user: cal + description: "Nginx Proxy Manager + Pi-hole DNS server" + config_paths: + scripts: /home/cal/scripts + services: + - nginx-proxy-manager + - pihole + documentation: server-configs/networking/nginx-proxy-manager-pihole.md + automation: + dns_sync: /home/cal/scripts/npm-pihole-sync.sh + cron: "0 * * * *" # Hourly + # Akamai Cloud Server akamai: type: docker diff --git a/server-configs/networking/nginx-proxy-manager-pihole.md b/server-configs/networking/nginx-proxy-manager-pihole.md new file mode 100644 index 0000000..cf0775e --- /dev/null +++ b/server-configs/networking/nginx-proxy-manager-pihole.md @@ -0,0 +1,302 @@ +# Nginx Proxy Manager + Pi-hole Setup + +**Host**: 10.10.0.16 +**Services**: Nginx Proxy Manager, Pi-hole DNS + +This host runs both NPM (reverse proxy) and Pi-hole (DNS server) as Docker containers. + +## Quick Info + +| Property | Value | +|----------|-------| +| **IP** | 10.10.0.16 | +| **User** | cal | +| **NPM Container** | nginx-proxy-manager_app_1 | +| **Pi-hole Container** | pihole | +| **Pi-hole Web** | http://10.10.0.16/admin | + +## Services + +### Nginx Proxy Manager +- **Container**: nginx-proxy-manager_app_1 +- **Web UI**: http://10.10.0.16:81 +- **Ports**: 80, 443, 81 +- **Database**: SQLite at `/data/database.sqlite` +- **Config**: `/data/nginx/` +- **Logs**: `/data/logs/` + +### Pi-hole +- **Container**: pihole +- **Web UI**: http://10.10.0.16/admin +- **Ports**: 53 (DNS), 80 (Web) +- **Config**: `/etc/pihole/` +- **Custom DNS**: `/etc/pihole/custom.list` + +## Architecture + +``` +┌─────────────┐ +│ Clients │ +│ (Homelab) │ +└──────â”Ŧ──────┘ + │ DNS Query: termix.manticorum.com? + â–ŧ +┌─────────────────────┐ +│ Pi-hole (10.10.0.16)│ +│ Returns: 10.10.0.16 │ ← Local DNS override +└──────â”Ŧ──────────────┘ + │ HTTPS Request + â–ŧ +┌──────────────────────────┐ +│ NPM (10.10.0.16) │ +│ - Checks access list │ ← Sees real client IP (10.0.0.206) +│ - Terminates SSL │ +│ - Proxies to backend │ +└──────â”Ŧ───────────────────┘ + │ HTTP Proxy + â–ŧ +┌──────────────────────┐ +│ Backend Service │ +│ (e.g., Termix) │ +│ 10.10.0.210:8180 │ +└──────────────────────┘ +``` + +## DNS Sync Automation + +### Problem Solved +When accessing homelab services by domain name (e.g., `termix.manticorum.com`), DNS would resolve to Cloudflare IPs, causing traffic to hairpin through the internet. This broke IP-based access lists since NPM would see your public IP instead of your internal IP. + +### Solution +Automatically sync all NPM proxy hosts to Pi-hole's local DNS, pointing them to NPM's internal IP (10.10.0.16). + +### Sync Script + +**Location**: `/home/cal/scripts/npm-pihole-sync.sh` (on 10.10.0.16) +**Repository**: `server-configs/networking/scripts/npm-pihole-sync.sh` + +The script: +1. Queries NPM's SQLite database for all enabled proxy hosts +2. Extracts domain names +3. Creates Pi-hole custom DNS entries pointing to NPM (10.10.0.16) +4. Reloads Pi-hole DNS + +**Usage:** +```bash +# Dry run (preview changes) +ssh cal@10.10.0.16 /home/cal/scripts/npm-pihole-sync.sh --dry-run + +# Apply changes +ssh cal@10.10.0.16 /home/cal/scripts/npm-pihole-sync.sh +``` + +**Automation:** +- Runs hourly via cron: `0 * * * * /home/cal/scripts/npm-pihole-sync.sh >> /home/cal/logs/npm-pihole-sync.log 2>&1` +- Logs: `/home/cal/logs/npm-pihole-sync.log` + +**Deployment:** +```bash +# Copy script to server +scp server-configs/networking/scripts/npm-pihole-sync.sh cal@10.10.0.16:/home/cal/scripts/ + +# Make executable +ssh cal@10.10.0.16 "chmod +x /home/cal/scripts/npm-pihole-sync.sh" + +# Test +ssh cal@10.10.0.16 "/home/cal/scripts/npm-pihole-sync.sh --dry-run" + +# Apply +ssh cal@10.10.0.16 "/home/cal/scripts/npm-pihole-sync.sh" +``` + +## Cloudflare Real IP Configuration + +To support external access through Cloudflare while maintaining proper IP detection for access lists, NPM is configured to trust Cloudflare's IP ranges and read the real client IP from the `CF-Connecting-IP` header. + +**Configuration** (in `/etc/nginx/nginx.conf` inside NPM container): + +```nginx +# Trust internal networks +set_real_ip_from 10.0.0.0/8; +set_real_ip_from 172.16.0.0/12; +set_real_ip_from 192.168.0.0/16; + +# Trust Cloudflare IPs +set_real_ip_from 173.245.48.0/20; +set_real_ip_from 103.21.244.0/22; +set_real_ip_from 103.22.200.0/22; +set_real_ip_from 103.31.4.0/22; +set_real_ip_from 141.101.64.0/18; +set_real_ip_from 108.162.192.0/18; +set_real_ip_from 190.93.240.0/20; +set_real_ip_from 188.114.96.0/20; +set_real_ip_from 197.234.240.0/22; +set_real_ip_from 198.41.128.0/17; +set_real_ip_from 162.158.0.0/15; +set_real_ip_from 104.16.0.0/13; +set_real_ip_from 104.24.0.0/14; +set_real_ip_from 172.64.0.0/13; +set_real_ip_from 131.0.72.0/22; + +# Read real IP from Cloudflare header +real_ip_header CF-Connecting-IP; +real_ip_recursive on; +``` + +This ensures: +- **Local access**: Client IP seen correctly (e.g., 10.0.0.206) +- **External access via Cloudflare**: Real public IP seen (not Cloudflare's proxy IP) + +## Access Lists + +NPM supports IP-based access restrictions. Common patterns: + +### Internal Only +Allows access only from local networks: +``` +allow 10.0.0.0/24; +allow 10.10.0.0/24; +deny all; +``` + +Use this for: +- Internal services (Termix, n8n, OmniTools) +- Admin interfaces +- Development environments + +### Cloudflare Only +Trusts Cloudflare's authenticated access (Cloudflare Access/Zero Trust): +- No IP restrictions needed +- Cloudflare handles authentication +- Use for public services behind Cloudflare Access + +## Management + +### Access Containers +```bash +# SSH to host +ssh cal@10.10.0.16 + +# Access NPM container +docker exec -it nginx-proxy-manager_app_1 bash + +# Access Pi-hole container +docker exec -it pihole bash +``` + +### NPM Commands +```bash +# View proxy host configs +docker exec nginx-proxy-manager_app_1 ls /data/nginx/proxy_host/ + +# View specific proxy host +docker exec nginx-proxy-manager_app_1 cat /data/nginx/proxy_host/21.conf + +# Test nginx config +docker exec nginx-proxy-manager_app_1 nginx -t + +# Reload nginx +docker exec nginx-proxy-manager_app_1 nginx -s reload + +# View access logs +docker exec nginx-proxy-manager_app_1 tail -f /data/logs/proxy-host-21_access.log +``` + +### Pi-hole Commands +```bash +# View custom DNS +docker exec pihole cat /etc/pihole/custom.list + +# Restart DNS +docker exec pihole pihole restartdns reload + +# View logs +docker logs pihole -f + +# Pi-hole status +docker exec pihole pihole status +``` + +### Query NPM Database +```bash +# List all proxy hosts +docker exec nginx-proxy-manager_app_1 python3 -c " +import sqlite3 +conn = sqlite3.connect('/data/database.sqlite') +cursor = conn.cursor() +cursor.execute('SELECT id, domain_names, forward_host, forward_port FROM proxy_host') +for row in cursor.fetchall(): + print(row) +conn.close() +" +``` + +## Troubleshooting + +### 403 Forbidden Errors + +**Symptom**: Getting 403 errors when accessing services with "Internal Only" access list. + +**Causes**: +1. **Hairpinning through Cloudflare**: DNS resolving to Cloudflare IPs instead of local + - **Fix**: Ensure Pi-hole has local DNS override (run sync script) + - **Check**: `dig +short service.manticorum.com @10.10.0.16` should return `10.10.0.16` + +2. **Access list doesn't include your subnet**: Your IP not in allowed ranges + - **Fix**: Add your subnet to access list (e.g., `10.0.0.0/24`) + - **Check**: View access logs to see what IP NPM is seeing + +3. **Real IP not detected**: NPM seeing proxy/Cloudflare IP instead of client IP + - **Fix**: Ensure Cloudflare IPs are in `set_real_ip_from` and `real_ip_header CF-Connecting-IP` is set + - **Check**: Access logs should show your real IP, not Cloudflare's + +**Debug Steps**: +```bash +# Check what IP NPM sees +docker exec nginx-proxy-manager_app_1 tail -20 /data/logs/proxy-host-21_access.log + +# Check DNS resolution +dig +short termix.manticorum.com @10.10.0.16 + +# Check access list in proxy host config +docker exec nginx-proxy-manager_app_1 cat /data/nginx/proxy_host/21.conf | grep -A 10 "Access Rules" +``` + +### DNS Not Resolving Locally + +**Symptom**: Domains still resolve to Cloudflare IPs. + +**Causes**: +1. DNS sync not run or failed +2. Desktop not using Pi-hole for DNS +3. DNS cache on desktop + +**Fix**: +```bash +# Re-run sync +ssh cal@10.10.0.16 /home/cal/scripts/npm-pihole-sync.sh + +# Check Pi-hole has the record +ssh cal@10.10.0.16 "docker exec pihole cat /etc/pihole/custom.list | grep termix" + +# Verify DNS server on desktop +cat /etc/resolv.conf # Should show 10.10.0.16 + +# Flush DNS cache on desktop +sudo resolvectl flush-caches # SystemD +# or +sudo systemd-resolve --flush-caches +``` + +## Related Documentation + +- [Gitea Server](../gitea/README.md) - Uses NPM proxy +- [n8n Automation](../../productivity/n8n/CONTEXT.md) - Uses NPM proxy +- [Akamai NPM](../akamai/docker-compose/nginx-proxy-manager/) - Public-facing NPM instance + +## Deployment Date + +**Created**: 2026-02-04 +**By**: Claude Code +**NPM Version**: (check with `docker exec nginx-proxy-manager_app_1 cat /app/package.json | grep version`) +**Pi-hole Version**: (check with `docker exec pihole pihole -v`) diff --git a/server-configs/networking/scripts/npm-pihole-sync.sh b/server-configs/networking/scripts/npm-pihole-sync.sh new file mode 100755 index 0000000..6050832 --- /dev/null +++ b/server-configs/networking/scripts/npm-pihole-sync.sh @@ -0,0 +1,74 @@ +#!/bin/bash +# NPM to Pi-hole DNS Sync +# Syncs Nginx Proxy Manager proxy hosts to Pi-hole local DNS +# All domains point to NPM's IP, not the forward destination + +set -e + +DRY_RUN=false +if [[ "$1" == "--dry-run" ]]; then + DRY_RUN=true +fi + +# NPM's IP address (where all domains should point) +NPM_IP="10.10.0.16" + +echo "NPM → Pi-hole DNS Sync" +echo "============================================================" + +# Query NPM database for all enabled proxy hosts +DOMAINS=$(docker exec nginx-proxy-manager_app_1 python3 -c ' +import sqlite3 +import json + +conn = sqlite3.connect("/data/database.sqlite") +cursor = conn.cursor() +cursor.execute("SELECT domain_names FROM proxy_host WHERE enabled = 1") + +domains = [] +for (domain_names,) in cursor.fetchall(): + for domain in json.loads(domain_names or "[]"): + domains.append(domain) + +for domain in sorted(domains): + print(domain) + +conn.close() +') + +# Count records +RECORD_COUNT=$(echo "$DOMAINS" | wc -l) +echo "Found $RECORD_COUNT enabled proxy hosts" +echo "" +echo "All domains will point to NPM at: $NPM_IP" +echo "" +echo "Domains to sync:" +echo "$DOMAINS" | awk -v ip="$NPM_IP" '{printf " %-15s %s\n", ip, $0}' + +if [ "$DRY_RUN" = true ]; then + echo "" + echo "[DRY RUN] Not applying changes" + exit 0 +fi + +# Build new custom.list +NEW_DNS="# Pi-hole Local DNS Records +# Auto-synced from Nginx Proxy Manager +# All domains point to NPM at $NPM_IP + +" + +while IFS= read -r domain; do + NEW_DNS+="$NPM_IP $domain"$'\n' +done <<< "$DOMAINS" + +# Write to Pi-hole +echo "$NEW_DNS" | docker exec -i pihole tee /etc/pihole/custom.list > /dev/null + +# Reload Pi-hole DNS +docker exec pihole pihole restartdns reload > /dev/null + +echo "" +echo "✓ Updated $RECORD_COUNT DNS records in Pi-hole" +echo "✓ All domains now point to NPM at $NPM_IP" +echo "✓ Reloaded Pi-hole DNS"