--- title: "CI/CD Deployment Strategies" description: "Decision framework and implementation guide for choosing deployment automation levels (manual through blue-green). Covers 5 progressive strategies with safety best practices, rollback procedures, and per-project recommendations." type: guide domain: server-configs tags: [gitea, ci-cd, deployment, docker, rollback, blue-green, health-checks] --- # Deployment Strategies for Gitea CI/CD Guide to choosing and implementing deployment automation for your projects. ## 🎯 Decision Framework ### Consider These Factors 1. **Criticality**: How important is uptime? - Discord bot for active league: HIGH - Personal project: LOW - Public API: VERY HIGH 2. **Test Coverage**: How confident are you in your tests? - Comprehensive unit + integration tests: AUTO-DEPLOY OK - Some tests: SEMI-AUTO - No tests: MANUAL ONLY 3. **Rollback Complexity**: Can you easily revert? - Docker with versioned images: EASY - Database migrations: HARD - Stateful services: MEDIUM 4. **Traffic Patterns**: When can you deploy? - 24/7 users: Need zero-downtime - Business hours only: Deploy off-hours - Low traffic: Restart acceptable ## 📊 Deployment Options Comparison | Strategy | Speed | Safety | Complexity | Best For | |----------|-------|--------|------------|----------| | **Manual** | Slow | High | Low | Critical systems, low test coverage | | **Semi-Auto** | Medium | High | Low | Most projects (recommended) | | **Auto + Health** | Fast | Medium | Medium | Good test coverage, health endpoints | | **Blue-Green** | Fast | High | High | Zero-downtime required | | **Canary** | Fast | Very High | Very High | Large user base, gradual rollout | ## 🚀 Recommended Progression ### Level 1: Manual Deploy (Start Here) ```bash # Simple, safe, controllable ssh production docker compose pull docker compose up -d ``` **Pros:** - ✅ Full control over timing - ✅ Can test/verify first - ✅ Simple to understand - ✅ Easy to abort **Cons:** - ❌ Requires manual SSH - ❌ Inconsistent process - ❌ Slower deployments ### Level 2: Deploy Script (Easy Win) ```bash # One command from local machine ./deploy.sh v1.8.1 ``` **Pros:** - ✅ Consistent process - ✅ Pre-deployment checks - ✅ Still manual control - ✅ Rollback instructions **Cons:** - ❌ Still requires human - ❌ Can't deploy from CI **Setup:** Copy `/tmp/deploy.sh` to your repo root ### Level 3: CI Notifies, Manual Deploy ```yaml # Workflow builds, then notifies: - Discord: "v1.8.1 ready to deploy" - You run: ./deploy.sh v1.8.1 ``` **Pros:** - ✅ Fast build process - ✅ Clear deployment signal - ✅ Choose deployment timing - ✅ Audit trail **Cons:** - ❌ Two-step process - ❌ Can forget to deploy **Current State:** This is where Paper Dynasty is now ✅ ### Level 4: Auto-Deploy with Safeguards ```yaml # Workflow builds, tests, deploys, health checks - Build image - Push to Docker Hub - SSH to production - Pull and restart - Health check - Rollback on failure - Notify Discord ``` **Pros:** - ✅ Fully automated - ✅ Fast deployment - ✅ Automatic rollback - ✅ Consistent process **Cons:** - ❌ Complexity - ❌ Brief downtime - ❌ Requires good health checks **Setup:** Use `/tmp/safe-auto-deploy-step.yml` ### Level 5: Blue-Green Deployment ```yaml # Run two environments, switch traffic - Deploy to "green" environment - Health check green - Switch load balancer to green - Keep "blue" as instant rollback ``` **Pros:** - ✅ Zero downtime - ✅ Instant rollback - ✅ Test before switching - ✅ Very safe **Cons:** - ❌ Requires two environments - ❌ Needs load balancer - ❌ Database complexity - ❌ 2x resources **Best For:** High-traffic web services ## 🎯 Recommendations by Project Type ### Discord Bot (Paper Dynasty) **Recommended:** Level 3 (CI Notify + Manual Deploy) **Why?** - Active users (league members) - Brief downtime acceptable (1-2 min) - Manual timing is valuable - Can deploy during off-hours **Next Step:** Add deploy script for consistency ### Internal Tool / Dashboard **Recommended:** Level 4 (Auto-Deploy with Health Check) **Why?** - Limited users - Downtime less critical - Faster iteration valuable - Easy rollback available ### Public API / High Traffic **Recommended:** Level 5 (Blue-Green) or Canary **Why?** - Zero downtime required - Large user base - Complex rollback scenarios - Worth the complexity ### Personal Project / Portfolio **Recommended:** Level 2 or 3 **Why?** - Simple is better - Automation overhead not worth it - Infrequent deploys - Learning opportunity ## 🛠️ Implementation Guide ### Adding Auto-Deploy to Paper Dynasty If you decide to try auto-deploy: 1. **Add deploy SSH key to secrets:** ```bash # Generate deploy-only SSH key ssh-keygen -t ed25519 -f ~/.ssh/paper_dynasty_deploy -C "gitea-deploy" # Add public key to sba-bots ssh-copy-id -i ~/.ssh/paper_dynasty_deploy.pub cal@10.10.0.88 # Add private key to Gitea secrets cat ~/.ssh/paper_dynasty_deploy # Copy to: Repo → Settings → Secrets → DEPLOY_SSH_KEY ``` 2. **Add production host secret:** - Secret name: `PRODUCTION_HOST` - Value: `10.10.0.88` 3. **Add deploy user secret:** - Secret name: `DEPLOY_USER` - Value: `cal` 4. **Add deploy step to workflow:** - Copy from `/tmp/safe-auto-deploy-step.yml` - Paste after "Build Docker image" step - Update webhook URL 5. **Test with a minor change:** - Make a small PR (comment change) - Merge and watch deployment - Verify bot restarts successfully - Check Discord for notifications 6. **Monitor first few deployments:** - Watch for issues - Check logs: `ssh sba-bots 'docker compose logs'` - Verify no errors ### Adding Health Checks For better auto-deploy safety, add a health endpoint to your bot: ```javascript // Express health check endpoint app.get('/health', (req, res) => { // Check Discord connection if (!client.ws.ping) { return res.status(503).json({ status: 'unhealthy', reason: 'Discord disconnected' }); } // Check database connection if (!db.isConnected()) { return res.status(503).json({ status: 'unhealthy', reason: 'Database disconnected' }); } res.json({ status: 'healthy', version: process.env.VERSION, uptime: process.uptime(), discord: { guilds: client.guilds.cache.size, ping: client.ws.ping } }); }); ``` Then health check in deploy script: ```bash curl -f http://localhost:3000/health || rollback ``` ## ⚠️ Safety Best Practices ### Before Auto-Deploying 1. **✅ Add Comprehensive Tests** - Unit tests for core logic - Integration tests for Discord API - Smoke tests for critical paths 2. **✅ Implement Health Checks** - HTTP endpoint for status - Check all critical services - Return proper status codes 3. **✅ Set Up Monitoring** - Error tracking (Sentry) - Performance monitoring (Datadog) - Discord webhook for errors 4. **✅ Plan Rollback Process** - Keep last 3 images cached - Document rollback steps - Test rollback procedure 5. **✅ Document Deployment** - What gets deployed - How to abort - How to rollback - Who to contact ### During Deployment 1. **Monitor Logs** ```bash docker compose logs -f ``` 2. **Check Metrics** - Response times - Error rates - Resource usage 3. **Verify Functionality** - Test critical commands - Check database connections - Verify integrations ### After Deployment 1. **Watch for Issues** - First 5 minutes are critical - Check error channels - Monitor user reports 2. **Validate Success** - All services healthy - No error spike - Performance normal 3. **Document Issues** - What went wrong - How you fixed it - How to prevent ## 🔄 Rollback Procedures ### Quick Rollback (Docker) ```bash ssh production cd /path/to/app docker compose down docker tag yourusername/repo:current yourusername/repo:rollback-backup docker pull yourusername/repo:v1.7.0 # Last known good docker compose up -d ``` ### Emergency Rollback (CI) ```yaml # Manually trigger workflow with old version git tag -f v1.7.0-redeploy git push -f origin v1.7.0-redeploy # Watch actions deploy old version ``` ## 📚 Further Reading - [Deployment Strategies](https://www.redhat.com/en/topics/devops/deployment-strategies) - [Blue-Green Deployments](https://martinfowler.com/bliki/BlueGreenDeployment.html) - [Canary Releases](https://martinfowler.com/bliki/CanaryRelease.html) - [Docker Health Checks](https://docs.docker.com/engine/reference/builder/#healthcheck) --- **Last Updated:** 2026-02-04 **Applies To:** Paper Dynasty Discord bot, future Gitea CI/CD projects