claude-home/server-configs/gitea/deployment-strategies.md

---
title: "CI/CD Deployment Strategies"
description: "Decision framework and implementation guide for choosing deployment automation levels (manual through blue-green). Covers 5 progressive strategies with safety best practices, rollback procedures, and per-project recommendations."
type: guide
domain: server-configs
tags: [gitea, ci-cd, deployment, docker, rollback, blue-green, health-checks]
---

# Deployment Strategies for Gitea CI/CD

Guide to choosing and implementing deployment automation for your projects.

## 🎯 Decision Framework

### Consider These Factors

1. **Criticality**: How important is uptime?
   - Discord bot for active league: HIGH
   - Personal project: LOW
   - Public API: VERY HIGH

2. **Test Coverage**: How confident are you in your tests?
   - Comprehensive unit + integration tests: AUTO-DEPLOY OK
   - Some tests: SEMI-AUTO
   - No tests: MANUAL ONLY

3. **Rollback Complexity**: Can you easily revert?
   - Docker with versioned images: EASY
   - Database migrations: HARD
   - Stateful services: MEDIUM

4. **Traffic Patterns**: When can you deploy?
   - 24/7 users: Need zero-downtime
   - Business hours only: Deploy off-hours
   - Low traffic: Restart acceptable

## 📊 Deployment Options Comparison

| Strategy | Speed | Safety | Complexity | Best For |
|----------|-------|--------|------------|----------|
| **Manual** | Slow | High | Low | Critical systems, low test coverage |
| **Semi-Auto** | Medium | High | Low | Most projects (recommended) |
| **Auto + Health** | Fast | Medium | Medium | Good test coverage, health endpoints |
| **Blue-Green** | Fast | High | High | Zero-downtime required |
| **Canary** | Fast | Very High | Very High | Large user base, gradual rollout |

## 🚀 Recommended Progression

### Level 1: Manual Deploy (Start Here)
```bash
# Simple, safe, controllable
ssh production
docker compose pull
docker compose up -d
```

**Pros:**
- ✅ Full control over timing
- ✅ Can test/verify first
- ✅ Simple to understand
- ✅ Easy to abort

**Cons:**
- ❌ Requires manual SSH
- ❌ Inconsistent process
- ❌ Slower deployments

### Level 2: Deploy Script (Easy Win)
```bash
# One command from local machine
./deploy.sh v1.8.1
```

**Pros:**
- ✅ Consistent process
- ✅ Pre-deployment checks
- ✅ Still manual control
- ✅ Rollback instructions

**Cons:**
- ❌ Still requires human
- ❌ Can't deploy from CI

**Setup:** Copy `/tmp/deploy.sh` to your repo root

### Level 3: CI Notifies, Manual Deploy
```yaml
# Workflow builds, then notifies:
- Discord: "v1.8.1 ready to deploy"
- You run: ./deploy.sh v1.8.1
```

**Pros:**
- ✅ Fast build process
- ✅ Clear deployment signal
- ✅ Choose deployment timing
- ✅ Audit trail

**Cons:**
- ❌ Two-step process
- ❌ Can forget to deploy

**Current State:** This is where Paper Dynasty is now ✅

### Level 4: Auto-Deploy with Safeguards
```yaml
# Workflow builds, tests, deploys, health checks
- Build image
- Push to Docker Hub
- SSH to production
- Pull and restart
- Health check
- Rollback on failure
- Notify Discord
```

**Pros:**
- ✅ Fully automated
- ✅ Fast deployment
- ✅ Automatic rollback
- ✅ Consistent process

**Cons:**
- ❌ Complexity
- ❌ Brief downtime
- ❌ Requires good health checks

**Setup:** Use `/tmp/safe-auto-deploy-step.yml`

### Level 5: Blue-Green Deployment
```yaml
# Run two environments, switch traffic
- Deploy to "green" environment
- Health check green
- Switch load balancer to green
- Keep "blue" as instant rollback
```

**Pros:**
- ✅ Zero downtime
- ✅ Instant rollback
- ✅ Test before switching
- ✅ Very safe

**Cons:**
- ❌ Requires two environments
- ❌ Needs load balancer
- ❌ Database complexity
- ❌ 2x resources

**Best For:** High-traffic web services

## 🎯 Recommendations by Project Type

### Discord Bot (Paper Dynasty)
**Recommended:** Level 3 (CI Notify + Manual Deploy)

**Why?**
- Active users (league members)
- Brief downtime acceptable (1-2 min)
- Manual timing is valuable
- Can deploy during off-hours

**Next Step:** Add deploy script for consistency

### Internal Tool / Dashboard
**Recommended:** Level 4 (Auto-Deploy with Health Check)

**Why?**
- Limited users
- Downtime less critical
- Faster iteration valuable
- Easy rollback available

### Public API / High Traffic
**Recommended:** Level 5 (Blue-Green) or Canary

**Why?**
- Zero downtime required
- Large user base
- Complex rollback scenarios
- Worth the complexity

### Personal Project / Portfolio
**Recommended:** Level 2 or 3

**Why?**
- Simple is better
- Automation overhead not worth it
- Infrequent deploys
- Learning opportunity

## 🛠️ Implementation Guide

### Adding Auto-Deploy to Paper Dynasty

If you decide to try auto-deploy:

1. **Add deploy SSH key to secrets:**
   ```bash
   # Generate deploy-only SSH key
   ssh-keygen -t ed25519 -f ~/.ssh/paper_dynasty_deploy -C "gitea-deploy"

   # Add public key to sba-bots
   ssh-copy-id -i ~/.ssh/paper_dynasty_deploy.pub cal@10.10.0.88

   # Add private key to Gitea secrets
   cat ~/.ssh/paper_dynasty_deploy
   # Copy to: Repo → Settings → Secrets → DEPLOY_SSH_KEY
   ```

2. **Add production host secret:**
   - Secret name: `PRODUCTION_HOST`
   - Value: `10.10.0.88`

3. **Add deploy user secret:**
   - Secret name: `DEPLOY_USER`
   - Value: `cal`

4. **Add deploy step to workflow:**
   - Copy from `/tmp/safe-auto-deploy-step.yml`
   - Paste after "Build Docker image" step
   - Update webhook URL

5. **Test with a minor change:**
   - Make a small PR (comment change)
   - Merge and watch deployment
   - Verify bot restarts successfully
   - Check Discord for notifications

6. **Monitor first few deployments:**
   - Watch for issues
   - Check logs: `ssh sba-bots 'docker compose logs'`
   - Verify no errors

### Adding Health Checks

For better auto-deploy safety, add a health endpoint to your bot:

```javascript
// Express health check endpoint
app.get('/health', (req, res) => {
  // Check Discord connection
  if (!client.ws.ping) {
    return res.status(503).json({ status: 'unhealthy', reason: 'Discord disconnected' });
  }

  // Check database connection
  if (!db.isConnected()) {
    return res.status(503).json({ status: 'unhealthy', reason: 'Database disconnected' });
  }

  res.json({
    status: 'healthy',
    version: process.env.VERSION,
    uptime: process.uptime(),
    discord: {
      guilds: client.guilds.cache.size,
      ping: client.ws.ping
    }
  });
});
```

Then health check in deploy script:
```bash
curl -f http://localhost:3000/health || rollback
```

## ⚠️ Safety Best Practices

### Before Auto-Deploying

1. **✅ Add Comprehensive Tests**
   - Unit tests for core logic
   - Integration tests for Discord API
   - Smoke tests for critical paths

2. **✅ Implement Health Checks**
   - HTTP endpoint for status
   - Check all critical services
   - Return proper status codes

3. **✅ Set Up Monitoring**
   - Error tracking (Sentry)
   - Performance monitoring (Datadog)
   - Discord webhook for errors

4. **✅ Plan Rollback Process**
   - Keep last 3 images cached
   - Document rollback steps
   - Test rollback procedure

5. **✅ Document Deployment**
   - What gets deployed
   - How to abort
   - How to rollback
   - Who to contact

### During Deployment

1. **Monitor Logs**
   ```bash
   docker compose logs -f
   ```

2. **Check Metrics**
   - Response times
   - Error rates
   - Resource usage

3. **Verify Functionality**
   - Test critical commands
   - Check database connections
   - Verify integrations

### After Deployment

1. **Watch for Issues**
   - First 5 minutes are critical
   - Check error channels
   - Monitor user reports

2. **Validate Success**
   - All services healthy
   - No error spike
   - Performance normal

3. **Document Issues**
   - What went wrong
   - How you fixed it
   - How to prevent

## 🔄 Rollback Procedures

### Quick Rollback (Docker)
```bash
ssh production
cd /path/to/app
docker compose down
docker tag yourusername/repo:current yourusername/repo:rollback-backup
docker pull yourusername/repo:v1.7.0  # Last known good
docker compose up -d
```

### Emergency Rollback (CI)
```yaml
# Manually trigger workflow with old version
git tag -f v1.7.0-redeploy
git push -f origin v1.7.0-redeploy
# Watch actions deploy old version
```

## 📚 Further Reading

- [Deployment Strategies](https://www.redhat.com/en/topics/devops/deployment-strategies)
- [Blue-Green Deployments](https://martinfowler.com/bliki/BlueGreenDeployment.html)
- [Canary Releases](https://martinfowler.com/bliki/CanaryRelease.html)
- [Docker Health Checks](https://docs.docker.com/engine/reference/builder/#healthcheck)

---

**Last Updated:** 2026-02-04
**Applies To:** Paper Dynasty Discord bot, future Gitea CI/CD projects