claude-home/vm-management/examples/proxmox-automation.md

# Proxmox VM Automation - Complete Examples

Complete working examples for automated VM provisioning in Proxmox environments using cloud-init and post-installation scripts.

## Overview

This guide provides real-world examples for automating Ubuntu Server VM deployment with:
- SSH key-based authentication
- Docker environment setup  
- Security hardening
- System updates and maintenance

## Example 1: Cloud-Init VM Creation

### Proxmox VM Setup
```bash
# Create VM with cloud-init support in Proxmox
# VM ID: 200, Name: homelab-docker-01, IP: 10.10.0.200

# 1. Create VM through Proxmox web interface or CLI
pvesh create /nodes/pve/qemu -vmid 200 -name homelab-docker-01 \
  -memory 2048 -cores 2 -sockets 1 -cpu cputype=host \
  -net0 virtio,bridge=vmbr0 \
  -ide2 local:cloudinit \
  -boot c -bootdisk scsi0 \
  -scsi0 local-lvm:10 \
  -ostype l26
```

### Cloud-Init Configuration
Use the complete `cloud-init-user-data.yaml` template:

```yaml
#cloud-config
hostname: homelab-docker-01
timezone: America/New_York

users:
  - name: cal
    groups: [sudo, docker]
    shell: /bin/bash
    sudo: ALL=(ALL) NOPASSWD:ALL
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDVVK02rOeeIw1e... homelab-cal@nobara-pc
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCPzqHDgdK9TlN4uzumBZKEOGt... emergency-homelab-cal@nobara-pc

ssh_pwauth: false
disable_root: true
package_update: true
package_upgrade: true

# Docker installation and configuration...
# (Full template available in scripts/vm-management/cloud-init-user-data.yaml)
```

### Deployment Process
1. **Create VM** with cloud-init drive attached
2. **Paste cloud-init config** into User Data field
3. **Set network** configuration (IP: 10.10.0.200/24, Gateway: 10.10.0.1)
4. **Start VM** - automatic provisioning begins
5. **Wait 3-5 minutes** for first boot provisioning
6. **Test connectivity**: `ssh cal@10.10.0.200`

## Example 2: Post-Installation Script

### Scenario: Existing VM Needs Provisioning
```bash
# VM already exists at 10.10.0.150 with password authentication
# Need to configure SSH keys, Docker, and security hardening

# Run the post-installation script
cd /mnt/NV2/Development/claude-home
./scripts/vm-management/vm-post-install.sh 10.10.0.150 cal
```

### Script Execution Flow
```
🔐 Starting VM provisioning for 10.10.0.150 as user cal
✅ System updates completed
✅ SSH key authentication successful  
✅ SSH hardened (password authentication disabled)
✅ Docker and Docker Compose installed successfully
✅ User environment configured
🎉 VM provisioning completed successfully!

=== PROVISIONING SUMMARY ===
VM IP: 10.10.0.150
User: cal
Hostname: homelab-vm-03
Docker: Docker version 24.0.7, build afdd53b
```

## Example 3: Bulk VM Provisioning

### Multiple VM Deployment Script
```bash
#!/bin/bash
# Bulk VM provisioning for home lab expansion
# Creates and configures multiple VMs with sequential IPs

VM_BASE_ID=300
VM_BASE_IP="10.10.0"
VM_COUNT=3

for i in $(seq 1 $VM_COUNT); do
    VM_ID=$((VM_BASE_ID + i))
    VM_IP="${VM_BASE_IP}.$((200 + i))"
    VM_NAME="homelab-worker-$(printf "%02d" $i)"
    
    echo "Creating VM: $VM_NAME (ID: $VM_ID, IP: $VM_IP)"
    
    # Create VM in Proxmox (adapt to your environment)
    pvesh create /nodes/pve/qemu -vmid $VM_ID -name $VM_NAME \
      -memory 2048 -cores 2 \
      -net0 virtio,bridge=vmbr0 \
      -ide2 local:cloudinit \
      -scsi0 local-lvm:20
    
    # Set cloud-init configuration
    pvesh set /nodes/pve/qemu/$VM_ID/config \
      -ipconfig0 ip=$VM_IP/24,gw=10.10.0.1 \
      -nameserver 10.10.0.16 \
      -user cal \
      -sshkeys /path/to/encoded/ssh-keys
    
    # Start VM
    pvesh create /nodes/pve/qemu/$VM_ID/status/start
    
    echo "VM $VM_NAME created and starting..."
    sleep 30  # Wait for boot
done

echo "Bulk provisioning completed. VMs available at:"
for i in $(seq 1 $VM_COUNT); do
    echo "  homelab-worker-$(printf "%02d" $i): ${VM_BASE_IP}.$((200 + i))"
done
```

## Example 4: Docker Service Deployment

### Post-Provisioning Service Setup
After VM provisioning, deploy services using Docker Compose:

```bash
# Connect to newly provisioned VM
ssh cal@10.10.0.200

# Create service directory
mkdir -p ~/services/nginx-proxy
cd ~/services/nginx-proxy

# Create docker-compose.yml
cat > docker-compose.yml << 'EOF'
version: '3.8'
services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./config:/etc/nginx/conf.d:ro
      - ./ssl:/etc/nginx/ssl:ro
    restart: unless-stopped
    
  portainer:
    image: portainer/portainer-ce:latest
    ports:
      - "9000:9000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    restart: unless-stopped

volumes:
  portainer_data:
EOF

# Deploy services
docker compose up -d
docker compose logs -f
```

## Example 5: VM Template Creation

### Creating Reusable VM Templates
```bash
# 1. Create and fully configure a base VM
./scripts/vm-management/vm-post-install.sh 10.10.0.199 cal

# 2. Clean up the VM for template use
ssh cal@10.10.0.199 '
    # Clear bash history
    history -c && history -w
    
    # Clear logs
    sudo truncate -s 0 /var/log/*log
    sudo truncate -s 0 /var/log/syslog
    
    # Clear SSH host keys (regenerated on first boot)
    sudo rm -f /etc/ssh/ssh_host_*
    
    # Clear machine ID
    sudo truncate -s 0 /etc/machine-id
    
    # Clear network configuration
    sudo rm -f /etc/udev/rules.d/70-persistent-net.rules
    
    # Shutdown
    sudo shutdown -h now
'

# 3. Convert VM to template in Proxmox
pvesh create /nodes/pve/qemu/199/template

# 4. Clone template to create new VMs
pvesh create /nodes/pve/qemu/199/clone -newid 201 -name homelab-from-template
```

## Example 6: Monitoring and Verification

### Post-Deployment Health Checks
```bash
#!/bin/bash
# VM health check script
# Usage: ./vm-health-check.sh <vm-ip>

VM_IP="$1"

echo "🔍 VM Health Check: $VM_IP"

# Test SSH connectivity
if ssh -o ConnectTimeout=5 cal@$VM_IP 'echo "SSH OK"' >/dev/null 2>&1; then
    echo "✅ SSH: Connected"
else
    echo "❌ SSH: Failed"
    exit 1
fi

# Check system info
INFO=$(ssh cal@$VM_IP '
    echo "Hostname: $(hostname)"
    echo "Uptime: $(uptime -p)"
    echo "Memory: $(free -h | grep Mem | awk "{print \$3\"/\"\$2}")"
    echo "Disk: $(df -h / | tail -1 | awk "{print \$3\"/\"\$2\" (\"\$5\" used)\"}")"
')

echo "$INFO"

# Test Docker
if ssh cal@$VM_IP 'docker --version && docker compose version' >/dev/null 2>&1; then
    echo "✅ Docker: Installed and working"
else
    echo "❌ Docker: Not working"
fi

# Test Docker permissions
if ssh cal@$VM_IP 'docker run --rm hello-world' >/dev/null 2>&1; then
    echo "✅ Docker: User permissions OK"
else
    echo "❌ Docker: User permissions issue"
fi

echo "🎉 Health check completed"
```

## Common Workflows

### 1. New Development Environment
```bash
# Create VM with cloud-init
# Wait for provisioning
# Deploy development stack
ssh cal@10.10.0.201 '
    mkdir -p ~/dev
    cd ~/dev
    
    # Clone your project
    git clone https://github.com/your-org/project.git
    cd project
    
    # Start development environment
    docker compose -f docker-compose.dev.yml up -d
'
```

### 2. Service Migration
```bash
# Provision new VM
./scripts/vm-management/vm-post-install.sh 10.10.0.202 cal

# Copy service configuration from existing server
scp -r cal@10.10.0.124:~/services/app-name cal@10.10.0.202:~/services/

# Start service on new VM
ssh cal@10.10.0.202 'cd ~/services/app-name && docker compose up -d'
```

### 3. Testing Environment
```bash
# Quick test environment with cloud-init
# Use minimal resources: 1GB RAM, 1 CPU, 10GB disk
# Deploy test configuration
# Run automated tests
# Destroy when done
```

## Troubleshooting Examples

### VM Won't Start After Cloud-Init
```bash
# Check cloud-init logs
ssh cal@<vm-ip> 'sudo cloud-init status --long'
ssh cal@<vm-ip> 'sudo cat /var/log/cloud-init-output.log'

# Manual cloud-init re-run if needed
ssh cal@<vm-ip> 'sudo cloud-init clean && sudo cloud-init init'
```

### SSH Key Issues
```bash
# Verify keys are properly installed
ssh cal@<vm-ip> 'cat ~/.ssh/authorized_keys | wc -l'  # Should show 2 keys

# Test specific key
ssh -i ~/.ssh/homelab_rsa cal@<vm-ip> 'echo "Primary key works"'
ssh -i ~/.ssh/emergency_homelab_rsa cal@<vm-ip> 'echo "Emergency key works"'
```

### Docker Permission Problems
```bash
# Check user groups
ssh cal@<vm-ip> 'groups'  # Should include 'docker'

# Re-login to apply group membership
ssh cal@<vm-ip> 'newgrp docker'

# Test Docker access
ssh cal@<vm-ip> 'docker ps'
```

## Related Documentation

- **Scripts**: `scripts/vm-management/README.md` - Implementation details
- **Patterns**: `patterns/vm-management/README.md` - Architectural guidance
- **SSH Setup**: `examples/networking/ssh-homelab-setup.md` - Key management
- **Docker**: `examples/docker/` - Container deployment patterns