cal/claude-home
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity
e2ec022a33
claude-home/vm-management/scripts/CONTEXT.md
Cal Corum ceb4dd36a0 Add docker scripts, media-tools, VM management, and n8n workflow docs 
Add CONTEXT.md for docker and VM management script directories.
Add media-tools documentation with Playwright scraping patterns.
Add Tdarr GPU monitor n8n workflow definition.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-07 22:26:10 -06:00

386 lines
12 KiB
Markdown
Raw Blame History

# VM Management Scripts - Operational Context
## Script Overview
This directory contains active operational scripts for VM provisioning, LXC container creation, Docker configuration in containers, and system migration.
## Core Scripts
### VM Post-Installation Provisioning
**Script**: `vm-post-install.sh`
**Purpose**: Automated provisioning of existing VMs with security hardening, SSH keys, and Docker
**Key Features**:
- System updates and essential packages installation
- SSH key deployment (primary + emergency keys)
- SSH security hardening (disable password authentication)
- Docker and Docker Compose installation
- User environment setup with bash aliases
- Automatic security updates configuration
**Usage**:
```bash
./vm-post-install.sh <vm-ip> [ssh-user]
# Example
./vm-post-install.sh 10.10.0.100 cal
```
**Requirements**:
- Target VM must have SSH access enabled initially
- Homelab SSH keys must exist: `~/.ssh/homelab_rsa` and `~/.ssh/emergency_homelab_rsa`
- Initial connection may require password authentication (disabled after provisioning)
**Post-Provision Verification**:
```bash
# Test SSH with key
ssh cal@<vm-ip>
# Verify Docker
docker --version
docker run --rm hello-world
# Check security
sudo sshd -T | grep -E "(passwordauth|pubkeyauth)"
```
### Cloud-Init Automated Provisioning
**File**: `cloud-init-user-data.yaml`
**Purpose**: Fully automated VM provisioning from first boot using Proxmox cloud-init
**Features**:
- User creation with sudo privileges
- SSH keys pre-installed (no password auth needed)
- Automatic package updates
- Docker and Docker Compose installation
- Security hardening from first boot
- Useful bash aliases and environment setup
- Welcome message with system status
**Usage in Proxmox**:
1. Create new VM with cloud-init support
2. Go to Cloud-Init tab in VM settings
3. Copy contents of `cloud-init-user-data.yaml`
4. Paste into "User Data" field
5. Start VM - fully provisioned automatically
**Benefits**:
- Zero-touch provisioning
- Consistent configuration across all VMs
- No password authentication ever enabled
- Faster deployment than post-install script
### Docker AppArmor Fix for LXC
**Script**: `fix-docker-apparmor.sh`
**Purpose**: Add AppArmor=unconfined to docker-compose.yml files for LXC compatibility
**Why Needed**: Docker containers inside LXC containers require AppArmor to be disabled. Without this fix, containers may fail to start or have permission issues.
**Usage**:
```bash
./fix-docker-apparmor.sh <LXC_IP> [COMPOSE_DIR]
# Example - use default directory (/home/cal/container-data)
./fix-docker-apparmor.sh 10.10.0.214
# Example - specify custom directory
./fix-docker-apparmor.sh 10.10.0.214 /home/cal/docker
```
**What It Does**:
1. SSHs into the LXC container
2. Finds all docker-compose.yml files in specified directory
3. Adds `security_opt: ["apparmor=unconfined"]` to each service
4. Creates `.bak` backups of original files before modification
**Safety Features**:
- Creates backups before modifications
- Color-coded output for easy monitoring
- Error handling with detailed logging
- Validates SSH connectivity before proceeding
### LXC Container Creation with Docker
**Script**: `lxc-docker-create.sh`
**Purpose**: Create new LXC containers pre-configured for Docker hosting
**Key Features**:
- Automated LXC container creation in Proxmox
- Docker and Docker Compose pre-installed
- AppArmor configuration for container compatibility
- Network configuration
- Security settings optimized for Docker hosting
**Usage**:
```bash
./lxc-docker-create.sh [options]
```
**Common Use Cases**:
- Creating Docker hosts for specific services (n8n, gitea, etc.)
- Rapid deployment of containerized applications
- Consistent LXC configuration across infrastructure
### LXC Migration Guide
**Document**: `LXC-MIGRATION-GUIDE.md`
**Purpose**: Step-by-step procedures for migrating LXC containers between hosts
**Covers**:
- Pre-migration planning and backups
- LXC configuration export/import
- Storage migration strategies
- Network reconfiguration
- Post-migration validation
- Rollback procedures
**When to Use**:
- Moving containers to new Proxmox host
- Hardware upgrades
- Load balancing across nodes
- Disaster recovery scenarios
## Operational Patterns
### VM Provisioning Workflow
**Option 1: New VMs (Preferred)**
```bash
# 1. Create VM in Proxmox with cloud-init support
# 2. Copy cloud-init-user-data.yaml to User Data field
# 3. Start VM
# 4. Verify provisioning completed:
ssh cal@<vm-ip>
docker --version
```
**Option 2: Existing VMs**
```bash
# 1. Ensure VM has SSH enabled and accessible
# 2. Run post-install script:
./vm-post-install.sh 10.10.0.100 cal
# 3. Verify provisioning:
ssh cal@10.10.0.100
docker --version
```
### LXC Docker Deployment Workflow
**Creating New LXC for Docker**:
```bash
# 1. Create LXC container
./lxc-docker-create.sh --id 220 --hostname docker-app --ip 10.10.0.220
# 2. Deploy docker-compose configurations
scp -r ./app-config/ cal@10.10.0.220:/home/cal/container-data/
# 3. Fix AppArmor compatibility
./fix-docker-apparmor.sh 10.10.0.220
# 4. Start containers
ssh cal@10.10.0.220 "cd /home/cal/container-data/app-config && docker compose up -d"
```
**Existing LXC with Docker Issues**:
```bash
# If containers failing to start in LXC:
./fix-docker-apparmor.sh <LXC_IP>
# Restart affected containers
ssh cal@<LXC_IP> "cd /home/cal/container-data && docker compose restart"
```
### SSH Key Integration
**Both provisioning methods use**:
- **Primary Key**: `~/.ssh/homelab_rsa` - Daily use authentication
- **Emergency Key**: `~/.ssh/emergency_homelab_rsa` - Backup access
**Security Configuration**:
- Password authentication completely disabled after provisioning
- Only key-based SSH access allowed
- Emergency keys provide backup access if primary key fails
- Automatic security updates enabled
**Key Management Integration**:
- Keys managed by `/networking/scripts/ssh_key_maintenance.sh`
- Monthly backups of all SSH keys
- Rotation recommendations for keys > 365 days old
## Configuration Dependencies
### Required Local Files
- `~/.ssh/homelab_rsa` - Primary SSH private key
- `~/.ssh/homelab_rsa.pub` - Primary SSH public key
- `~/.ssh/emergency_homelab_rsa` - Emergency SSH private key
- `~/.ssh/emergency_homelab_rsa.pub` - Emergency SSH public key
### Target VM Requirements
- **For post-install script**: SSH enabled, initial authentication method available
- **For cloud-init**: Proxmox cloud-init support, fresh VM
- **For LXC**: Proxmox host with LXC support
### Network Requirements
- VMs/LXCs on 10.10.0.0/24 network (homelab subnet)
- SSH access (port 22) to target systems
- Internet access on target systems for package installation
## Troubleshooting Context
### Common Issues
**1. vm-post-install.sh Connection Failures**
```bash
# Verify VM is accessible
ping <vm-ip>
nc -z <vm-ip> 22
# Check SSH service on target
ssh <vm-ip> "systemctl status sshd"
# Verify SSH keys exist locally
ls -la ~/.ssh/homelab_rsa*
```
**2. Cloud-Init Not Working**
```bash
# On Proxmox host, check cloud-init support
qm cloudinit dump <vmid>
# On VM, check cloud-init logs
sudo cloud-init status --long
sudo cat /var/log/cloud-init.log
```
**3. Docker Containers Fail in LXC**
```bash
# Symptom: Containers won't start, permission errors
# Solution: Run AppArmor fix
./fix-docker-apparmor.sh <LXC_IP>
# Verify security_opt was added
ssh cal@<LXC_IP> "grep -r 'security_opt' ~/container-data/"
# Check Docker logs
ssh cal@<LXC_IP> "docker compose logs"
```
**4. SSH Key Authentication Fails After Provisioning**
```bash
# Verify key permissions
ls -la ~/.ssh/homelab_rsa
chmod 600 ~/.ssh/homelab_rsa
# Check authorized_keys on target
ssh <vm-ip> "cat ~/.ssh/authorized_keys"
# Test with verbose output
ssh -v cal@<vm-ip>
```
**5. Docker Installation Issues**
```bash
# Check internet connectivity on VM
ssh <vm-ip> "ping -c 3 8.8.8.8"
# Verify Docker GPG key
ssh <vm-ip> "apt-key list | grep -i docker"
# Check Docker service
ssh <vm-ip> "systemctl status docker"
# Manual Docker install if needed
ssh <vm-ip> "curl -fsSL https://get.docker.com | sh"
```
### Diagnostic Commands
```bash
# Post-provisioning validation
ssh cal@<vm-ip> "groups" # Should include: sudo docker
ssh cal@<vm-ip> "docker run --rm hello-world"
ssh cal@<vm-ip> "sudo sshd -T | grep passwordauth" # Should be "no"
# Cloud-init status check
ssh cal@<vm-ip> "cloud-init status"
ssh cal@<vm-ip> "cloud-init query -f '{{ds.meta_data.hostname}}'"
# Docker in LXC verification
ssh cal@<LXC_IP> "docker info | grep -i apparmor"
ssh cal@<LXC_IP> "docker compose config" # Validate compose files
# SSH key connectivity test
ssh -o ConnectTimeout=5 cal@<vm-ip> "echo 'SSH OK'"
```
## Integration Points
### External Dependencies
- **Proxmox VE**: For VM/LXC creation and cloud-init support
- **SSH**: For remote provisioning and management
- **Docker**: Installed on target systems
- **Cloud-init**: For automated VM provisioning
- **AppArmor**: Security framework (configured for LXC compatibility)
### File System Dependencies
- **Script Directory**: `/mnt/NV2/Development/claude-home/vm-management/scripts/`
- **SSH Keys**: `~/.ssh/homelab_rsa*`, `~/.ssh/emergency_homelab_rsa*`
- **LXC Compose Directories**: Typically `/home/cal/container-data/` on target
- **Backup Files**: `.bak` files created by AppArmor fix script
### Network Dependencies
- **Management Network**: 10.10.0.0/24 subnet
- **Internet Access**: Required for package installation
- **Proxmox API**: For LXC creation operations
- **DNS**: For hostname resolution
## Security Considerations
### SSH Security
- Password authentication disabled after provisioning
- Only key-based authentication allowed
- Emergency keys provide backup access
- Root login disabled
### Docker Security
- User in docker group (no sudo needed for docker commands)
- AppArmor unconfined in LXC (required for functionality)
- Containers run as non-root when possible
- Network isolation via Docker networks
### VM/LXC Security
- Automatic security updates enabled
- Minimal package installation (only essentials)
- Firewall configuration recommended post-provisioning
- Regular key rotation via SSH key maintenance
## Performance Considerations
### Cloud-Init vs Post-Install
- **Cloud-init**: Faster (0-touch), no manual SSH needed, better for multiple VMs
- **Post-install**: More flexible, works with existing VMs, easier debugging
### LXC vs VM
- **LXC**: Lower overhead, faster startup, shared kernel
- **VM**: Better isolation, GPU passthrough support, different kernels possible
### Docker in LXC
- **Performance**: Near-native, minimal overhead with AppArmor disabled
- **I/O**: Use local storage for best performance, NFS for shared data
- **Networking**: Bridge mode for simplicity, macvlan for direct network access
## Related Documentation
- **Technology Overview**: `/vm-management/CONTEXT.md`
- **Troubleshooting**: `/vm-management/troubleshooting.md`
- **Examples**: `/vm-management/examples/` - Configuration templates
- **SSH Management**: `/networking/scripts/ssh_key_maintenance.sh`
- **Docker Patterns**: `/docker/CONTEXT.md`
- **Main Instructions**: `/CLAUDE.md` - Context loading rules
## Notes
These scripts form the foundation of the homelab VM and LXC provisioning strategy. They ensure consistent configuration, security hardening, and Docker compatibility across all virtualized infrastructure.
The cloud-init approach is preferred for new deployments due to zero-touch provisioning, while the post-install script provides flexibility for existing systems or troubleshooting scenarios.
AppArmor configuration is critical for Docker-in-LXC deployments and should be applied to all LXC containers running Docker to prevent container startup failures.
Reference in New Issue View Git Blame Copy Permalink