claude-home/patterns/docker/distributed-transcoding.md

# Tdarr Distributed Transcoding Pattern

## Overview
Tdarr distributed transcoding with unmapped nodes provides optimal performance for enterprise-scale video processing across multiple machines.

## Architecture Pattern

### Unmapped Node Deployment (Recommended)
```
┌─────────────────┐    ┌──────────────────────────────────┐
│   Tdarr Server  │    │        Unmapped Nodes            │
│                 │    │  ┌─────────┐ ┌─────────┐         │
│  - Web Interface│◄──►│  │ Node 1  │ │ Node 2  │ ...     │
│  - Job Queue    │    │  │ GPU+CPU │ │ GPU+CPU │         │
│  - File Mgmt    │    │  │NVMe Cache│ │NVMe Cache│         │
│                 │    │  └─────────┘ └─────────┘         │
└─────────────────┘    └──────────────────────────────────┘
         │                              │
         └──────── Shared Storage ──────┘
              (NAS/SAN for media files)
```

### Key Components
- **Server**: Centralizes job management and web interface
- **Unmapped Nodes**: Independent transcoding with local cache
- **Shared Storage**: Source and final file repository

## Configuration Templates

### Server Configuration
```yaml
# docker-compose.yml
version: "3.4"
services:
  tdarr-server:
    image: ghcr.io/haveagitgat/tdarr:latest
    ports:
      - "8265:8265"  # Web UI
      - "8266:8266"  # Server API
    environment:
      - TZ=America/Chicago
      - serverIP=0.0.0.0
      - serverPort=8266
      - webUIPort=8265
    volumes:
      - ./server:/app/server
      - ./configs:/app/configs  
      - ./logs:/app/logs
      - /path/to/media:/media
    # Note: No temp/cache volume needed for server with unmapped nodes
```

### Unmapped Node Configuration  
```bash
#!/bin/bash
# Optimal unmapped node with local NVMe cache

podman run -d --name "tdarr-node-1" \
    --gpus all \
    -e TZ=America/Chicago \
    -e nodeName="transcoding-node-1" \
    -e serverIP="10.10.0.43" \
    -e serverPort="8266" \
    -e nodeType=unmapped \
    -e inContainer=true \
    -e ffmpegVersion=6 \
    -e NVIDIA_DRIVER_CAPABILITIES=all \
    -e NVIDIA_VISIBLE_DEVICES=all \
    -v "/mnt/nvme/tdarr-cache:/cache" \
    ghcr.io/haveagitgat/tdarr_node:latest
```

## Performance Optimization

### Cache Storage Strategy
```bash
# Optimal cache storage hierarchy
/mnt/nvme/tdarr-cache/              # NVMe SSD (fastest)
├── tdarr-workDir-{jobId}/          # Active transcoding  
├── download/                       # Source file staging
└── upload/                         # Result file staging

# Alternative: RAM disk for ultra-performance (limited size)
/dev/shm/tdarr-cache/               # RAM disk (fastest, volatile)

# Avoid: Network mounted cache (slowest)
/mnt/nas/tdarr-cache/               # Network storage (not recommended)
```

### Network I/O Pattern
```
Optimized Workflow:
1. 📥 Download source (once): NAS → Local NVMe
2. ⚡ Transcode: Local NVMe → Local NVMe  
3. 📤 Upload result (once): Local NVMe → NAS

vs Legacy Mapped Workflow:
1. 🐌 Read source: NAS → Node (streaming)
2. 🐌 Write temp: Node → NAS (streaming) 
3. 🐌 Read temp: NAS → Node (streaming)
4. 🐌 Write final: Node → NAS (streaming)
```

## Scaling Patterns

### Horizontal Scaling
```yaml
# Multiple nodes with load balancing
nodes:
  - name: "gpu-node-1"      # RTX 4090 + NVMe
    role: "heavy-transcode"
  - name: "gpu-node-2"      # RTX 3080 + NVMe  
    role: "standard-transcode"
  - name: "cpu-node-1"      # Multi-core + SSD
    role: "audio-processing"
```

### Resource Specialization
```bash
# GPU-optimized node
-e hardwareEncoding=true
-e nvencTemporalAQ=1
-e processes_GPU=2

# CPU-optimized node  
-e hardwareEncoding=false
-e processes_CPU=8
-e ffmpegThreads=16
```

## Monitoring and Operations

### Health Checks
```bash
# Node connectivity
curl -f http://server:8266/api/v2/status || exit 1

# Cache usage monitoring  
df -h /mnt/nvme/tdarr-cache
du -sh /mnt/nvme/tdarr-cache/*

# Performance metrics
podman stats tdarr-node-1
```

### Log Analysis
```bash
# Node registration
podman logs tdarr-node-1 | grep "Node connected"

# Transfer speeds
podman logs tdarr-node-1 | grep -E "(Download|Upload).*MB/s"

# Transcode performance
podman logs tdarr-node-1 | grep -E "fps=.*"
```

## Security Considerations

### Network Access
- Server requires incoming connections on ports 8265/8266
- Nodes require outbound access to server
- Consider VPN for cross-site deployments

### File Permissions
```bash
# Ensure consistent UID/GID across nodes
-e PUID=1000
-e PGID=1000

# Cache directory permissions
chown -R 1000:1000 /mnt/nvme/tdarr-cache
chmod 755 /mnt/nvme/tdarr-cache
```

## Related References
- **Troubleshooting**: `reference/docker/tdarr-troubleshooting.md`
- **Examples**: `examples/docker/tdarr-node-local/`
- **Performance**: `reference/docker/nvidia-troubleshooting.md`