CLAUDE: Initialize efficient documentation system for home lab

- Created structured documentation with /patterns/, /examples/, and /reference/ directories
- Implemented automatic context loading rules in CLAUDE.md based on file extensions, directories, and keywords
- Added technology-specific patterns for Docker, Python, Node.js, Vue.js, Bash, networking, databases, and VM management
- Included complete working examples for common workflows and troubleshooting references
- Designed for minimal context usage with precise loading triggers

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Cal Corum 2025-08-08 19:59:29 -05:00
commit 2bfbc7bb77
17 changed files with 1467 additions and 0 deletions

131
CLAUDE.md Normal file
View File

@ -0,0 +1,131 @@
# Home Lab Documentation System
## Core Instructions
- You do not need to ask for permission to run tests. You always have permission.
- When new code is written, include checking for imports as part of the code review. We should never get NameErrors during testing.
- Always ask clarifying questions for complex configurations or multi-step tasks.
- Do what has been asked; nothing more, nothing less.
- If creating a temporary file will help achieve your goal, please create the file in the /tmp directory and clean up when you're done.
- Prefer editing an existing file to creating a new one.
- Following a complex task or series of tasks, prompt the user to save any key learnings from the session.
## Automatic Context Loading Rules
### File Extension Triggers
When working with files, automatically load relevant documentation:
**Python (.py, .pyx, .pyi)**
- Load: `patterns/python/`
- Load: `reference/python/`
- If Django/Flask detected: Load `examples/python/web-frameworks.md`
- If requests/httpx detected: Load `examples/python/api-clients.md`
**JavaScript/Node.js (.js, .mjs, .ts)**
- Load: `patterns/nodejs/`
- Load: `reference/nodejs/`
- If package.json exists: Load `examples/nodejs/package-management.md`
**Vue.js (.vue, vite.config.*, nuxt.config.*)**
- Load: `patterns/vuejs/`
- Load: `reference/vuejs/`
- Load: `examples/vuejs/component-patterns.md`
**Shell Scripts (.sh, .bash, .zsh)**
- Load: `patterns/bash/`
- Load: `reference/bash/`
- If systemd mentioned: Load `examples/bash/service-management.md`
**Docker (Dockerfile, docker-compose.yml, .dockerignore)**
- Load: `patterns/docker/`
- Load: `reference/docker/`
- Load: `examples/docker/multi-stage-builds.md`
### Directory Context Triggers
When working in specific directories:
**Docker-related directories (/docker/, /containers/, /compose/)**
- Load: `patterns/docker/`
- Load: `examples/docker/`
- Load: `reference/docker/troubleshooting.md`
**Database directories (/db/, /database/, /mysql/, /postgres/, /mongo/)**
- Load: `patterns/databases/`
- Load: `examples/databases/`
- Load: `reference/databases/`
**Network directories (/network/, /networking/, /nginx/, /traefik/)**
- Load: `patterns/networking/`
- Load: `examples/networking/`
- Load: `reference/networking/troubleshooting.md`
**VM directories (/vm/, /virtual/, /proxmox/, /kvm/)**
- Load: `patterns/vm-management/`
- Load: `examples/vm-management/`
- Load: `reference/vm-management/`
### Keyword Triggers
When user mentions specific terms, automatically load relevant docs:
**Troubleshooting Keywords**
- "debug", "error", "fail", "broken", "not working", "issue"
- Load: `reference/{relevant-tech}/troubleshooting.md`
- Load: `examples/{relevant-tech}/debugging.md`
**Configuration Keywords**
- "config", "configure", "setup", "install", "deploy"
- Load: `patterns/{relevant-tech}/`
- Load: `examples/{relevant-tech}/configuration.md`
**Performance Keywords**
- "slow", "performance", "optimize", "memory", "cpu"
- Load: `reference/{relevant-tech}/performance.md`
- Load: `examples/{relevant-tech}/optimization.md`
**Security Keywords**
- "secure", "ssl", "tls", "certificate", "auth", "firewall"
- Load: `patterns/networking/security.md`
- Load: `reference/networking/security.md`
**Database Keywords**
- "database", "db", "sql", "mysql", "postgres", "mongo", "redis"
- Load: `patterns/databases/`
- Load: `examples/databases/`
**Container Keywords**
- "docker", "container", "image", "compose", "kubernetes", "k8s"
- Load: `patterns/docker/`
- Load: `examples/docker/`
**Network Keywords**
- "network", "nginx", "proxy", "load balancer", "dns", "port", "firewall"
- Load: `patterns/networking/`
- Load: `examples/networking/`
**VM Keywords**
- "virtual machine", "vm", "proxmox", "kvm", "hypervisor", "guest"
- Load: `patterns/vm-management/`
- Load: `examples/vm-management/`
### Priority Rules
1. **File extension triggers** take highest priority
2. **Directory context** takes second priority
3. **Keyword triggers** are additive and load supplementary context
4. If multiple technologies detected, load all relevant patterns
5. Always prefer specific over general (e.g., `vuejs/` over `nodejs/`)
### Context Loading Behavior
- Load pattern files first for overview
- Load relevant examples for implementation details
- Load reference files for troubleshooting and edge cases
- Maximum of 3 documentation files per trigger to maintain efficiency
- If context becomes too large, prioritize most recent/specific files
## Documentation Structure
```
/patterns/ # Technology overviews and best practices
/examples/ # Complete working implementations
/reference/ # Troubleshooting, cheat sheets, fallback info
```
Each pattern file should reference relevant examples and reference materials.

View File

@ -0,0 +1,145 @@
# Bash Service Management Examples
## Systemd Service Script
```bash
#!/bin/bash
set -euo pipefail
SERVICE_NAME="myapp"
SERVICE_FILE="/etc/systemd/system/${SERVICE_NAME}.service"
usage() {
echo "Usage: $0 {install|start|stop|restart|status|logs|uninstall}"
exit 1
}
install_service() {
echo "Installing ${SERVICE_NAME} service..."
sudo tee "$SERVICE_FILE" > /dev/null << EOF
[Unit]
Description=My Application Service
After=network.target
Wants=network.target
[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/bin/myapp
Restart=always
RestartSec=10
Environment=NODE_ENV=production
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl daemon-reload
sudo systemctl enable "$SERVICE_NAME"
echo "Service installed and enabled"
}
manage_service() {
local action="$1"
echo "${action^}ing ${SERVICE_NAME} service..."
sudo systemctl "$action" "$SERVICE_NAME"
echo "Service ${action}ed successfully"
}
show_status() {
echo "=== Service Status ==="
sudo systemctl status "$SERVICE_NAME" --no-pager -l
echo
echo "=== Service Logs (last 20 lines) ==="
sudo journalctl -u "$SERVICE_NAME" --no-pager -n 20
}
show_logs() {
echo "Following logs for ${SERVICE_NAME}..."
sudo journalctl -u "$SERVICE_NAME" -f
}
uninstall_service() {
echo "Uninstalling ${SERVICE_NAME} service..."
sudo systemctl stop "$SERVICE_NAME" || true
sudo systemctl disable "$SERVICE_NAME" || true
sudo rm -f "$SERVICE_FILE"
sudo systemctl daemon-reload
echo "Service uninstalled"
}
case "${1:-}" in
install)
install_service
;;
start|stop|restart)
manage_service "$1"
;;
status)
show_status
;;
logs)
show_logs
;;
uninstall)
uninstall_service
;;
*)
usage
;;
esac
```
## Process Monitoring Script
```bash
#!/bin/bash
set -euo pipefail
PROCESS_NAME="myapp"
RESTART_COMMAND="/opt/myapp/start.sh"
LOG_FILE="/var/log/myapp-monitor.log"
PID_FILE="/var/run/myapp.pid"
log_message() {
echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}
check_process() {
if [[ -f "$PID_FILE" ]]; then
local pid=$(cat "$PID_FILE")
if kill -0 "$pid" 2>/dev/null; then
return 0 # Process is running
else
rm -f "$PID_FILE"
return 1 # Process is not running
fi
else
return 1 # PID file doesn't exist
fi
}
restart_process() {
log_message "Attempting to restart $PROCESS_NAME..."
if $RESTART_COMMAND; then
log_message "$PROCESS_NAME restarted successfully"
else
log_message "Failed to restart $PROCESS_NAME"
exit 1
fi
}
main() {
if ! check_process; then
log_message "$PROCESS_NAME is not running"
restart_process
else
log_message "$PROCESS_NAME is running normally"
fi
}
main "$@"
```

View File

@ -0,0 +1,51 @@
# Multi-Stage Docker Builds
## Basic Multi-Stage Pattern
```dockerfile
# Build stage
FROM node:18-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
# Production stage
FROM node:18-alpine AS production
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY . .
EXPOSE 3000
CMD ["npm", "start"]
```
## Python with Virtual Environment
```dockerfile
# Build stage
FROM python:3.11-slim AS builder
WORKDIR /app
COPY requirements.txt .
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
RUN pip install --no-cache-dir -r requirements.txt
# Production stage
FROM python:3.11-slim AS production
WORKDIR /app
COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY . .
CMD ["python", "app.py"]
```
## Benefits
- Smaller final image size
- Separation of build and runtime dependencies
- Better caching strategies
- Security (no build tools in production image)
## Best Practices
- Use specific base image versions
- Copy only necessary files to production stage
- Run as non-root user in production stage
- Use .dockerignore to exclude unnecessary files

View File

@ -0,0 +1,124 @@
# Nginx Configuration Examples
## Reverse Proxy with SSL
```nginx
# /etc/nginx/sites-available/myapp
server {
listen 80;
server_name myapp.example.com;
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2;
server_name myapp.example.com;
# SSL Configuration
ssl_certificate /etc/letsencrypt/live/myapp.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/myapp.example.com/privkey.pem;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!aNULL:!MD5;
ssl_prefer_server_ciphers on;
# Security Headers
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
# Logging
access_log /var/log/nginx/myapp.access.log;
error_log /var/log/nginx/myapp.error.log;
# Proxy to backend
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_cache_bypass $http_upgrade;
# Timeouts
proxy_connect_timeout 60s;
proxy_send_timeout 60s;
proxy_read_timeout 60s;
}
# Static files
location /static/ {
alias /var/www/myapp/static/;
expires 30d;
add_header Cache-Control "public, immutable";
}
# Health check endpoint
location /health {
access_log off;
return 200 "healthy\n";
add_header Content-Type text/plain;
}
}
```
## Load Balancing Configuration
```nginx
# /etc/nginx/conf.d/upstream.conf
upstream backend {
least_conn;
server 10.0.1.10:3000 weight=3 max_fails=3 fail_timeout=30s;
server 10.0.1.11:3000 weight=3 max_fails=3 fail_timeout=30s;
server 10.0.1.12:3000 weight=1 backup;
}
server {
listen 80;
server_name api.example.com;
location / {
proxy_pass http://backend;
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503;
proxy_next_upstream_tries 3;
proxy_next_upstream_timeout 30s;
# Standard proxy headers
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
```
## Rate Limiting
```nginx
# /etc/nginx/nginx.conf (in http block)
http {
# Rate limiting zones
limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
limit_req_zone $binary_remote_addr zone=login:10m rate=1r/s;
# Connection limiting
limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;
}
# In server block
server {
location /api/ {
limit_req zone=api burst=20 nodelay;
limit_conn conn_limit_per_ip 10;
proxy_pass http://backend;
}
location /auth/login {
limit_req zone=login burst=5;
proxy_pass http://backend;
}
}
```

View File

@ -0,0 +1,96 @@
# Python Web Framework Examples
## Flask API Structure
```python
# app.py
from flask import Flask, jsonify, request
from flask_sqlalchemy import SQLAlchemy
import logging
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///app.db'
db = SQLAlchemy(app)
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True, nullable=False)
email = db.Column(db.String(120), unique=True, nullable=False)
@app.route('/api/users', methods=['GET'])
def get_users():
try:
users = User.query.all()
return jsonify([{'id': u.id, 'username': u.username, 'email': u.email} for u in users])
except Exception as e:
logger.error(f"Error fetching users: {e}")
return jsonify({'error': 'Internal server error'}), 500
@app.route('/api/users', methods=['POST'])
def create_user():
try:
data = request.get_json()
user = User(username=data['username'], email=data['email'])
db.session.add(user)
db.session.commit()
return jsonify({'id': user.id}), 201
except Exception as e:
logger.error(f"Error creating user: {e}")
return jsonify({'error': 'Bad request'}), 400
if __name__ == '__main__':
with app.app_context():
db.create_all()
app.run(debug=True)
```
## FastAPI Async Pattern
```python
# main.py
from fastapi import FastAPI, HTTPException, Depends
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine
from sqlalchemy.orm import sessionmaker
import logging
app = FastAPI()
# Database setup
DATABASE_URL = "sqlite+aiosqlite:///./app.db"
engine = create_async_engine(DATABASE_URL)
AsyncSessionLocal = sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
async def get_db():
async with AsyncSessionLocal() as session:
yield session
@app.get("/api/users")
async def get_users(db: AsyncSession = Depends(get_db)):
try:
# Database query logic here
return {"users": []}
except Exception as e:
logging.error(f"Error: {e}")
raise HTTPException(status_code=500, detail="Internal server error")
```
## Configuration Management
```python
# config.py
import os
from dataclasses import dataclass
@dataclass
class Config:
SECRET_KEY: str = os.getenv('SECRET_KEY', 'dev-secret-key')
DATABASE_URL: str = os.getenv('DATABASE_URL', 'sqlite:///app.db')
DEBUG: bool = os.getenv('DEBUG', 'False').lower() == 'true'
LOG_LEVEL: str = os.getenv('LOG_LEVEL', 'INFO')
config = Config()
```

32
patterns/bash/README.md Normal file
View File

@ -0,0 +1,32 @@
# Bash Scripting Patterns
## Script Structure
- Use proper shebang (`#!/bin/bash`)
- Set error handling (`set -euo pipefail`)
- Include help/usage functions
- Implement proper argument parsing
## Best Practices
- **Input validation** before processing
- **Logging and error messages** to stderr
- **Exit codes** for script status
- **Function-based organization** for reusability
## Common Patterns
- **Service management** with systemd integration
- **Backup scripts** with rotation and compression
- **Deployment automation** with rollback capabilities
- **System monitoring** with alerting
## Security Considerations
- Avoid hardcoded credentials
- Use proper file permissions
- Sanitize user inputs
- Quote variables to prevent word splitting
## Related Documentation
- Examples: `/examples/bash/service-management.md`
- Examples: `/examples/bash/backup-scripts.md`
- Examples: `/examples/bash/deployment-automation.md`
- Reference: `/reference/bash/troubleshooting.md`
- Reference: `/reference/bash/security.md`

View File

@ -0,0 +1,32 @@
# Database Patterns
## Database Design
- **Normalization** vs **denormalization** strategies
- **Indexing** for query performance
- **Schema versioning** and migrations
- **Data modeling** for specific use cases
## Performance Optimization
- **Connection pooling** and management
- **Query optimization** and execution plans
- **Caching layers** (Redis, Memcached)
- **Read replicas** and load distribution
## Backup & Recovery
- **Automated backup** scheduling
- **Point-in-time recovery** strategies
- **Cross-region replication** for disaster recovery
- **Backup testing** and validation
## Security & Access Control
- **User privilege management** and roles
- **Data encryption** at rest and in transit
- **Audit logging** and compliance
- **Network security** and access restrictions
## Related Documentation
- Examples: `/examples/databases/mysql-setup.md`
- Examples: `/examples/databases/postgres-optimization.md`
- Examples: `/examples/databases/redis-caching.md`
- Reference: `/reference/databases/troubleshooting.md`
- Reference: `/reference/databases/performance.md`

26
patterns/docker/README.md Normal file
View File

@ -0,0 +1,26 @@
# Docker Patterns
## Container Best Practices
- Use multi-stage builds for production images
- Minimize layer count and image size
- Run containers as non-root users
- Use specific version tags, avoid `latest`
- Implement health checks
## Common Patterns
- **Multi-service applications**: Use docker-compose for local development
- **Production deployments**: Single-container per service with orchestration
- **Development environments**: Volume mounts for code changes
- **CI/CD integration**: Build, test, and push in pipeline stages
## Security Considerations
- Scan images for vulnerabilities
- Use distroless or minimal base images
- Implement resource limits
- Network isolation between services
## Related Documentation
- Examples: `/examples/docker/multi-stage-builds.md`
- Examples: `/examples/docker/compose-patterns.md`
- Reference: `/reference/docker/troubleshooting.md`
- Reference: `/reference/docker/security-checklist.md`

View File

@ -0,0 +1,32 @@
# Networking Patterns
## Infrastructure Setup
- **Reverse proxy** configuration (Nginx/Traefik)
- **Load balancing** strategies and health checks
- **SSL/TLS termination** and certificate management
- **Network segmentation** and VLANs
## Service Discovery
- **DNS-based** service resolution
- **Container networking** with Docker networks
- **Service mesh** patterns for microservices
- **API gateway** implementation
## Security Patterns
- **Firewall rules** and port management
- **VPN setup** for remote access
- **Zero-trust networking** principles
- **Network monitoring** and intrusion detection
## Performance Optimization
- **CDN integration** for static assets
- **Connection pooling** and keep-alives
- **Bandwidth management** and QoS
- **Caching strategies** at network level
## Related Documentation
- Examples: `/examples/networking/nginx-config.md`
- Examples: `/examples/networking/vpn-setup.md`
- Examples: `/examples/networking/load-balancing.md`
- Reference: `/reference/networking/troubleshooting.md`
- Reference: `/reference/networking/security.md`

32
patterns/nodejs/README.md Normal file
View File

@ -0,0 +1,32 @@
# Node.js Patterns
## Project Setup
- Use package.json with proper scripts
- Implement environment-based configuration
- Use ES modules or consistent CommonJS
- Proper dependency management (dev vs prod)
## Application Architecture
- **Express.js patterns** for REST APIs
- **Middleware chains** for request processing
- **Event-driven architecture** for scalability
- **Microservices** with proper service boundaries
## Error Handling
- Centralized error handling middleware
- Proper async/await error catching
- Process-level error handling
- Graceful shutdown patterns
## Performance & Security
- Connection pooling for databases
- Rate limiting and input validation
- Environment variable management
- Dependency vulnerability scanning
## Related Documentation
- Examples: `/examples/nodejs/express-api.md`
- Examples: `/examples/nodejs/microservices.md`
- Examples: `/examples/nodejs/package-management.md`
- Reference: `/reference/nodejs/debugging.md`
- Reference: `/reference/nodejs/security.md`

32
patterns/python/README.md Normal file
View File

@ -0,0 +1,32 @@
# Python Patterns
## Project Structure
- Use virtual environments (venv)
- Implement proper package structure with `__init__.py`
- Separate configuration from code
- Use requirements.txt for dependencies
## Code Organization
- **MVC/MVT patterns** for web applications
- **Factory pattern** for object creation
- **Context managers** for resource handling
- **Async/await** for I/O-bound operations
## Error Handling
- Use specific exception types
- Implement proper logging with levels
- Graceful degradation for external dependencies
- Validation at API boundaries
## Performance Considerations
- Use generators for large datasets
- Profile before optimizing
- Leverage caching appropriately
- Consider async for concurrent operations
## Related Documentation
- Examples: `/examples/python/web-frameworks.md`
- Examples: `/examples/python/api-clients.md`
- Examples: `/examples/python/async-patterns.md`
- Reference: `/reference/python/debugging.md`
- Reference: `/reference/python/performance.md`

View File

@ -0,0 +1,32 @@
# Virtual Machine Management Patterns
## VM Provisioning
- **Template-based deployment** for consistency
- **Resource allocation** sizing and planning
- **Network configuration** and VLAN assignment
- **Storage provisioning** and disk management
## Lifecycle Management
- **Automated provisioning** with infrastructure as code
- **Configuration management** with Ansible/Puppet
- **Snapshot management** and rollback strategies
- **Scaling policies** for resource optimization
## Monitoring & Maintenance
- **Resource monitoring** (CPU, memory, disk, network)
- **Health checks** and alerting systems
- **Patch management** and update strategies
- **Performance tuning** and optimization
## Backup & Recovery
- **VM-level backups** vs **application-level backups**
- **Disaster recovery** planning and testing
- **High availability** configurations
- **Migration strategies** between hosts
## Related Documentation
- Examples: `/examples/vm-management/proxmox-automation.md`
- Examples: `/examples/vm-management/ansible-provisioning.md`
- Examples: `/examples/vm-management/backup-strategies.md`
- Reference: `/reference/vm-management/troubleshooting.md`
- Reference: `/reference/vm-management/performance.md`

32
patterns/vuejs/README.md Normal file
View File

@ -0,0 +1,32 @@
# Vue.js Patterns
## Component Architecture
- Single File Components (SFC) structure
- Composition API for complex logic
- Props down, events up communication
- Provide/inject for deep hierarchies
## State Management
- **Pinia** for application state
- **Local state** for component-specific data
- **Computed properties** for derived state
- **Watchers** for side effects
## Performance Optimization
- Lazy loading with dynamic imports
- Virtual scrolling for large lists
- Proper key usage in v-for
- Component-level code splitting
## Development Patterns
- TypeScript integration
- Testing with Vitest/Jest
- Build optimization with Vite
- Progressive Web App features
## Related Documentation
- Examples: `/examples/vuejs/component-patterns.md`
- Examples: `/examples/vuejs/state-management.md`
- Examples: `/examples/vuejs/performance-optimization.md`
- Reference: `/reference/vuejs/debugging.md`
- Reference: `/reference/vuejs/best-practices.md`

View File

@ -0,0 +1,245 @@
# Bash Script Troubleshooting Reference
## Script Debugging
### Enable debugging modes
```bash
#!/bin/bash
set -euo pipefail # Exit on error, undefined vars, pipe failures
set -x # Print commands as they're executed (xtrace)
# Or run with debug flags
bash -x script.sh
bash -n script.sh # Check syntax without execution
```
### Common error handling
```bash
#!/bin/bash
set -euo pipefail
# Function to handle errors
error_handler() {
echo "Error occurred in script at line: $1" >&2
echo "Command that failed: $2" >&2
exit 1
}
# Set error trap
trap 'error_handler ${LINENO} "$BASH_COMMAND"' ERR
# Example with error checking
if ! command -v docker >/dev/null 2>&1; then
echo "Error: docker is not installed" >&2
exit 1
fi
```
## Variable and Parameter Issues
### Check if variables are set
```bash
# Check if variable is set and not empty
if [[ -z "${VAR:-}" ]]; then
echo "VAR is not set or empty"
exit 1
fi
# Set default values
VAR="${VAR:-default_value}"
VAR="${1:-default_value}" # For parameters
# Required variables
: "${REQUIRED_VAR:?Error: REQUIRED_VAR must be set}"
```
### Array debugging
```bash
# Declare array properly
declare -a array_name
array_name=("item1" "item2" "item3")
# Check array contents
declare -p array_name
printf '%s\n' "${array_name[@]}"
# Array length
echo "Array length: ${#array_name[@]}"
```
## File and Directory Issues
### File existence and permissions
```bash
# Check file existence
if [[ ! -f "$filename" ]]; then
echo "File $filename does not exist"
exit 1
fi
# Check directory existence
if [[ ! -d "$dirname" ]]; then
echo "Directory $dirname does not exist"
mkdir -p "$dirname"
fi
# Check permissions
if [[ ! -r "$filename" ]]; then
echo "File $filename is not readable"
exit 1
fi
if [[ ! -w "$filename" ]]; then
echo "File $filename is not writable"
exit 1
fi
```
### Path issues
```bash
# Get absolute path
filename="$(realpath "$1")"
dirname="$(dirname "$filename")"
basename="$(basename "$filename")"
# Change to script directory
cd "$(dirname "$0")"
```
## Process and Command Issues
### Command existence checking
```bash
# Check if command exists
if ! command -v docker >/dev/null 2>&1; then
echo "docker is required but not installed"
exit 1
fi
# Alternative method
if ! which docker >/dev/null 2>&1; then
echo "docker not found in PATH"
exit 1
fi
```
### Process management
```bash
# Check if process is running
if pgrep -x "process_name" >/dev/null; then
echo "Process is running"
else
echo "Process is not running"
fi
# Kill process safely
if pgrep -x "process_name" >/dev/null; then
pkill -x "process_name"
sleep 2
if pgrep -x "process_name" >/dev/null; then
pkill -9 -x "process_name"
fi
fi
```
## String and Text Processing
### String comparison issues
```bash
# Safe string comparison
string1="test"
string2=""
# Always quote variables
if [[ "$string1" == "test" ]]; then
echo "Match"
fi
# Check for empty strings
if [[ -z "$string2" ]]; then
echo "String is empty"
fi
if [[ -n "$string1" ]]; then
echo "String is not empty"
fi
```
### Input validation
```bash
# Validate numeric input
if ! [[ "$input" =~ ^[0-9]+$ ]]; then
echo "Error: Input must be a number"
exit 1
fi
# Validate email format (basic)
if ! [[ "$email" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$ ]]; then
echo "Error: Invalid email format"
exit 1
fi
```
## System Resource Issues
### Disk space checking
```bash
# Check available disk space
available=$(df / | awk 'NR==2 {print $4}')
if [[ $available -lt 1000000 ]]; then # Less than 1GB
echo "Warning: Low disk space"
fi
# Check if directory has enough space
required_space=1000000 # 1GB in KB
available=$(df "$directory" | awk 'NR==2 {print $4}')
if [[ $available -lt $required_space ]]; then
echo "Error: Not enough disk space"
exit 1
fi
```
### Memory checking
```bash
# Check available memory
available_mem=$(free | awk 'NR==2 {print $7}')
if [[ $available_mem -lt 1000000 ]]; then # Less than 1GB
echo "Warning: Low available memory"
fi
```
## Debugging Techniques
### Add debug output
```bash
# Debug function
debug() {
if [[ "${DEBUG:-}" == "1" ]]; then
echo "DEBUG: $*" >&2
fi
}
# Usage
debug "Processing file: $filename"
```
### Temporary file handling
```bash
# Create temporary file safely
tmpfile=$(mktemp)
trap 'rm -f "$tmpfile"' EXIT
# Create temporary directory
tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT
```
### Script execution tracing
```bash
# Trace specific functions
trace_function() {
set -x
your_function_here
set +x
}
```

View File

@ -0,0 +1,87 @@
# Docker Troubleshooting Reference
## Container Won't Start
### Check logs
```bash
docker logs <container_name>
docker logs --tail 50 -f <container_name>
```
### Common issues
- **Port already in use**: `docker ps` to find conflicting containers
- **Permission errors**: Check file ownership and container user
- **Missing environment variables**: Verify with `docker exec -it <container> env`
- **Resource limits**: Check memory/CPU constraints
### Debug running container
```bash
docker exec -it <container> /bin/bash
docker exec -it <container> /bin/sh # if bash not available
```
## Build Issues
### Clear build cache
```bash
docker system prune -a
docker builder prune
```
### Build with verbose output
```bash
docker build --progress=plain --no-cache .
```
### Common build failures
- **COPY/ADD errors**: Check file paths and .dockerignore
- **Package installation**: Update package lists first
- **Network issues**: Check DNS and proxy settings
## Performance Issues
### Container resource usage
```bash
docker stats
docker exec <container> top
docker exec <container> free -h
```
### Image size optimization
```bash
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}"
docker history <image>
```
## Network Debugging
### List networks and connections
```bash
docker network ls
docker network inspect <network_name>
docker port <container>
```
### Test connectivity
```bash
docker exec <container> ping <target>
docker exec <container> nslookup <hostname>
docker exec <container> netstat -tulpn
```
## Quick Fixes
### Remove all stopped containers
```bash
docker container prune
```
### Remove unused images
```bash
docker image prune -a
```
### Reset Docker completely
```bash
docker system prune -a --volumes
```

View File

@ -0,0 +1,183 @@
# Network Troubleshooting Reference
## Connectivity Issues
### Basic connectivity tests
```bash
# Test connectivity
ping <target_ip>
ping -c 4 <target_ip> # Linux (4 packets)
# Test specific port
telnet <target_ip> <port>
nc -zv <target_ip> <port> # netcat
nmap -p <port> <target_ip>
# DNS resolution
nslookup <hostname>
dig <hostname>
host <hostname>
```
### Network interface debugging
```bash
# Show interfaces
ip addr show
ifconfig
# Show routing table
ip route
route -n
# Show network statistics
netstat -i
ss -i
```
## Service Issues
### Port and process debugging
```bash
# Show listening ports
netstat -tulpn
ss -tulpn
lsof -i :<port>
# Find process using port
fuser <port>/tcp
lsof -i tcp:<port>
# Show all connections
netstat -an
ss -an
```
### Service status
```bash
# Systemd services
systemctl status <service>
systemctl is-active <service>
systemctl is-enabled <service>
# Service logs
journalctl -u <service> -f
journalctl -u <service> --since "1 hour ago"
```
## Firewall Debugging
### iptables
```bash
# List rules
iptables -L -n -v
iptables -t nat -L -n -v
# Check if rule exists
iptables -C INPUT -p tcp --dport 80 -j ACCEPT
# Temporarily disable firewall
systemctl stop iptables # CentOS/RHEL
ufw disable # Ubuntu
```
### UFW (Ubuntu)
```bash
# Check status
ufw status verbose
# Show rules by number
ufw status numbered
# Check logs
tail -f /var/log/ufw.log
```
## SSL/TLS Issues
### Certificate debugging
```bash
# Check certificate details
openssl x509 -in certificate.crt -text -noout
openssl s_client -connect <hostname>:443 -servername <hostname>
# Check certificate chain
openssl s_client -connect <hostname>:443 -showcerts
# Test SSL connection
curl -vI https://<hostname>
wget --no-check-certificate -O /dev/null https://<hostname>
```
### Certificate expiration
```bash
# Check expiration date
openssl x509 -in certificate.crt -enddate -noout
# Check remote certificate expiration
echo | openssl s_client -servername <hostname> -connect <hostname>:443 2>/dev/null | openssl x509 -noout -dates
```
## DNS Issues
### DNS server testing
```bash
# Test specific DNS server
nslookup <hostname> <dns_server>
dig @<dns_server> <hostname>
# Flush DNS cache
# Linux (systemd-resolved)
systemctl restart systemd-resolved
# Linux (nscd)
systemctl restart nscd
```
### DNS configuration
```bash
# Check DNS settings
cat /etc/resolv.conf
cat /etc/systemd/resolved.conf
# Test DNS resolution order
getent hosts <hostname>
```
## Performance Issues
### Bandwidth testing
```bash
# iperf3 testing
# Server: iperf3 -s
# Client: iperf3 -c <server_ip>
# wget speed test
wget -O /dev/null http://speedtest.tele2.net/100MB.zip
```
### Network latency
```bash
# Continuous ping with timestamps
ping -D <target>
# MTR (better than traceroute)
mtr <target>
mtr --report <target>
```
## Quick Diagnostics
### One-liner network check
```bash
# Basic network health
ping -c 1 8.8.8.8 && echo "Internet OK" || echo "No Internet"
# Service accessibility
nc -zv localhost 80 && echo "HTTP service running" || echo "HTTP service down"
```
### Network interface stats
```bash
# Interface statistics
cat /proc/net/dev
watch -n 1 cat /proc/net/dev
```

View File

@ -0,0 +1,155 @@
# Python Debugging Reference
## Common Error Patterns
### ImportError / ModuleNotFoundError
```bash
# Check Python path
python -c "import sys; print('\n'.join(sys.path))"
# Check installed packages
pip list
pip show <package_name>
# Install missing packages
pip install -r requirements.txt
pip install <package_name>
```
### Virtual Environment Issues
```bash
# Check current environment
which python
which pip
# Recreate virtual environment
rm -rf venv/
python -m venv venv
source venv/bin/activate # Linux/Mac
# venv\Scripts\activate # Windows
pip install -r requirements.txt
```
### Memory Issues
```python
import tracemalloc
import psutil
import os
# Memory profiling
tracemalloc.start()
# ... your code ...
current, peak = tracemalloc.get_traced_memory()
print(f"Current memory usage: {current / 1024 / 1024:.1f} MB")
print(f"Peak memory usage: {peak / 1024 / 1024:.1f} MB")
# System memory
process = psutil.Process(os.getpid())
print(f"RSS memory: {process.memory_info().rss / 1024 / 1024:.1f} MB")
```
## Debugging Tools
### Using pdb debugger
```python
import pdb
def problematic_function():
x = 10
pdb.set_trace() # Execution will pause here
y = x * 2
return y
# Or use breakpoint() in Python 3.7+
def modern_debug():
x = 10
breakpoint() # Same as pdb.set_trace()
return x
```
### Logging setup
```python
import logging
# Configure logging
logging.basicConfig(
level=logging.DEBUG,
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[
logging.FileHandler('debug.log'),
logging.StreamHandler()
]
)
logger = logging.getLogger(__name__)
def example_function():
logger.debug("Starting function")
try:
result = risky_operation()
logger.info(f"Operation successful: {result}")
return result
except Exception as e:
logger.error(f"Operation failed: {e}", exc_info=True)
raise
```
## Performance Debugging
### Profile code execution
```python
import cProfile
import pstats
# Profile a function
cProfile.run('your_function()', 'profile_output.prof')
# Analyze results
stats = pstats.Stats('profile_output.prof')
stats.sort_stats('cumulative').print_stats(10)
```
### Line-by-line profiling
```bash
# Install line_profiler
pip install line_profiler
# Add @profile decorator to functions
# Run with:
kernprof -l -v your_script.py
```
### Check for memory leaks
```python
import gc
import objgraph
# Show most common types
objgraph.show_most_common_types()
# Track object growth
objgraph.show_growth()
```
## Database Debugging
### SQLAlchemy query debugging
```python
import logging
# Enable SQL query logging
logging.basicConfig()
logging.getLogger('sqlalchemy.engine').setLevel(logging.INFO)
logging.getLogger('sqlalchemy.dialects').setLevel(logging.DEBUG)
logging.getLogger('sqlalchemy.pool').setLevel(logging.DEBUG)
```
### Connection issues
```python
# Test database connection
try:
engine.connect()
print("Database connection successful")
except Exception as e:
print(f"Database connection failed: {e}")
```