major-domo-v2/utils/README.md

Utils Package Documentation

Discord Bot v2.0 - Utility Functions and Helpers

This package contains utility functions, helpers, and shared components used throughout the Discord bot application.

📋 Table of Contents

1. Structured Logging - Contextual logging with Discord integration
2. Future Utilities - Planned utility modules

---

🔍 Structured Logging

Location: utils/logging.py
Purpose: Provides hybrid logging system with contextual information for Discord bot debugging and monitoring.

Quick Start

from utils.logging import get_contextual_logger, set_discord_context

class YourCommandCog(commands.Cog):
    def __init__(self, bot):
        self.bot = bot
        self.logger = get_contextual_logger(f'{__name__}.YourCommandCog')
    
    async def your_command(self, interaction: discord.Interaction, param: str):
        # Set Discord context for all subsequent log entries
        set_discord_context(
            interaction=interaction,
            command="/your-command",
            param_value=param
        )
        
        # Start operation timing and get trace ID
        trace_id = self.logger.start_operation("your_command_operation")
        
        try:
            self.logger.info("Command started")
            
            # Your command logic here
            result = await some_api_call(param)
            self.logger.debug("API call completed", result_count=len(result))
            
            self.logger.info("Command completed successfully")
            
        except Exception as e:
            self.logger.error("Command failed", error=e)
            raise

Key Features

🎯 Contextual Information

Every log entry automatically includes:

• Discord Context: User ID, guild ID, guild name, channel ID
• Command Context: Command name, parameters
• Operation Context: Trace ID, operation name, execution duration
• Custom Fields: Additional context via keyword arguments

⏱️ Automatic Timing

trace_id = self.logger.start_operation("complex_operation")
# ... do work ...
self.logger.info("Operation completed")  # Automatically includes duration_ms

🔗 Request Tracing

Track a single request through all log entries using trace IDs:

# Find all logs for a specific request
jq '.context.trace_id == "abc12345"' logs/discord_bot_v2.json

📤 Hybrid Output

• Console: Human-readable for development
• Traditional File (discord_bot_v2.log): Human-readable with debug info
• JSON File (discord_bot_v2.json): Structured for analysis

API Reference

Core Functions

get_contextual_logger(logger_name: str) -> ContextualLogger

# Get a logger instance for your module
logger = get_contextual_logger(f'{__name__}.MyClass')

set_discord_context(interaction=None, user_id=None, guild_id=None, **kwargs)

# Set context from Discord interaction (recommended)
set_discord_context(interaction=interaction, command="/player", player_name="Mike Trout")

# Or set context manually
set_discord_context(user_id="123456", guild_id="987654", custom_field="value")

clear_context()

# Clear the current logging context (usually not needed)
clear_context()

ContextualLogger Methods

start_operation(operation_name: str = None) -> str

# Start timing and get trace ID
trace_id = logger.start_operation("player_search")

info(message: str, **kwargs)

logger.info("Player found", player_id=123, team_name="Yankees")

debug(message: str, **kwargs)

logger.debug("API call started", endpoint="players/search", timeout=30)

warning(message: str, **kwargs)

logger.warning("Multiple players found", candidates=["Player A", "Player B"])

error(message: str, error: Exception = None, **kwargs)

# With exception
logger.error("API call failed", error=e, retry_count=3)

# Without exception
logger.error("Validation failed", field="player_name", value="invalid")

exception(message: str, **kwargs)

# Automatically captures current exception
try:
    risky_operation()
except:
    logger.exception("Unexpected error in operation", operation_id=123)

Output Examples

Console Output (Development)

2025-08-14 14:32:15,123 - commands.players.info.PlayerInfoCommands - INFO - Player info command started
2025-08-14 14:32:16,456 - commands.players.info.PlayerInfoCommands - DEBUG - Starting player search
2025-08-14 14:32:18,789 - commands.players.info.PlayerInfoCommands - INFO - Command completed successfully

JSON Output (Monitoring & Analysis)

{
  "timestamp": "2025-08-14T14:32:15.123Z",
  "level": "INFO",
  "logger": "commands.players.info.PlayerInfoCommands",
  "message": "Player info command started",
  "function": "player_info",
  "line": 50,
  "context": {
    "user_id": "123456789",
    "guild_id": "987654321",
    "guild_name": "SBA League",
    "channel_id": "555666777",
    "command": "/player",
    "player_name": "Mike Trout",
    "season": 12,
    "trace_id": "abc12345",
    "operation": "player_info_command"
  },
  "extra": {
    "duration_ms": 0
  }
}

Error Output with Exception

{
  "timestamp": "2025-08-14T14:32:18.789Z",
  "level": "ERROR",
  "logger": "commands.players.info.PlayerInfoCommands",
  "message": "API call failed",
  "function": "player_info",
  "line": 125,
  "exception": {
    "type": "APITimeout",
    "message": "Request timed out after 30s",
    "traceback": "Traceback (most recent call last):\n  File ..."
  },
  "context": {
    "user_id": "123456789",
    "guild_id": "987654321",
    "command": "/player",
    "player_name": "Mike Trout",
    "trace_id": "abc12345"
  },
  "extra": {
    "duration_ms": 30000,
    "retry_count": 3,
    "endpoint": "players/search"
  }
}

Advanced Usage Patterns

API Call Logging

async def fetch_player_data(self, player_name: str):
    self.logger.debug("API call started", 
                     api_endpoint="players/search",
                     search_term=player_name,
                     timeout_ms=30000)
    
    try:
        result = await api_client.get("players", params=[("name", player_name)])
        self.logger.info("API call successful", 
                        results_found=len(result) if result else 0,
                        response_size_kb=len(str(result)) // 1024)
        return result
        
    except TimeoutError as e:
        self.logger.error("API timeout", 
                         error=e,
                         endpoint="players/search",
                         search_term=player_name)
        raise

Performance Monitoring

async def complex_operation(self, data):
    trace_id = self.logger.start_operation("complex_operation")
    
    # Step 1
    self.logger.debug("Processing step 1", step="validation")
    validate_data(data)
    
    # Step 2  
    self.logger.debug("Processing step 2", step="transformation")
    processed = transform_data(data)
    
    # Step 3
    self.logger.debug("Processing step 3", step="persistence")
    result = await save_data(processed)
    
    self.logger.info("Complex operation completed",
                    input_size=len(data),
                    output_size=len(result),
                    steps_completed=3)
    
    # Final log automatically includes total duration_ms

Error Context Enrichment

async def handle_player_command(self, interaction, player_name):
    set_discord_context(
        interaction=interaction,
        command="/player",
        player_name=player_name,
        # Add additional context that helps debugging
        user_permissions=interaction.user.guild_permissions.administrator,
        guild_member_count=len(interaction.guild.members),
        request_timestamp=discord.utils.utcnow().isoformat()
    )
    
    try:
        # Command logic
        pass
    except Exception as e:
        # Error logs will include all the above context automatically
        self.logger.error("Player command failed", 
                         error=e,
                         # Additional error-specific context
                         error_code="PLAYER_NOT_FOUND",
                         suggestion="Try using the full player name")
        raise

Querying JSON Logs

Using jq for Analysis

Find all errors:

jq 'select(.level == "ERROR")' logs/discord_bot_v2.json

Find slow operations (>5 seconds):

jq 'select(.extra.duration_ms > 5000)' logs/discord_bot_v2.json

Track a specific user's activity:

jq 'select(.context.user_id == "123456789")' logs/discord_bot_v2.json

Find API timeout errors:

jq 'select(.exception.type == "APITimeout")' logs/discord_bot_v2.json

Get error summary by type:

jq -r 'select(.level == "ERROR") | .exception.type' logs/discord_bot_v2.json | sort | uniq -c

Trace a complete request:

jq 'select(.context.trace_id == "abc12345")' logs/discord_bot_v2.json | jq -s 'sort_by(.timestamp)'

Performance Analysis

Average command execution time:

jq -r 'select(.message == "Command completed successfully") | .extra.duration_ms' logs/discord_bot_v2.json | awk '{sum+=$1; n++} END {print sum/n}'

Most active users:

jq -r '.context.user_id' logs/discord_bot_v2.json | sort | uniq -c | sort -nr | head -10

Command usage statistics:

jq -r '.context.command' logs/discord_bot_v2.json | sort | uniq -c | sort -nr

Best Practices

✅ Do:

1. Always set Discord context at the start of command handlers
2. Use start_operation() for timing critical operations
3. Include relevant context in log messages via keyword arguments
4. Log at appropriate levels (debug for detailed flow, info for milestones, warning for recoverable issues, error for failures)
5. Include error context when logging exceptions

❌ Don't:

1. Don't log sensitive information (passwords, tokens, personal data)
2. Don't over-log in tight loops (use sampling or conditional logging)
3. Don't use string formatting in log messages (use keyword arguments instead)
4. Don't forget to handle exceptions in logging code itself

Performance Considerations

• JSON serialization adds minimal overhead (~1-2ms per log entry)
• Context variables are async-safe and thread-local
• Log rotation prevents disk space issues
• Structured queries are much faster than grep on large files

Troubleshooting

Common Issues

Logs not appearing:

• Check log level configuration in environment
• Verify logs/ directory permissions
• Ensure handlers are properly configured

JSON serialization errors:

• Avoid logging complex objects directly
• Convert objects to strings or dicts before logging
• The JSONFormatter handles most common types automatically

Context not appearing in logs:

• Ensure set_discord_context() is called before logging
• Context is tied to the current async task
• Check that context is not cleared prematurely

Performance issues:

• Monitor log file sizes and rotation
• Consider reducing log level in production
• Use sampling for high-frequency operations

---

🚀 Future Utilities

Additional utility modules planned for future implementation:

Discord Helpers (Planned)

• Embed builders and formatters
• Permission checking decorators
• User mention and role utilities
• Message pagination helpers

API Utilities (Planned)

• Rate limiting decorators
• Response caching mechanisms
• Retry logic with exponential backoff
• Request validation helpers

Data Processing (Planned)

• CSV/JSON export utilities
• Statistical calculation helpers
• Date/time formatting for baseball seasons
• Text processing and search utilities

Testing Utilities (Planned)

• Mock Discord objects for testing
• Fixture generators for common test data
• Assertion helpers for Discord responses
• Test database setup and teardown

---

📚 Usage Examples by Module

Logging Integration in Commands

# commands/teams/roster.py
from utils.logging import get_contextual_logger, set_discord_context

class TeamRosterCommands(commands.Cog):
    def __init__(self, bot):
        self.bot = bot
        self.logger = get_contextual_logger(f'{__name__}.TeamRosterCommands')
    
    @discord.app_commands.command(name="roster")
    async def team_roster(self, interaction, team_name: str, season: int = None):
        set_discord_context(
            interaction=interaction,
            command="/roster", 
            team_name=team_name,
            season=season
        )
        
        trace_id = self.logger.start_operation("team_roster_command")
        
        try:
            self.logger.info("Team roster command started")
            
            # Command implementation
            team = await team_service.find_team(team_name)
            self.logger.debug("Team found", team_id=team.id, team_abbreviation=team.abbrev)
            
            players = await team_service.get_roster(team.id, season)
            self.logger.info("Roster retrieved", player_count=len(players))
            
            # Create and send response
            embed = create_roster_embed(team, players)
            await interaction.followup.send(embed=embed)
            
            self.logger.info("Team roster command completed")
            
        except TeamNotFoundError as e:
            self.logger.warning("Team not found", search_term=team_name)
            await interaction.followup.send(f"❌ Team '{team_name}' not found", ephemeral=True)
            
        except Exception as e:
            self.logger.error("Team roster command failed", error=e)
            await interaction.followup.send("❌ Error retrieving team roster", ephemeral=True)

Service Layer Logging

# services/team_service.py  
from utils.logging import get_contextual_logger

class TeamService(BaseService[Team]):
    def __init__(self):
        super().__init__(Team, 'teams')
        self.logger = get_contextual_logger(f'{__name__}.TeamService')
    
    async def find_team(self, team_name: str) -> Team:
        self.logger.debug("Starting team search", search_term=team_name)
        
        # Try exact match first
        teams = await self.get_by_field('name', team_name)
        if len(teams) == 1:
            self.logger.debug("Exact team match found", team_id=teams[0].id)
            return teams[0]
        
        # Try abbreviation match
        teams = await self.get_by_field('abbrev', team_name.upper())
        if len(teams) == 1:
            self.logger.debug("Team abbreviation match found", team_id=teams[0].id)
            return teams[0]
        
        # Try fuzzy search
        all_teams = await self.get_all_items()
        matches = [t for t in all_teams if team_name.lower() in t.name.lower()]
        
        if len(matches) == 0:
            self.logger.warning("No team matches found", search_term=team_name)
            raise TeamNotFoundError(f"No team found matching '{team_name}'")
        elif len(matches) > 1:
            match_names = [t.name for t in matches]
            self.logger.warning("Multiple team matches found", 
                              search_term=team_name,
                              matches=match_names)
            raise MultipleTeamsFoundError(f"Multiple teams found: {', '.join(match_names)}")
        
        self.logger.debug("Fuzzy team match found", team_id=matches[0].id)
        return matches[0]

---

📁 File Structure

utils/
├── README.md          # This documentation
├── __init__.py        # Package initialization
└── logging.py         # Structured logging implementation

# Future files:
├── discord_helpers.py # Discord utility functions
├── api_utils.py       # API helper functions  
├── data_processing.py # Data manipulation utilities
└── testing.py         # Testing helper functions

---

Last Updated: Phase 2.1 - Structured Logging Implementation
Next Update: When additional utility modules are added

For questions or improvements to the logging system, check the implementation in utils/logging.py or refer to the JSON log outputs in logs/discord_bot_v2.json.