strat-gameplay-webapp/backend/tests/websocket_test_coverage_report.md

# WebSocket Module Test Coverage Report

**Analysis Date:** 2025-11-04
**Analyst:** Claude Code
**Module Path:** `backend/app/websocket/`

---

## Executive Summary

**Overall Coverage Assessment: FAIR (36% handler coverage, 0% connection manager coverage)**

The websocket module has **selective testing** focusing on the most critical gameplay handlers (dice rolling and outcome submission), but lacks comprehensive coverage of connection lifecycle, room management, and substitution handlers.

### Key Metrics
- **Total Files:** 3 (1 implementation + 1 handler registry)
- **Total Event Handlers:** 11
- **Handlers with Tests:** 4 (36%)
- **Total Tests:** 14 (12 unit + 2 integration)
- **Lines of Code:** 1,107 lines
- **Test Status:** All 14 tests passing ✅

---

## Module Structure

### Files Analyzed

1. **`connection_manager.py`** (80 lines)
   - Purpose: WebSocket connection lifecycle and broadcasting
   - Key Classes: `ConnectionManager`
   - Test Coverage: ❌ **0% - NO TESTS**

2. **`handlers.py`** (1,027 lines)
   - Purpose: Event handler registration and processing
   - Event Handlers: 11 handlers
   - Test Coverage: ⚠️ **36% - PARTIAL** (4 of 11 handlers tested)

3. **`__init__.py`** (0 lines)
   - Empty package marker

---

## Test Coverage Breakdown

### Well-Tested Functionality ✅

#### 1. `roll_dice` Handler (5 tests)
**Coverage:** Excellent
**Test File:** `tests/unit/websocket/test_manual_outcome_handlers.py`

**Tests:**
- ✅ Successful dice roll and broadcast
- ✅ Missing game_id validation
- ✅ Invalid UUID format validation
- ✅ Game not found error handling
- ✅ Pending roll storage in game state

**Assessment:** This is the most critical gameplay handler and is thoroughly tested with all major paths and edge cases covered.

---

#### 2. `submit_manual_outcome` Handler (7 tests)
**Coverage:** Excellent
**Test File:** `tests/unit/websocket/test_manual_outcome_handlers.py`

**Tests:**
- ✅ Successful outcome submission and play resolution
- ✅ Missing game_id validation
- ✅ Missing outcome validation
- ✅ Invalid outcome enum value
- ✅ Invalid hit_location value
- ✅ Missing required hit_location for groundballs
- ✅ No pending dice roll validation
- ✅ Walk submission without location (valid case)

**Assessment:** Comprehensive coverage of the manual outcome submission flow including validation, error handling, and successful resolution paths.

---

#### 3. X-Check Integration (2 tests)
**Coverage:** Basic
**Test File:** `tests/integration/test_xcheck_websocket.py`

**Tests:**
- ✅ Submit manual X-Check outcome with full result details
- ✅ Non-X-Check plays don't include x_check_details

**Assessment:** Basic integration testing ensures X-Check details are correctly included in WebSocket broadcasts. Good coverage of the critical path.

---

### Undertested or Missing Coverage ❌

#### 4. ConnectionManager Class (0 tests)
**Coverage:** ❌ **NONE**
**Lines of Code:** ~80 lines
**Critical Methods Untested:**
- `connect(sid, user_id)` - User registration
- `disconnect(sid)` - Cleanup and notifications
- `join_game(sid, game_id, role)` - Room membership
- `leave_game(sid, game_id)` - Room departure
- `broadcast_to_game(game_id, event, data)` - Game room broadcasting
- `emit_to_user(sid, event, data)` - Direct user messaging
- `get_game_participants(game_id)` - Participant tracking

**Business Impact:** HIGH - This is the foundation of real-time communication
**Risk:** Connection tracking issues, memory leaks, broadcast failures could go undetected

**Recommended Tests:**
```python
# Unit tests needed:
- test_connect_registers_user_session
- test_disconnect_removes_from_all_rooms
- test_disconnect_broadcasts_to_affected_games
- test_join_game_adds_to_room
- test_join_game_broadcasts_user_connected
- test_leave_game_removes_from_room
- test_broadcast_to_game_reaches_all_participants
- test_emit_to_user_targets_specific_session
- test_get_game_participants_returns_correct_sids
- test_multiple_games_isolated_rooms
```

---

#### 5. `connect` Handler (0 tests)
**Coverage:** ❌ **NONE**
**Purpose:** JWT authentication and connection acceptance

**Critical Paths Untested:**
- ✅ Valid JWT token acceptance
- ✅ Invalid token rejection
- ✅ Missing token rejection
- ✅ Expired token handling
- ✅ User ID extraction
- ✅ Connection confirmation emission

**Business Impact:** HIGH - Security critical (authentication)
**Risk:** Unauthorized connections, token validation bypass

---

#### 6. `disconnect` Handler (0 tests)
**Coverage:** ❌ **NONE**
**Purpose:** Connection cleanup on disconnect

**Critical Paths Untested:**
- Session removal from user_sessions
- Removal from all game_rooms
- Broadcast of user_disconnected events
- Graceful handling of already-disconnected users

**Business Impact:** MEDIUM - Can cause memory leaks and stale connections
**Risk:** Connection tracking errors, broadcast failures

---

#### 7. `join_game` Handler (0 tests)
**Coverage:** ❌ **NONE**
**Purpose:** Add user to game room for real-time updates

**Critical Paths Untested:**
- ✅ Valid game join
- ✅ Missing game_id validation
- ✅ Invalid game_id format
- ✅ Game room broadcasting
- ✅ User role tracking (player vs spectator)
- ❌ Authorization check (TODO in code)

**Business Impact:** HIGH - Required for all real-time gameplay
**Risk:** Users unable to receive game updates

---

#### 8. `leave_game` Handler (0 tests)
**Coverage:** ❌ **NONE**
**Purpose:** Remove user from game room

**Critical Paths Untested:**
- Valid game leave
- Cleanup of room membership
- Handling of already-left users

**Business Impact:** LOW - Not frequently used
**Risk:** Minor - room membership tracking issues

---

#### 9. `heartbeat` Handler (0 tests)
**Coverage:** ❌ **NONE**
**Purpose:** Keep-alive ping/pong mechanism

**Critical Paths Untested:**
- Heartbeat reception
- Heartbeat acknowledgment emission

**Business Impact:** LOW - Simple functionality
**Risk:** Connection timeout issues (if implemented)

---

#### 10. Substitution Handlers (0 tests)
**Coverage:** ❌ **NONE**
**Handlers:** 3 handlers (pinch hitter, defensive replacement, pitching change)
**Lines of Code:** ~750 lines (73% of handlers.py)

**Handlers Untested:**
- `request_pinch_hitter` (~180 lines)
- `request_defensive_replacement` (~180 lines)
- `request_pitching_change` (~180 lines)

**Critical Paths Untested:**
Each handler needs tests for:
- ✅ Successful substitution
- ✅ Missing required fields (game_id, player_out_lineup_id, player_in_card_id, team_id)
- ✅ Invalid game_id format
- ✅ Game not found
- ✅ Validation failures (NOT_CURRENT_BATTER, PLAYER_ALREADY_OUT, etc.)
- ✅ Success broadcasts (player_substituted to room)
- ✅ Confirmation messages (substitution_confirmed to requester)
- ✅ Error responses (substitution_error with error codes)
- ❌ Authorization checks (TODO in code)

**Business Impact:** HIGH - Core gameplay feature (Phase 3F implementation)
**Risk:** Substitution bugs could corrupt game state

**Note:** Substitution business logic is tested in:
- `tests/unit/core/test_substitution_rules.py` - Validation rules
- `tests/integration/test_substitution_manager.py` - Manager integration

But WebSocket event handlers themselves are NOT tested.

---

#### 11. `get_lineup` Handler (0 tests)
**Coverage:** ❌ **NONE**
**Purpose:** Retrieve current lineup for UI refresh

**Critical Paths Untested:**
- ✅ Successful lineup retrieval from cache
- ✅ Fallback to database when not cached
- ✅ Missing game_id validation
- ✅ Missing team_id validation
- ✅ Lineup not found error
- ✅ Active players filtering
- ❌ Authorization check (TODO in code)

**Business Impact:** MEDIUM - Required for lineup UI
**Risk:** Lineup display issues after substitutions

---

## Coverage Statistics

### By Handler Type

| Handler Type | Handlers | Tested | Coverage | Status |
|-------------|----------|--------|----------|--------|
| Connection Lifecycle | 4 | 0 | 0% | ❌ None |
| Gameplay (Dice/Outcome) | 2 | 2 | 100% | ✅ Complete |
| Substitutions | 3 | 0 | 0% | ❌ None |
| Lineup Management | 1 | 0 | 0% | ❌ None |
| Utility (Heartbeat) | 1 | 0 | 0% | ❌ None |
| **TOTAL** | **11** | **4** | **36%** | ⚠️ **Partial** |

### By Lines of Code

| Component | LOC | Tests | Test Lines | Status |
|-----------|-----|-------|-----------|--------|
| connection_manager.py | 80 | 0 | 0 | ❌ None |
| handlers.py (tested) | ~250 | 14 | ~460 | ✅ Good |
| handlers.py (untested) | ~750 | 0 | 0 | ❌ None |
| **TOTAL** | **1,107** | **14** | **~460** | ⚠️ **Partial** |

---

## Risk Assessment

### Critical Gaps (High Priority)

1. **ConnectionManager Testing** - HIGHEST PRIORITY
   - **Risk:** Connection tracking bugs, memory leaks, broadcast failures
   - **Impact:** Could affect ALL real-time gameplay
   - **Recommendation:** Add comprehensive unit tests (10-15 tests)

2. **Authentication Handler (`connect`)** - HIGH PRIORITY
   - **Risk:** Security vulnerabilities, unauthorized access
   - **Impact:** Authentication bypass
   - **Recommendation:** Add security-focused tests (5-7 tests)

3. **Substitution Handlers** - HIGH PRIORITY
   - **Risk:** Game state corruption, invalid substitutions
   - **Impact:** Gameplay bugs, user frustration
   - **Recommendation:** Add comprehensive tests for all 3 handlers (~30 tests)

### Medium Priority Gaps

4. **Room Management (`join_game`, `leave_game`)**
   - **Risk:** Users not receiving game updates
   - **Impact:** Poor user experience
   - **Recommendation:** Add integration tests (8-10 tests)

5. **Lineup Handler (`get_lineup`)**
   - **Risk:** Incorrect lineup display
   - **Impact:** UI bugs after substitutions
   - **Recommendation:** Add tests (5-6 tests)

### Low Priority Gaps

6. **Disconnect Handler**
   - **Risk:** Memory leaks (minor - disconnect cleanup)
   - **Recommendation:** Add cleanup verification tests (3-4 tests)

7. **Heartbeat Handler**
   - **Risk:** Minimal (simple ping/pong)
   - **Recommendation:** Add basic tests if time permits (1-2 tests)

---

## Testing Patterns Found

### Good Patterns ✅

1. **Mocking Strategy:**
   ```python
   @pytest.fixture
   def mock_manager():
       manager = MagicMock()
       manager.emit_to_user = AsyncMock()
       manager.broadcast_to_game = AsyncMock()
       return manager
   ```
   - Clean separation of concerns
   - Async mock support
   - Easy verification of emissions

2. **Comprehensive Edge Cases:**
   - Missing fields validation
   - Invalid format validation
   - Not found error handling
   - Business rule validation

3. **Integration Testing:**
   - Full flow tests (roll → submit → resolve)
   - Mocked game engine for isolation
   - Verification of broadcast content

### Areas for Improvement ⚠️

1. **No ConnectionManager Tests:**
   - Core infrastructure untested
   - Need unit tests for all methods

2. **Missing Authorization Tests:**
   - All handlers have TODO comments for auth checks
   - Need to test authorization failures once implemented

3. **Limited Error Scenario Coverage:**
   - Most handlers have happy path tests only
   - Need comprehensive error condition tests

---

## Recommendations

### Immediate Actions (High Priority)

1. **Add ConnectionManager Unit Tests** (~2-3 hours)
   - Test all 6 public methods
   - Test room isolation
   - Test memory cleanup on disconnect
   - Test broadcast delivery

2. **Add Substitution Handler Tests** (~4-5 hours)
   - Test all 3 substitution handlers
   - Cover validation errors
   - Test success broadcasts
   - Test error responses with codes

3. **Add Authentication Handler Tests** (~1-2 hours)
   - Test JWT validation
   - Test token rejection paths
   - Test connection acceptance

### Medium Priority Actions

4. **Add Room Management Tests** (~2-3 hours)
   - Test join_game handler
   - Test leave_game handler
   - Test room membership tracking

5. **Add Lineup Handler Tests** (~1-2 hours)
   - Test cache hits and misses
   - Test database fallback
   - Test validation errors

### Future Improvements

6. **Add Authorization Tests** (when auth is implemented)
   - Test user verification
   - Test team ownership checks
   - Test active player checks

7. **Integration Tests for Full Flows**
   - Complete substitution flow end-to-end
   - Multi-user game scenarios
   - Reconnection and state recovery

---

## Testing Infrastructure

### Current Setup ✅

- **pytest-asyncio** - Async test support
- **unittest.mock** - Mocking framework
- **AsyncMock** - Async function mocking
- **Fixtures** - Reusable test setup

### What Works Well

- Unit tests run fast (~0.33s for 12 tests)
- Clear test organization
- Good mocking patterns
- Comprehensive docstrings

### What Needs Improvement

- No ConnectionManager test fixtures
- No end-to-end WebSocket client tests
- No load/stress testing

---

## Test File Locations

### Existing Tests

```
tests/
├── unit/
│   └── websocket/
│       └── test_manual_outcome_handlers.py  (12 tests) ✅
└── integration/
    └── test_xcheck_websocket.py              (2 tests) ✅
```

### Recommended New Test Files

```
tests/
├── unit/
│   └── websocket/
│       ├── test_manual_outcome_handlers.py     (existing)
│       ├── test_connection_manager.py          (NEW - 10-15 tests)
│       ├── test_connection_handlers.py         (NEW - 5-7 tests)
│       ├── test_substitution_handlers.py       (NEW - 30 tests)
│       ├── test_room_management_handlers.py    (NEW - 8-10 tests)
│       └── test_lineup_handler.py              (NEW - 5-6 tests)
└── integration/
    ├── test_xcheck_websocket.py                (existing)
    ├── test_substitution_websocket_flow.py     (NEW - 5-8 tests)
    └── test_multi_user_game.py                 (NEW - 3-5 tests)
```

---

## Specific Test Recommendations

### ConnectionManager Tests (Priority 1)

```python
# tests/unit/websocket/test_connection_manager.py

class TestConnectionManager:
    """Unit tests for ConnectionManager"""

    def test_connect_registers_user():
        """Test user session registration"""
        # Arrange, Act, Assert

    def test_disconnect_removes_from_all_rooms():
        """Test cleanup on disconnect"""

    def test_disconnect_broadcasts_to_affected_games():
        """Test user_disconnected events"""

    def test_join_game_adds_to_room():
        """Test room membership tracking"""

    def test_join_game_broadcasts_user_connected():
        """Test join broadcast"""

    def test_leave_game_removes_from_room():
        """Test room departure"""

    def test_broadcast_to_game_reaches_all_participants():
        """Test game room broadcasting"""

    def test_broadcast_to_game_isolated_rooms():
        """Test room isolation (messages don't leak)"""

    def test_emit_to_user_targets_specific_session():
        """Test direct user messaging"""

    def test_get_game_participants_returns_correct_sids():
        """Test participant tracking"""

    def test_disconnect_handles_user_not_found():
        """Test disconnect of unknown user"""

    def test_multiple_games_same_user():
        """Test user in multiple games simultaneously"""
```

### Substitution Handler Tests (Priority 2)

```python
# tests/unit/websocket/test_substitution_handlers.py

class TestPinchHitterHandler:
    """Tests for request_pinch_hitter handler"""

    async def test_pinch_hitter_success():
        """Test successful pinch hitter substitution"""

    async def test_pinch_hitter_missing_game_id():
        """Test validation: missing game_id"""

    async def test_pinch_hitter_invalid_game_id():
        """Test validation: invalid UUID format"""

    async def test_pinch_hitter_game_not_found():
        """Test error: game doesn't exist"""

    async def test_pinch_hitter_missing_player_out():
        """Test validation: missing player_out_lineup_id"""

    async def test_pinch_hitter_missing_player_in():
        """Test validation: missing player_in_card_id"""

    async def test_pinch_hitter_missing_team_id():
        """Test validation: missing team_id"""

    async def test_pinch_hitter_not_current_batter():
        """Test business rule: can only pinch hit for current batter"""

    async def test_pinch_hitter_player_already_out():
        """Test business rule: no re-entry"""

    async def test_pinch_hitter_broadcasts_success():
        """Test player_substituted broadcast to room"""

    async def test_pinch_hitter_confirms_to_requester():
        """Test substitution_confirmed to requester"""

class TestDefensiveReplacementHandler:
    """Tests for request_defensive_replacement handler"""
    # Similar structure (10 tests)

class TestPitchingChangeHandler:
    """Tests for request_pitching_change handler"""
    # Similar structure (10 tests)
```

### Connection Handler Tests (Priority 3)

```python
# tests/unit/websocket/test_connection_handlers.py

class TestConnectHandler:
    """Tests for connect handler (authentication)"""

    async def test_connect_valid_token():
        """Test connection with valid JWT"""

    async def test_connect_invalid_token():
        """Test rejection of invalid JWT"""

    async def test_connect_missing_token():
        """Test rejection when no token provided"""

    async def test_connect_expired_token():
        """Test rejection of expired JWT"""

    async def test_connect_emits_connected_event():
        """Test connected event emission"""

    async def test_connect_extracts_user_id():
        """Test user_id extraction from token"""

class TestDisconnectHandler:
    """Tests for disconnect handler"""

    async def test_disconnect_removes_session():
        """Test session cleanup"""

    async def test_disconnect_removes_from_rooms():
        """Test room cleanup"""

    async def test_disconnect_broadcasts_to_games():
        """Test user_disconnected broadcast"""
```

---

## Example Test Implementation

Here's a complete example for ConnectionManager testing:

```python
"""
Unit tests for ConnectionManager.

Tests connection lifecycle, room management, and broadcasting.
"""
import pytest
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4

from app.websocket.connection_manager import ConnectionManager


@pytest.fixture
def mock_sio():
    """Mock Socket.io server"""
    sio = MagicMock()
    sio.enter_room = AsyncMock()
    sio.leave_room = AsyncMock()
    sio.emit = AsyncMock()
    return sio


@pytest.fixture
def manager(mock_sio):
    """Create ConnectionManager instance"""
    return ConnectionManager(mock_sio)


class TestConnectionManager:
    """Unit tests for ConnectionManager"""

    async def test_connect_registers_user(self, manager):
        """Test user session registration"""
        # Arrange
        sid = "test-session-123"
        user_id = "user-456"

        # Act
        await manager.connect(sid, user_id)

        # Assert
        assert sid in manager.user_sessions
        assert manager.user_sessions[sid] == user_id

    async def test_disconnect_removes_session(self, manager):
        """Test session cleanup on disconnect"""
        # Arrange
        sid = "test-session-123"
        user_id = "user-456"
        await manager.connect(sid, user_id)

        # Act
        await manager.disconnect(sid)

        # Assert
        assert sid not in manager.user_sessions

    async def test_disconnect_removes_from_rooms(self, manager):
        """Test removal from all game rooms on disconnect"""
        # Arrange
        sid = "test-session-123"
        user_id = "user-456"
        game_id = str(uuid4())

        await manager.connect(sid, user_id)
        await manager.join_game(sid, game_id, "player")

        # Act
        await manager.disconnect(sid)

        # Assert
        assert sid not in manager.game_rooms.get(game_id, set())

    async def test_join_game_adds_to_room(self, manager, mock_sio):
        """Test adding user to game room"""
        # Arrange
        sid = "test-session-123"
        user_id = "user-456"
        game_id = str(uuid4())

        await manager.connect(sid, user_id)

        # Act
        await manager.join_game(sid, game_id, "player")

        # Assert
        mock_sio.enter_room.assert_called_once_with(sid, game_id)
        assert game_id in manager.game_rooms
        assert sid in manager.game_rooms[game_id]

    async def test_broadcast_to_game(self, manager, mock_sio):
        """Test broadcasting to game room"""
        # Arrange
        game_id = str(uuid4())
        event = "test_event"
        data = {"key": "value"}

        # Act
        await manager.broadcast_to_game(game_id, event, data)

        # Assert
        mock_sio.emit.assert_called_once_with(event, data, room=game_id)

    async def test_emit_to_user(self, manager, mock_sio):
        """Test emitting to specific user"""
        # Arrange
        sid = "test-session-123"
        event = "test_event"
        data = {"key": "value"}

        # Act
        await manager.emit_to_user(sid, event, data)

        # Assert
        mock_sio.emit.assert_called_once_with(event, data, room=sid)

    async def test_get_game_participants(self, manager):
        """Test getting participants in game room"""
        # Arrange
        game_id = str(uuid4())
        sid1 = "session-1"
        sid2 = "session-2"

        await manager.connect(sid1, "user-1")
        await manager.connect(sid2, "user-2")
        await manager.join_game(sid1, game_id, "player")
        await manager.join_game(sid2, game_id, "player")

        # Act
        participants = manager.get_game_participants(game_id)

        # Assert
        assert participants == {sid1, sid2}
```

---

## Success Criteria

### Minimum Acceptable Coverage

To reach a "Good" coverage rating, the following tests must be added:

- ✅ ConnectionManager: 10-15 tests (currently 0)
- ✅ Authentication handler: 5-7 tests (currently 0)
- ✅ Substitution handlers: 30 tests (currently 0)
- ✅ Room management: 8-10 tests (currently 0)
- ✅ Lineup handler: 5-6 tests (currently 0)

**Total New Tests Needed:** ~58-73 tests
**Estimated Effort:** ~10-15 hours

### Target Coverage

| Metric | Current | Target | Status |
|--------|---------|--------|--------|
| Handler Coverage | 36% | 100% | ❌ Gap |
| ConnectionManager Tests | 0 | 10-15 | ❌ Gap |
| Total Tests | 14 | 70-80 | ❌ Gap |
| Critical Paths | 18% | 90%+ | ❌ Gap |

---

## Conclusion

The websocket module has **selective but excellent** coverage of the most critical gameplay handlers (dice rolling and outcome submission), but lacks comprehensive testing of infrastructure (ConnectionManager) and newer features (substitutions).

**Key Strengths:**
- Core gameplay handlers well-tested
- Good testing patterns established
- All existing tests passing
- Clear test organization

**Key Weaknesses:**
- ConnectionManager completely untested
- Substitution handlers untested (73% of handler code)
- Connection lifecycle untested
- Room management untested

**Recommendation:** Prioritize adding tests for ConnectionManager and substitution handlers to bring coverage to acceptable levels before production deployment. The existing test patterns provide a good template for expansion.

**Overall Grade:** C+ (Fair)
- Well-tested critical path
- Major infrastructure gaps
- Security concerns (auth handler untested)
- New features untested

---

**Report Generated:** 2025-11-04
**Next Review:** After adding recommended tests
**Contact:** Development Team