strat-gameplay-webapp/backend/tests/websocket_test_coverage_report.md

WebSocket Module Test Coverage Report

Analysis Date: 2025-11-04 Analyst: Claude Code Module Path: backend/app/websocket/

---

Executive Summary

Overall Coverage Assessment: FAIR (36% handler coverage, 0% connection manager coverage)

The websocket module has selective testing focusing on the most critical gameplay handlers (dice rolling and outcome submission), but lacks comprehensive coverage of connection lifecycle, room management, and substitution handlers.

Key Metrics

• Total Files: 3 (1 implementation + 1 handler registry)
• Total Event Handlers: 11
• Handlers with Tests: 4 (36%)
• Total Tests: 14 (12 unit + 2 integration)
• Lines of Code: 1,107 lines
• Test Status: All 14 tests passing ✅

---

Module Structure

Files Analyzed

1. connection_manager.py (80 lines)

   • Purpose: WebSocket connection lifecycle and broadcasting
   • Key Classes: ConnectionManager
   • Test Coverage: ❌ 0% - NO TESTS

2. handlers.py (1,027 lines)

   • Purpose: Event handler registration and processing
   • Event Handlers: 11 handlers
   • Test Coverage: ⚠️ 36% - PARTIAL (4 of 11 handlers tested)

3. __init__.py (0 lines)

   • Empty package marker

---

Test Coverage Breakdown

Well-Tested Functionality ✅

1. roll_dice Handler (5 tests)

Coverage: Excellent Test File: tests/unit/websocket/test_manual_outcome_handlers.py

Tests:

• ✅ Successful dice roll and broadcast
• ✅ Missing game_id validation
• ✅ Invalid UUID format validation
• ✅ Game not found error handling
• ✅ Pending roll storage in game state

Assessment: This is the most critical gameplay handler and is thoroughly tested with all major paths and edge cases covered.

---

2. submit_manual_outcome Handler (7 tests)

Coverage: Excellent Test File: tests/unit/websocket/test_manual_outcome_handlers.py

Tests:

• ✅ Successful outcome submission and play resolution
• ✅ Missing game_id validation
• ✅ Missing outcome validation
• ✅ Invalid outcome enum value
• ✅ Invalid hit_location value
• ✅ Missing required hit_location for groundballs
• ✅ No pending dice roll validation
• ✅ Walk submission without location (valid case)

Assessment: Comprehensive coverage of the manual outcome submission flow including validation, error handling, and successful resolution paths.

---

3. X-Check Integration (2 tests)

Coverage: Basic Test File: tests/integration/test_xcheck_websocket.py

Tests:

• ✅ Submit manual X-Check outcome with full result details
• ✅ Non-X-Check plays don't include x_check_details

Assessment: Basic integration testing ensures X-Check details are correctly included in WebSocket broadcasts. Good coverage of the critical path.

---

Undertested or Missing Coverage ❌

4. ConnectionManager Class (0 tests)

Coverage: ❌ NONE Lines of Code: ~80 lines Critical Methods Untested:

• connect(sid, user_id) - User registration
• disconnect(sid) - Cleanup and notifications
• join_game(sid, game_id, role) - Room membership
• leave_game(sid, game_id) - Room departure
• broadcast_to_game(game_id, event, data) - Game room broadcasting
• emit_to_user(sid, event, data) - Direct user messaging
• get_game_participants(game_id) - Participant tracking

Business Impact: HIGH - This is the foundation of real-time communication Risk: Connection tracking issues, memory leaks, broadcast failures could go undetected

Recommended Tests:

# Unit tests needed:
- test_connect_registers_user_session
- test_disconnect_removes_from_all_rooms
- test_disconnect_broadcasts_to_affected_games
- test_join_game_adds_to_room
- test_join_game_broadcasts_user_connected
- test_leave_game_removes_from_room
- test_broadcast_to_game_reaches_all_participants
- test_emit_to_user_targets_specific_session
- test_get_game_participants_returns_correct_sids
- test_multiple_games_isolated_rooms

---

5. connect Handler (0 tests)

Coverage: ❌ NONE Purpose: JWT authentication and connection acceptance

Critical Paths Untested:

• ✅ Valid JWT token acceptance
• ✅ Invalid token rejection
• ✅ Missing token rejection
• ✅ Expired token handling
• ✅ User ID extraction
• ✅ Connection confirmation emission

Business Impact: HIGH - Security critical (authentication) Risk: Unauthorized connections, token validation bypass

---

6. disconnect Handler (0 tests)

Coverage: ❌ NONE Purpose: Connection cleanup on disconnect

Critical Paths Untested:

• Session removal from user_sessions
• Removal from all game_rooms
• Broadcast of user_disconnected events
• Graceful handling of already-disconnected users

Business Impact: MEDIUM - Can cause memory leaks and stale connections Risk: Connection tracking errors, broadcast failures

---

7. join_game Handler (0 tests)

Coverage: ❌ NONE Purpose: Add user to game room for real-time updates

Critical Paths Untested:

• ✅ Valid game join
• ✅ Missing game_id validation
• ✅ Invalid game_id format
• ✅ Game room broadcasting
• ✅ User role tracking (player vs spectator)
• ❌ Authorization check (TODO in code)

Business Impact: HIGH - Required for all real-time gameplay Risk: Users unable to receive game updates

---

8. leave_game Handler (0 tests)

Coverage: ❌ NONE Purpose: Remove user from game room

Critical Paths Untested:

• Valid game leave
• Cleanup of room membership
• Handling of already-left users

Business Impact: LOW - Not frequently used Risk: Minor - room membership tracking issues

---

9. heartbeat Handler (0 tests)

Coverage: ❌ NONE Purpose: Keep-alive ping/pong mechanism

Critical Paths Untested:

• Heartbeat reception
• Heartbeat acknowledgment emission

Business Impact: LOW - Simple functionality Risk: Connection timeout issues (if implemented)

---

10. Substitution Handlers (0 tests)

Coverage: ❌ NONE Handlers: 3 handlers (pinch hitter, defensive replacement, pitching change) Lines of Code: ~750 lines (73% of handlers.py)

Handlers Untested:

• request_pinch_hitter (~180 lines)
• request_defensive_replacement (~180 lines)
• request_pitching_change (~180 lines)

Critical Paths Untested: Each handler needs tests for:

• ✅ Successful substitution
• ✅ Missing required fields (game_id, player_out_lineup_id, player_in_card_id, team_id)
• ✅ Invalid game_id format
• ✅ Game not found
• ✅ Validation failures (NOT_CURRENT_BATTER, PLAYER_ALREADY_OUT, etc.)
• ✅ Success broadcasts (player_substituted to room)
• ✅ Confirmation messages (substitution_confirmed to requester)
• ✅ Error responses (substitution_error with error codes)
• ❌ Authorization checks (TODO in code)

Business Impact: HIGH - Core gameplay feature (Phase 3F implementation) Risk: Substitution bugs could corrupt game state

Note: Substitution business logic is tested in:

• tests/unit/core/test_substitution_rules.py - Validation rules
• tests/integration/test_substitution_manager.py - Manager integration

But WebSocket event handlers themselves are NOT tested.

---

11. get_lineup Handler (0 tests)

Coverage: ❌ NONE Purpose: Retrieve current lineup for UI refresh

Critical Paths Untested:

• ✅ Successful lineup retrieval from cache
• ✅ Fallback to database when not cached
• ✅ Missing game_id validation
• ✅ Missing team_id validation
• ✅ Lineup not found error
• ✅ Active players filtering
• ❌ Authorization check (TODO in code)

Business Impact: MEDIUM - Required for lineup UI Risk: Lineup display issues after substitutions

---

Coverage Statistics

By Handler Type

Handler Type	Handlers	Tested	Coverage	Status
Connection Lifecycle	4	0	0%	❌ None
Gameplay (Dice/Outcome)	2	2	100%	✅ Complete
Substitutions	3	0	0%	❌ None
Lineup Management	1	0	0%	❌ None
Utility (Heartbeat)	1	0	0%	❌ None
TOTAL	11	4	36%	⚠️ Partial

By Lines of Code

Component	LOC	Tests	Test Lines	Status
connection_manager.py	80	0	0	❌ None
handlers.py (tested)	~250	14	~460	✅ Good
handlers.py (untested)	~750	0	0	❌ None
TOTAL	1,107	14	~460	⚠️ Partial

---

Risk Assessment

Critical Gaps (High Priority)

1. ConnectionManager Testing - HIGHEST PRIORITY

   • Risk: Connection tracking bugs, memory leaks, broadcast failures
   • Impact: Could affect ALL real-time gameplay
   • Recommendation: Add comprehensive unit tests (10-15 tests)

2. Authentication Handler (connect) - HIGH PRIORITY

   • Risk: Security vulnerabilities, unauthorized access
   • Impact: Authentication bypass
   • Recommendation: Add security-focused tests (5-7 tests)

3. Substitution Handlers - HIGH PRIORITY

   • Risk: Game state corruption, invalid substitutions
   • Impact: Gameplay bugs, user frustration
   • Recommendation: Add comprehensive tests for all 3 handlers (~30 tests)

Medium Priority Gaps

4. Room Management (join_game, leave_game)

   • Risk: Users not receiving game updates
   • Impact: Poor user experience
   • Recommendation: Add integration tests (8-10 tests)

5. Lineup Handler (get_lineup)

   • Risk: Incorrect lineup display
   • Impact: UI bugs after substitutions
   • Recommendation: Add tests (5-6 tests)

Low Priority Gaps

6. Disconnect Handler

   • Risk: Memory leaks (minor - disconnect cleanup)
   • Recommendation: Add cleanup verification tests (3-4 tests)

7. Heartbeat Handler

   • Risk: Minimal (simple ping/pong)
   • Recommendation: Add basic tests if time permits (1-2 tests)

---

Testing Patterns Found

Good Patterns ✅

1. Mocking Strategy:

   @pytest.fixture
   def mock_manager():
       manager = MagicMock()
       manager.emit_to_user = AsyncMock()
       manager.broadcast_to_game = AsyncMock()
       return manager
   

   • Clean separation of concerns
   • Async mock support
   • Easy verification of emissions

2. Comprehensive Edge Cases:

   • Missing fields validation
   • Invalid format validation
   • Not found error handling
   • Business rule validation

3. Integration Testing:

   • Full flow tests (roll → submit → resolve)
   • Mocked game engine for isolation
   • Verification of broadcast content

Areas for Improvement ⚠️

1. No ConnectionManager Tests:

   • Core infrastructure untested
   • Need unit tests for all methods

2. Missing Authorization Tests:

   • All handlers have TODO comments for auth checks
   • Need to test authorization failures once implemented

3. Limited Error Scenario Coverage:

   • Most handlers have happy path tests only
   • Need comprehensive error condition tests

---

Recommendations

Immediate Actions (High Priority)

1. Add ConnectionManager Unit Tests (~2-3 hours)

   • Test all 6 public methods
   • Test room isolation
   • Test memory cleanup on disconnect
   • Test broadcast delivery

2. Add Substitution Handler Tests (~4-5 hours)

   • Test all 3 substitution handlers
   • Cover validation errors
   • Test success broadcasts
   • Test error responses with codes

3. Add Authentication Handler Tests (~1-2 hours)

   • Test JWT validation
   • Test token rejection paths
   • Test connection acceptance

Medium Priority Actions

4. Add Room Management Tests (~2-3 hours)

   • Test join_game handler
   • Test leave_game handler
   • Test room membership tracking

5. Add Lineup Handler Tests (~1-2 hours)

   • Test cache hits and misses
   • Test database fallback
   • Test validation errors

Future Improvements

6. Add Authorization Tests (when auth is implemented)

   • Test user verification
   • Test team ownership checks
   • Test active player checks

7. Integration Tests for Full Flows

   • Complete substitution flow end-to-end
   • Multi-user game scenarios
   • Reconnection and state recovery

---

Testing Infrastructure

Current Setup ✅

• pytest-asyncio - Async test support
• unittest.mock - Mocking framework
• AsyncMock - Async function mocking
• Fixtures - Reusable test setup

What Works Well

• Unit tests run fast (~0.33s for 12 tests)
• Clear test organization
• Good mocking patterns
• Comprehensive docstrings

What Needs Improvement

• No ConnectionManager test fixtures
• No end-to-end WebSocket client tests
• No load/stress testing

---

Test File Locations

Existing Tests

tests/
├── unit/
│   └── websocket/
│       └── test_manual_outcome_handlers.py  (12 tests) ✅
└── integration/
    └── test_xcheck_websocket.py              (2 tests) ✅

Recommended New Test Files

tests/
├── unit/
│   └── websocket/
│       ├── test_manual_outcome_handlers.py     (existing)
│       ├── test_connection_manager.py          (NEW - 10-15 tests)
│       ├── test_connection_handlers.py         (NEW - 5-7 tests)
│       ├── test_substitution_handlers.py       (NEW - 30 tests)
│       ├── test_room_management_handlers.py    (NEW - 8-10 tests)
│       └── test_lineup_handler.py              (NEW - 5-6 tests)
└── integration/
    ├── test_xcheck_websocket.py                (existing)
    ├── test_substitution_websocket_flow.py     (NEW - 5-8 tests)
    └── test_multi_user_game.py                 (NEW - 3-5 tests)

---

Specific Test Recommendations

ConnectionManager Tests (Priority 1)

# tests/unit/websocket/test_connection_manager.py

class TestConnectionManager:
    """Unit tests for ConnectionManager"""

    def test_connect_registers_user():
        """Test user session registration"""
        # Arrange, Act, Assert

    def test_disconnect_removes_from_all_rooms():
        """Test cleanup on disconnect"""

    def test_disconnect_broadcasts_to_affected_games():
        """Test user_disconnected events"""

    def test_join_game_adds_to_room():
        """Test room membership tracking"""

    def test_join_game_broadcasts_user_connected():
        """Test join broadcast"""

    def test_leave_game_removes_from_room():
        """Test room departure"""

    def test_broadcast_to_game_reaches_all_participants():
        """Test game room broadcasting"""

    def test_broadcast_to_game_isolated_rooms():
        """Test room isolation (messages don't leak)"""

    def test_emit_to_user_targets_specific_session():
        """Test direct user messaging"""

    def test_get_game_participants_returns_correct_sids():
        """Test participant tracking"""

    def test_disconnect_handles_user_not_found():
        """Test disconnect of unknown user"""

    def test_multiple_games_same_user():
        """Test user in multiple games simultaneously"""

Substitution Handler Tests (Priority 2)

# tests/unit/websocket/test_substitution_handlers.py

class TestPinchHitterHandler:
    """Tests for request_pinch_hitter handler"""

    async def test_pinch_hitter_success():
        """Test successful pinch hitter substitution"""

    async def test_pinch_hitter_missing_game_id():
        """Test validation: missing game_id"""

    async def test_pinch_hitter_invalid_game_id():
        """Test validation: invalid UUID format"""

    async def test_pinch_hitter_game_not_found():
        """Test error: game doesn't exist"""

    async def test_pinch_hitter_missing_player_out():
        """Test validation: missing player_out_lineup_id"""

    async def test_pinch_hitter_missing_player_in():
        """Test validation: missing player_in_card_id"""

    async def test_pinch_hitter_missing_team_id():
        """Test validation: missing team_id"""

    async def test_pinch_hitter_not_current_batter():
        """Test business rule: can only pinch hit for current batter"""

    async def test_pinch_hitter_player_already_out():
        """Test business rule: no re-entry"""

    async def test_pinch_hitter_broadcasts_success():
        """Test player_substituted broadcast to room"""

    async def test_pinch_hitter_confirms_to_requester():
        """Test substitution_confirmed to requester"""

class TestDefensiveReplacementHandler:
    """Tests for request_defensive_replacement handler"""
    # Similar structure (10 tests)

class TestPitchingChangeHandler:
    """Tests for request_pitching_change handler"""
    # Similar structure (10 tests)

Connection Handler Tests (Priority 3)

# tests/unit/websocket/test_connection_handlers.py

class TestConnectHandler:
    """Tests for connect handler (authentication)"""

    async def test_connect_valid_token():
        """Test connection with valid JWT"""

    async def test_connect_invalid_token():
        """Test rejection of invalid JWT"""

    async def test_connect_missing_token():
        """Test rejection when no token provided"""

    async def test_connect_expired_token():
        """Test rejection of expired JWT"""

    async def test_connect_emits_connected_event():
        """Test connected event emission"""

    async def test_connect_extracts_user_id():
        """Test user_id extraction from token"""

class TestDisconnectHandler:
    """Tests for disconnect handler"""

    async def test_disconnect_removes_session():
        """Test session cleanup"""

    async def test_disconnect_removes_from_rooms():
        """Test room cleanup"""

    async def test_disconnect_broadcasts_to_games():
        """Test user_disconnected broadcast"""

---

Example Test Implementation

Here's a complete example for ConnectionManager testing:

"""
Unit tests for ConnectionManager.

Tests connection lifecycle, room management, and broadcasting.
"""
import pytest
from unittest.mock import AsyncMock, MagicMock
from uuid import uuid4

from app.websocket.connection_manager import ConnectionManager


@pytest.fixture
def mock_sio():
    """Mock Socket.io server"""
    sio = MagicMock()
    sio.enter_room = AsyncMock()
    sio.leave_room = AsyncMock()
    sio.emit = AsyncMock()
    return sio


@pytest.fixture
def manager(mock_sio):
    """Create ConnectionManager instance"""
    return ConnectionManager(mock_sio)


class TestConnectionManager:
    """Unit tests for ConnectionManager"""

    async def test_connect_registers_user(self, manager):
        """Test user session registration"""
        # Arrange
        sid = "test-session-123"
        user_id = "user-456"

        # Act
        await manager.connect(sid, user_id)

        # Assert
        assert sid in manager.user_sessions
        assert manager.user_sessions[sid] == user_id

    async def test_disconnect_removes_session(self, manager):
        """Test session cleanup on disconnect"""
        # Arrange
        sid = "test-session-123"
        user_id = "user-456"
        await manager.connect(sid, user_id)

        # Act
        await manager.disconnect(sid)

        # Assert
        assert sid not in manager.user_sessions

    async def test_disconnect_removes_from_rooms(self, manager):
        """Test removal from all game rooms on disconnect"""
        # Arrange
        sid = "test-session-123"
        user_id = "user-456"
        game_id = str(uuid4())

        await manager.connect(sid, user_id)
        await manager.join_game(sid, game_id, "player")

        # Act
        await manager.disconnect(sid)

        # Assert
        assert sid not in manager.game_rooms.get(game_id, set())

    async def test_join_game_adds_to_room(self, manager, mock_sio):
        """Test adding user to game room"""
        # Arrange
        sid = "test-session-123"
        user_id = "user-456"
        game_id = str(uuid4())

        await manager.connect(sid, user_id)

        # Act
        await manager.join_game(sid, game_id, "player")

        # Assert
        mock_sio.enter_room.assert_called_once_with(sid, game_id)
        assert game_id in manager.game_rooms
        assert sid in manager.game_rooms[game_id]

    async def test_broadcast_to_game(self, manager, mock_sio):
        """Test broadcasting to game room"""
        # Arrange
        game_id = str(uuid4())
        event = "test_event"
        data = {"key": "value"}

        # Act
        await manager.broadcast_to_game(game_id, event, data)

        # Assert
        mock_sio.emit.assert_called_once_with(event, data, room=game_id)

    async def test_emit_to_user(self, manager, mock_sio):
        """Test emitting to specific user"""
        # Arrange
        sid = "test-session-123"
        event = "test_event"
        data = {"key": "value"}

        # Act
        await manager.emit_to_user(sid, event, data)

        # Assert
        mock_sio.emit.assert_called_once_with(event, data, room=sid)

    async def test_get_game_participants(self, manager):
        """Test getting participants in game room"""
        # Arrange
        game_id = str(uuid4())
        sid1 = "session-1"
        sid2 = "session-2"

        await manager.connect(sid1, "user-1")
        await manager.connect(sid2, "user-2")
        await manager.join_game(sid1, game_id, "player")
        await manager.join_game(sid2, game_id, "player")

        # Act
        participants = manager.get_game_participants(game_id)

        # Assert
        assert participants == {sid1, sid2}

---

Success Criteria

Minimum Acceptable Coverage

To reach a "Good" coverage rating, the following tests must be added:

• ✅ ConnectionManager: 10-15 tests (currently 0)
• ✅ Authentication handler: 5-7 tests (currently 0)
• ✅ Substitution handlers: 30 tests (currently 0)
• ✅ Room management: 8-10 tests (currently 0)
• ✅ Lineup handler: 5-6 tests (currently 0)

Total New Tests Needed: ~58-73 tests Estimated Effort: ~10-15 hours

Target Coverage

Metric	Current	Target	Status
Handler Coverage	36%	100%	❌ Gap
ConnectionManager Tests	0	10-15	❌ Gap
Total Tests	14	70-80	❌ Gap
Critical Paths	18%	90%+	❌ Gap

---

Conclusion

The websocket module has selective but excellent coverage of the most critical gameplay handlers (dice rolling and outcome submission), but lacks comprehensive testing of infrastructure (ConnectionManager) and newer features (substitutions).

Key Strengths:

• Core gameplay handlers well-tested
• Good testing patterns established
• All existing tests passing
• Clear test organization

Key Weaknesses:

• ConnectionManager completely untested
• Substitution handlers untested (73% of handler code)
• Connection lifecycle untested
• Room management untested

Recommendation: Prioritize adding tests for ConnectionManager and substitution handlers to bring coverage to acceptable levels before production deployment. The existing test patterns provide a good template for expansion.

Overall Grade: C+ (Fair)

• Well-tested critical path
• Major infrastructure gaps
• Security concerns (auth handler untested)
• New features untested

---

Report Generated: 2025-11-04 Next Review: After adding recommended tests Contact: Development Team