strat-chatbot/tests/adapters/test_api.py
Cal Corum c3218f70c4 refactor: hexagonal architecture with ports & adapters, DI, and test-first development
Domain layer (zero framework imports):
- domain/models.py: pure dataclasses (RuleDocument, RuleSearchResult,
  Conversation, ChatMessage, LLMResponse, ChatResult)
- domain/ports.py: ABC interfaces (RuleRepository, LLMPort,
  ConversationStore, IssueTracker)
- domain/services.py: ChatService orchestrates Q&A flow using only ports

Outbound adapters (implement domain ports):
- adapters/outbound/openrouter.py: OpenRouterLLM with persistent httpx
  client, robust JSON parsing, regex citation fallback
- adapters/outbound/sqlite_convos.py: SQLiteConversationStore with
  async_sessionmaker, timezone-aware datetimes, cleanup support
- adapters/outbound/gitea_issues.py: GiteaIssueTracker with markdown
  injection protection (fenced code blocks)
- adapters/outbound/chroma_rules.py: ChromaRuleRepository with clamped
  similarity scores

Inbound adapter:
- adapters/inbound/api.py: thin FastAPI router with input validation
  (max_length constraints), proper HTTP status codes (503 for missing LLM)

Configuration & wiring:
- config/settings.py: Pydantic v2 SettingsConfigDict (no module-level singleton)
- config/container.py: create_app() factory with lifespan-managed DI
- main.py: minimal entry point

Test infrastructure (90 tests, all passing):
- tests/fakes/: in-memory implementations of all 4 ports
- tests/domain/: 26 tests for models and ChatService
- tests/adapters/: 64 tests for all adapters using fakes/mocks
- No real API calls, no model downloads, no disk I/O in fast tests

Also fixes: aiosqlite version constraint (>=0.19.0), adds hatch build
targets for new package layout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 15:51:16 -05:00

439 lines
15 KiB
Python

"""Tests for the FastAPI inbound adapter (adapters/inbound/api.py).
Strategy
--------
We build a minimal FastAPI app in each fixture by wiring fakes into app.state,
then drive it with httpx.AsyncClient using ASGITransport so no real HTTP server
is needed. This means:
- No real ChromaDB, SQLite, OpenRouter, or Gitea calls.
- Tests are fast, deterministic, and isolated.
- The test app mirrors exactly what the production container does — the only
difference is which objects sit in app.state.
What is tested
--------------
- POST /chat returns 200 and a well-formed ChatResponse for a normal message.
- POST /chat stores the conversation and returns a stable conversation_id on a
second call with the same conversation_id (conversation continuation).
- GET /health returns {"status": "healthy", ...} with rule counts.
- GET /stats returns a knowledge_base sub-dict and a config sub-dict.
- POST /chat with missing required fields returns HTTP 422 (Unprocessable Entity).
- POST /chat with a message that exceeds 4000 characters returns HTTP 422.
- POST /chat with a user_id that exceeds 64 characters returns HTTP 422.
- POST /chat when ChatService.answer_question raises returns HTTP 500.
"""
from __future__ import annotations
import pytest
import httpx
from fastapi import FastAPI
from httpx import ASGITransport
from domain.models import RuleDocument
from domain.services import ChatService
from adapters.inbound.api import router
from tests.fakes import (
FakeRuleRepository,
FakeLLM,
FakeConversationStore,
FakeIssueTracker,
)
# ---------------------------------------------------------------------------
# Test app factory
# ---------------------------------------------------------------------------
def make_test_app(
*,
rules: FakeRuleRepository | None = None,
llm: FakeLLM | None = None,
conversations: FakeConversationStore | None = None,
issues: FakeIssueTracker | None = None,
top_k_rules: int = 5,
) -> FastAPI:
"""Build a minimal FastAPI app with fakes wired into app.state.
The factory mirrors what config/container.py does in production, but uses
in-memory fakes so no external services are needed. Each test that calls
this gets a fresh, isolated set of fakes unless shared fixtures are passed.
"""
_rules = rules or FakeRuleRepository()
_llm = llm or FakeLLM()
_conversations = conversations or FakeConversationStore()
_issues = issues or FakeIssueTracker()
service = ChatService(
rules=_rules,
llm=_llm,
conversations=_conversations,
issues=_issues,
top_k_rules=top_k_rules,
)
app = FastAPI()
app.include_router(router)
app.state.chat_service = service
app.state.rule_repository = _rules
app.state.config_snapshot = {
"openrouter_model": "fake-model",
"top_k_rules": top_k_rules,
"embedding_model": "fake-embeddings",
}
return app
# ---------------------------------------------------------------------------
# Shared fixture: an async client backed by the test app
# ---------------------------------------------------------------------------
@pytest.fixture()
async def client() -> httpx.AsyncClient:
"""Return an AsyncClient wired to a fresh test app.
Each test function gets its own completely isolated set of fakes so that
state from one test cannot leak into another.
"""
app = make_test_app()
async with httpx.AsyncClient(
transport=ASGITransport(app=app), base_url="http://test"
) as ac:
yield ac
# ---------------------------------------------------------------------------
# POST /chat — successful response
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_chat_returns_200_with_valid_payload(client: httpx.AsyncClient):
"""A well-formed POST /chat request must return HTTP 200 and a response body
that maps one-to-one with the ChatResponse Pydantic model.
We verify every field so a structural change to ChatResult or ChatResponse
is caught immediately rather than silently producing a wrong value.
"""
payload = {
"message": "How many strikes to strike out?",
"user_id": "user-001",
"channel_id": "channel-001",
}
resp = await client.post("/chat", json=payload)
assert resp.status_code == 200
body = resp.json()
assert isinstance(body["response"], str)
assert len(body["response"]) > 0
assert isinstance(body["conversation_id"], str)
assert isinstance(body["message_id"], str)
assert isinstance(body["cited_rules"], list)
assert isinstance(body["confidence"], float)
assert isinstance(body["needs_human"], bool)
@pytest.mark.asyncio
async def test_chat_uses_rules_when_available():
"""When the FakeRuleRepository has documents matching the query, the FakeLLM
receives them and returns a high-confidence answer with cited_rules populated.
This exercises the full ChatService flow through the inbound adapter.
"""
rules_repo = FakeRuleRepository()
rules_repo.add_documents(
[
RuleDocument(
rule_id="1.1",
title="Batting Order",
section="Batting",
content="A batter gets three strikes before striking out.",
source_file="rules.pdf",
)
]
)
app = make_test_app(rules=rules_repo)
async with httpx.AsyncClient(
transport=ASGITransport(app=app), base_url="http://test"
) as ac:
resp = await ac.post(
"/chat",
json={
"message": "How many strikes before a batter strikes out?",
"user_id": "user-abc",
"channel_id": "ch-xyz",
},
)
assert resp.status_code == 200
body = resp.json()
# FakeLLM returns cited_rules when rules are found
assert len(body["cited_rules"]) > 0
assert body["confidence"] > 0.5
# ---------------------------------------------------------------------------
# POST /chat — conversation continuation
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_chat_continues_existing_conversation():
"""Supplying conversation_id in the request should resume the same
conversation rather than creating a new one.
We make two requests: the first creates a conversation and returns its ID;
the second passes that ID back and must return the same conversation_id.
This ensures the FakeConversationStore (and real SQLite adapter) behave
consistently from the router's perspective.
"""
conversations = FakeConversationStore()
app = make_test_app(conversations=conversations)
async with httpx.AsyncClient(
transport=ASGITransport(app=app), base_url="http://test"
) as ac:
# First turn — no conversation_id
resp1 = await ac.post(
"/chat",
json={
"message": "First question",
"user_id": "user-42",
"channel_id": "ch-1",
},
)
assert resp1.status_code == 200
conv_id = resp1.json()["conversation_id"]
# Second turn — same conversation
resp2 = await ac.post(
"/chat",
json={
"message": "Follow-up question",
"user_id": "user-42",
"channel_id": "ch-1",
"conversation_id": conv_id,
},
)
assert resp2.status_code == 200
assert resp2.json()["conversation_id"] == conv_id
# ---------------------------------------------------------------------------
# GET /health
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_health_returns_healthy_status(client: httpx.AsyncClient):
"""GET /health must return {"status": "healthy", ...} with integer rule count
and a sections dict.
The FakeRuleRepository starts empty so rules_count should be 0.
"""
resp = await client.get("/health")
assert resp.status_code == 200
body = resp.json()
assert body["status"] == "healthy"
assert isinstance(body["rules_count"], int)
assert isinstance(body["sections"], dict)
@pytest.mark.asyncio
async def test_health_reflects_loaded_rules():
"""After adding documents to FakeRuleRepository, GET /health must show the
updated rule count. This confirms the router reads a live reference to the
repository, not a snapshot taken at startup.
"""
rules_repo = FakeRuleRepository()
rules_repo.add_documents(
[
RuleDocument(
rule_id="2.1",
title="Pitching",
section="Pitching",
content="The pitcher throws the ball.",
source_file="rules.pdf",
)
]
)
app = make_test_app(rules=rules_repo)
async with httpx.AsyncClient(
transport=ASGITransport(app=app), base_url="http://test"
) as ac:
resp = await ac.get("/health")
assert resp.status_code == 200
assert resp.json()["rules_count"] == 1
# ---------------------------------------------------------------------------
# GET /stats
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_stats_returns_knowledge_base_and_config(client: httpx.AsyncClient):
"""GET /stats must include a knowledge_base sub-dict (from RuleRepository.get_stats)
and a config sub-dict (from app.state.config_snapshot set by the container).
This ensures the stats endpoint exposes enough information for an operator
to confirm what model and retrieval settings are active.
"""
resp = await client.get("/stats")
assert resp.status_code == 200
body = resp.json()
assert "knowledge_base" in body
assert "config" in body
assert "total_rules" in body["knowledge_base"]
# ---------------------------------------------------------------------------
# POST /chat — validation errors (HTTP 422)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_chat_missing_message_returns_422(client: httpx.AsyncClient):
"""Omitting the required 'message' field must trigger Pydantic validation and
return HTTP 422 Unprocessable Entity with a detail array describing the error.
We do NOT want a 500 — a missing field is a client error, not a server error.
"""
resp = await client.post("/chat", json={"user_id": "u1", "channel_id": "ch1"})
assert resp.status_code == 422
@pytest.mark.asyncio
async def test_chat_missing_user_id_returns_422(client: httpx.AsyncClient):
"""Omitting 'user_id' must return HTTP 422."""
resp = await client.post("/chat", json={"message": "Hello", "channel_id": "ch1"})
assert resp.status_code == 422
@pytest.mark.asyncio
async def test_chat_missing_channel_id_returns_422(client: httpx.AsyncClient):
"""Omitting 'channel_id' must return HTTP 422."""
resp = await client.post("/chat", json={"message": "Hello", "user_id": "u1"})
assert resp.status_code == 422
@pytest.mark.asyncio
async def test_chat_message_too_long_returns_422(client: httpx.AsyncClient):
"""A message that exceeds 4000 characters must fail field-level validation
and return HTTP 422 rather than passing to the service layer.
The max_length constraint on ChatRequest.message enforces this.
"""
long_message = "x" * 4001
resp = await client.post(
"/chat",
json={"message": long_message, "user_id": "u1", "channel_id": "ch1"},
)
assert resp.status_code == 422
@pytest.mark.asyncio
async def test_chat_user_id_too_long_returns_422(client: httpx.AsyncClient):
"""A user_id that exceeds 64 characters must return HTTP 422.
Discord snowflakes are at most 20 digits; 64 chars is a generous cap that
still prevents runaway strings from reaching the database layer.
"""
long_user_id = "u" * 65
resp = await client.post(
"/chat",
json={"message": "Hello", "user_id": long_user_id, "channel_id": "ch1"},
)
assert resp.status_code == 422
@pytest.mark.asyncio
async def test_chat_channel_id_too_long_returns_422(client: httpx.AsyncClient):
"""A channel_id that exceeds 64 characters must return HTTP 422."""
long_channel_id = "c" * 65
resp = await client.post(
"/chat",
json={"message": "Hello", "user_id": "u1", "channel_id": long_channel_id},
)
assert resp.status_code == 422
@pytest.mark.asyncio
async def test_chat_empty_message_returns_422(client: httpx.AsyncClient):
"""An empty string for 'message' must fail min_length=1 and return HTTP 422.
We never want an empty string propagated to the LLM — it would produce a
confusing response and waste tokens.
"""
resp = await client.post(
"/chat", json={"message": "", "user_id": "u1", "channel_id": "ch1"}
)
assert resp.status_code == 422
# ---------------------------------------------------------------------------
# POST /chat — service-layer exception bubbles up as 500
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_chat_service_exception_returns_500():
"""When ChatService.answer_question raises an unexpected exception the router
must catch it and return HTTP 500, not let the exception propagate and crash
the server process.
We use FakeLLM(force_error=...) to inject the failure deterministically.
"""
broken_llm = FakeLLM(force_error=RuntimeError("LLM exploded"))
app = make_test_app(llm=broken_llm)
async with httpx.AsyncClient(
transport=ASGITransport(app=app), base_url="http://test"
) as ac:
resp = await ac.post(
"/chat",
json={"message": "Hello", "user_id": "u1", "channel_id": "ch1"},
)
assert resp.status_code == 500
assert "LLM exploded" in resp.json()["detail"]
# ---------------------------------------------------------------------------
# POST /chat — parent_message_id thread reply
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_chat_with_parent_message_id_returns_200(client: httpx.AsyncClient):
"""Supplying the optional parent_message_id must not cause an error.
The field passes through to ChatService and ends up in the conversation
store. We just assert a 200 here — the service-layer tests cover the
parent_id wiring in more detail.
"""
resp = await client.post(
"/chat",
json={
"message": "Thread reply",
"user_id": "u1",
"channel_id": "ch1",
"parent_message_id": "some-parent-uuid",
},
)
assert resp.status_code == 200
body = resp.json()
# The response's parent_message_id is the user turn message id,
# not the one we passed in — that's the service's threading model.
assert body["parent_message_id"] is not None