claude-home/paper-dynasty/card-evolution-prd/02-architecture.md

# 2. System Architecture Overview

[< Back to Index](README.md) | [Next: Tracks >](03-tracks.md)

---

## Stats Tracking Approach: Lazy Evaluation Against Existing Data

Evolution progress is computed on-demand by querying the existing `stratplay`, `decision`, and
`stratgame` tables at the `player_id` level. There are no new event tables or pipeline changes
required.

The `stratplay` table already records `player_id` for every plate appearance, with full counting
stats (hits, HR, BB, SO, etc.). The `decision` table already records pitcher wins, saves, holds.
The `stratgame` table records game outcomes by team. All milestone types can be satisfied by
querying this existing data.

**Progress is team-scoped.** Milestones count stats accumulated by `player_id` in games played
by the *owning team*, starting from when the team acquired the card (`progress_since`). This means
evolution rewards your team's history with a player. If a card is traded (when trading ships),
progress resets for the new team.

A card that has been on a roster for months before this system launches gets credit for all past
games played by that team. This is intentional — it gives the feature immediate retroactive depth
rather than a cold start.

**What this means in practice:**
- "Hit 10 HRs" → `SELECT SUM(homerun) FROM stratplay sp JOIN stratgame sg ON sp.game_id = sg.id WHERE sp.batter_id = {player_id} AND (sg.away_team = {team_id} OR sg.home_team = {team_id}) AND sg.created_at >= {progress_since}`
- "Win 5 games" → `SELECT COUNT(*) FROM stratgame WHERE (away_team = {team_id} AND away_score > home_score OR home_team = {team_id} AND home_score > away_score) AND created_at >= {progress_since}`
- "Record 8 saves" → `SELECT COUNT(*) FROM decision d JOIN stratgame sg ON d.game_id = sg.id WHERE d.pitcher = {player_id} AND d.is_save = true AND (sg.away_team = {team_id} OR sg.home_team = {team_id})`

### Materialized Aggregate Stats Table

Evolution milestone evaluation requires counting stats like hits, HRs, saves, etc. for a
`(player_id, team_id)` pair. Currently, all stat aggregation is done on-demand via
`GROUP BY + SUM()` against raw `stratplay` rows (one row per plate appearance). This gets
progressively slower as the table grows.

A **`player_season_stats`** table should be created as an early deliverable to provide
pre-computed season totals. This table is updated incrementally in the post-game callback
(add the game's deltas to running totals) rather than recomputed from scratch. Milestone
evaluation then becomes a single row lookup instead of a table scan.

**This pattern is already implemented in the Major Domo project** (SBA fantasy baseball) for
the same purpose — quick lookup of large sample size stats. Reference that implementation for
schema design and the incremental update logic.

The aggregate table benefits the entire system beyond evolution: leaderboards, scouting,
player comparison, and any future feature that needs season totals.

See [06-database.md](06-database.md) for the `player_season_stats` schema.

**The `evolution_card_state` table** is a cached materialization of the current tier per
`(player_id, team_id)` pair. It is the source of display truth for Discord embeds and the card
API. The source of truth for progress is always the raw game data tables. The cached state is
refreshed after each game completes (post-game callback) and can be invalidated on-demand via
the evaluate endpoint.

**Duplicate cards of the same player_id** on the same team are identical for evolution purposes.
They share the same `evolution_card_state`, the same tier, the same boosts, the same cosmetics,
and the same variant hash. The evolution system sees `(player_id, team_id)` as its fundamental
unit — individual `card_id` instances are not differentiated.

**Different-year versions of the same real-world player** (e.g., 2019 Mike Trout vs 2024 Mike
Trout) have different `player_id` values and evolve independently. They are completely separate
players in the database with separate stats, ratings, and evolution tracks.

## Component Map

```
Discord Bot (game engine)
    |
    |-- [existing] stratplay pipeline (player_id level, UNCHANGED)
    |
    |-- [NEW] post-game evolution evaluator
            |
            +-- queries stratplay/decision/stratgame for milestone progress
            +-- updates evolution_card_state table (cached tier per card)
            +-- sends Discord notifications on milestone/tier completion
            +-- applies auto-detected boost profile and cosmetics

Paper Dynasty API (database service)
    |
    +-- [NEW] /v2/evolution/* endpoints
    +-- [NEW] /v2/evolution/evaluate/{card_id} -- recalculate tier from source tables
    +-- [MODIFIED] /v2/players/{id}/battingcard -- aware of evolution variant
    +-- [MODIFIED] /v2/players/{id}/pitchingcard -- aware of evolution variant

Card Creation System (this repo)
    |
    +-- [MODIFIED] variant=0 -> variant=1+ for evolved card ratings
    +-- [NEW] apply_evolution_boosts() utility
    +-- [NEW] APNG render pipeline for animated cosmetics
    +-- [MODIFIED] concurrent upload pipeline with semaphore-bounded asyncio.gather
```

---

## Card Render Pipeline Optimization

### Current State (Pre-Optimization)

The existing card image render pipeline in `database/app/routers_v2/players.py` launches a
**new Chromium process for every single card render**:

```python
# Current: ~3 seconds per card
async with async_playwright() as p:
    browser = await p.chromium.launch()          # ~1.0-1.5s: spawn Chromium
    page = await browser.new_page()
    await page.set_content(html_string)          # ~0.3-0.5s: parse HTML + fetch Google Fonts from CDN
    await page.screenshot(path=..., clip=...)    # ~0.5-0.8s: layout + paint + PNG encode
    await browser.close()                        # teardown
```

The upload pipeline in `pd_cards/core/upload.py` is fully sequential — one card at a time in a
`for` loop, no concurrency. A typical 800-card run takes **~40 minutes**.

**Breakdown of the ~3 seconds per card:**

| Step | Time | Notes |
|---|---|---|
| Chromium launch | ~1.0-1.5s | New OS process per card |
| Google Fonts CDN fetch | ~0.3-0.5s | Network round-trip during `set_content()` |
| HTML parse + layout + paint | ~0.5-0.8s | Skia software rasterizer, 1200x600 |
| PNG screenshot encode | ~0.3-0.5s | PNG compression of framebuffer |
| DB rating write-back + disk write | ~0.2-0.3s | Two UPDATE statements + file I/O |

### Optimization 1: Persistent Browser (Priority: Immediate)

Launch Chromium **once** at API startup and reuse it across all render requests. This eliminates
the ~1.0-1.5 second launch/teardown overhead per card.

**Implementation in `database/app/routers_v2/players.py`:**

```python
# Module-level: persistent browser instance
_browser: Browser | None = None
_playwright: Playwright | None = None

async def get_browser() -> Browser:
    """Get or create persistent Chromium browser instance."""
    global _browser, _playwright
    if _browser is None or not _browser.is_connected():
        _playwright = await async_playwright().start()
        _browser = await _playwright.chromium.launch(
            args=["--no-sandbox", "--disable-dev-shm-usage"]
        )
    return _browser

async def shutdown_browser():
    """Clean shutdown on API exit. Register with FastAPI lifespan."""
    global _browser, _playwright
    if _browser:
        await _browser.close()
    if _playwright:
        await _playwright.stop()
```

**Per-render request (replaces current `async with async_playwright()` block):**

```python
browser = await get_browser()
page = await browser.new_page(viewport={"width": 1280, "height": 720})
try:
    await page.set_content(html_response.body.decode("UTF-8"))
    await page.screenshot(
        path=file_path,
        type="png",
        clip={"x": 0, "y": 0, "width": 1200, "height": 600},
    )
finally:
    await page.close()  # ~2ms vs ~1500ms for full browser close+relaunch
```

**FastAPI lifespan integration:**

```python
from contextlib import asynccontextmanager

@asynccontextmanager
async def lifespan(app: FastAPI):
    # Startup: warm up browser
    await get_browser()
    yield
    # Shutdown: clean up
    await shutdown_browser()

app = FastAPI(lifespan=lifespan)
```

**Expected result:** ~3.0s → ~1.0-1.5s per card (browser overhead eliminated).

**Safety:** If the browser crashes or disconnects, `get_browser()` detects via
`_browser.is_connected()` and relaunches automatically. Page-level errors are isolated — a
crashed page does not affect other pages in the same browser.

### Optimization 2: Self-Hosted Fonts (Priority: Immediate)

The card HTML template loads Google Fonts via CDN during every `set_content()` call:

```css
/* Current: network round-trip on every render */
@import url('https://fonts.googleapis.com/css2?family=Source+Sans+3:wght@400;700&display=swap');
```

Self-hosting eliminates the ~0.3-0.5s network dependency. Two approaches:

**Option A: Base64-embed in template (simplest, no infrastructure change):**

```css
@font-face {
  font-family: 'Source Sans 3';
  font-weight: 400;
  src: url(data:font/woff2;base64,<base64_data>) format('woff2');
}
@font-face {
  font-family: 'Source Sans 3';
  font-weight: 700;
  src: url(data:font/woff2;base64,<base64_data>) format('woff2');
}
```

**Option B: Local filesystem served via Playwright's `route` API:**

```python
await page.route("**/fonts.googleapis.com/**", lambda route: route.fulfill(
    path="/app/storage/fonts/source-sans-3.css"
))
await page.route("**/fonts.gstatic.com/**", lambda route: route.fulfill(
    path=f"/app/storage/fonts/{route.request.url.split('/')[-1]}"
))
```

Option A is preferred — it makes the template fully self-contained with zero external
dependencies, which also makes it work in airgapped/offline environments.

**Expected result:** Additional ~0.3-0.5s saved. Combined with persistent browser: **~0.6-1.0s per card**.

### Optimization 3: Concurrent Upload Pipeline (Priority: High)

The upload loop in `pd_cards/core/upload.py` processes cards sequentially. Adding
semaphore-bounded concurrency allows multiple cards to render and upload in parallel:

```python
import asyncio

async def upload_cardset(all_players, session, concurrency=8):
    """Upload cards with bounded concurrency."""
    sem = asyncio.Semaphore(concurrency)
    results = []

    async def process_card(player):
        async with sem:
            image_bytes = await fetch_card_image(session, card_url, timeout=10)
            if image_bytes:
                s3_url = await upload_card_to_s3(image_bytes, ...)
                await update_player_url(session, player_id, s3_url)
                return player_id
            return None

    results = await asyncio.gather(
        *[process_card(p) for p in all_players],
        return_exceptions=True
    )

    successes = [r for r in results if r is not None and not isinstance(r, Exception)]
    failures = [r for r in results if isinstance(r, Exception)]
    return successes, failures
```

**Concurrency tuning:** The API server's Chromium instance handles multiple pages concurrently
within a single browser process. Each page consumes ~50-100 MB RAM. With 16 GB server RAM:

| Concurrency | RAM Usage | Cards/Minute | 800 Cards |
|---|---|---|---|
| 1 (current) | ~200 MB | ~60-100 | ~8-13 min |
| 4 | ~600 MB | ~240 | ~3-4 min |
| 8 | ~1 GB | ~480 | ~2 min |
| 16 | ~2 GB | ~800+ | ~1 min |

Start with `concurrency=8` and tune based on server load. The `fetch_card_image` timeout should
increase from 6s to 10s to account for render queue contention.

**Expected result:** Combined with persistent browser + local fonts, an 800-card run goes from
**~40 minutes to ~2-4 minutes**.

### Optimization 4: GPU Acceleration (Priority: Skip)

Chromium supports `--enable-gpu` and `--use-gl=egl` for hardware-accelerated rendering. However:

- Card images are flat 2D layout — Skia's software rasterizer is already fast for this
- GPU passthrough in Docker requires NVIDIA container toolkit + EGL libraries
- Expected gain for 2D card images: **~5-10%** — not worth the infrastructure complexity
- GPU becomes relevant if we add WebGL effects or complex CSS 3D transforms in the future

**Recommendation:** Skip GPU acceleration. The persistent browser + concurrency changes deliver
a **20-40x total speedup** without touching infrastructure.

### Summary: Expected Performance After Optimization

| Metric | Before | After | Improvement |
|---|---|---|---|
| Per-card render time | ~3.0s | ~0.6-1.0s | ~3-5x |
| 800-card sequential run | ~40 min | ~8-13 min | ~3-5x |
| 800-card concurrent run (8x) | N/A | ~2-4 min | ~10-20x vs current |
| External dependencies | Google Fonts CDN | None | Offline-capable |
| Chromium processes spawned | 800 | 1 | 800x fewer |

### Implementation Files

| Change | File | Scope |
|---|---|---|
| Persistent browser | `database/app/routers_v2/players.py` | Replace `async with async_playwright()` block |
| FastAPI lifespan | `database/app/main.py` (or equivalent) | Add startup/shutdown hooks |
| Self-hosted fonts | `database/storage/templates/style.html` | Replace `@import` with `@font-face` |
| Font files | `database/storage/fonts/` | Download Source Sans 3 + Open Sans WOFF2 files |
| Concurrent upload | `pd_cards/core/upload.py` | Replace sequential loop with `asyncio.gather` |
| Timeout increase | `pd_cards/core/upload.py` | `fetch_card_image` timeout: 6s → 10s |