paper-dynasty-card-creation/docs/REFRACTOR_PHASE2_VALIDATION_SPEC.md

# Refractor Phase 2 — Design Validation Spec

## Purpose

This document captures the design validation test cases that must be verified before and during
Phase 2 (rating boosts) of the Refractor card progression system. Phase 1 — tracking,
milestone evaluation, and tier state persistence — is implemented. Phase 2 adds the rating boost
application logic (`apply_evolution_boosts`), rarity upgrade at T4, and variant hash creation.

**When to reference this document:**

- Before beginning Phase 2 implementation: review all cases to understand the design constraints
  and edge cases the implementation must handle.
- During implementation: use each test case as an acceptance gate before the corresponding
  feature is considered complete.
- During code review: each case documents the "risk if failed" so reviewers can assess whether
  a proposed implementation correctly handles that scenario.
- After Phase 2 ships: run the cases as a regression checklist before any future change to the
  boost logic, rarity assignment, or milestone evaluator.

## Background: Rating Model

Batter cards have 22 outcome columns summing to exactly 108 chances (derived from the D20
probability system: 2d6 x 3 columns x 6 rows). Each Refractor tier (T1 through T4) awards a
1.0-chance budget — a flat shift from out columns to positive-outcome columns. The total
accumulated budget across all four tiers is 4.0 chances, equal to approximately 3.7% of the
108-chance total (4 / 108 ≈ 0.037).

Rarity IDs in the codebase (from `rarity_thresholds.py`):

| Rarity Name  | ID  |
|---|---|
| Common       | 5   |
| Bronze       | 4   |
| Silver       | 3   |
| Gold         | 2   |
| Diamond      | 1   |
| Hall of Fame | 99  |

The special value `99` for Hall of Fame means a naive `rarity_id + 1` increment is incorrect;
the upgrade logic must use an ordered rarity ladder, not arithmetic.

---

## Test Cases

---

### T4-1: 108-sum preservation under profile-based boosts

**Status:** Pending — Phase 2

**Scenario:**

`apply_evolution_boosts(card_ratings, boost_tier, player_profile)` redistributes 1.0 chance per
tier across outcome columns according to the player's detected profile (power hitter, contact
hitter, patient hitter, starting pitcher, relief pitcher). Every combination of profile and tier
must leave the 22-column sum exactly equal to 108 after the boost is applied. This must hold for
all four tier applications, cumulative as well as individual.

The edge case: a batter card where `flyout_a = 0`. The power and contact hitter profiles draw
reductions from out columns including `flyout_a`. If the preferred reduction column is at zero,
the implementation must not produce a negative value and must not silently drop the remainder of
the budget. The 0-floor cap is enforced per column (see `05-rating-boosts.md` section 5.1:
"Truncated points are lost, not redistributed").

Verify:
- After each of T1, T2, T3, T4 boost applications, `sum(all outcome columns) == 108` exactly.
- A card with `flyout_a = 0` does not raise an error and does not produce a column below 0.
- When truncation occurs (column already at 0), the lost budget is discarded, not moved
  elsewhere — the post-boost sum will be less than 108 + budget_added only in the case of
  truncation, but must never exceed 108.

**Expected Outcome:**

Sum remains 108 after every boost under non-truncation conditions. Under truncation conditions
(a column hits 0), the sum is reduced by the truncated amount — the implementation discards the
excess rather than redistributing it. No column value falls below 0.

**Risk If Failed:**

A broken 108-sum produces invalid game probabilities. The D20 engine derives per-outcome
probabilities from `column / 108`. If the sum drifts above or below 108, every outcome
probability on that card is subtly wrong for every future game that uses it. This error silently
corrupts game results without any visible failure.

**Files Involved:**

- `docs/prd-evolution/05-rating-boosts.md` — boost budget, profile definitions, cap behavior
- Phase 2: `pd_cards/evo/boost_profiles.py` (to be created) — `apply_evolution_boosts`
- `batters/creation.py` — `battingcardratings` model column set (22 columns)
- `pitchers/creation.py` — `pitchingcardratings` model column set (18 columns + 9 x-checks)

---

### T4-2: D20 probability shift at T4

**Status:** Pending — Phase 2

**Scenario:**

Take a representative Bronze-rarity batter (e.g., a player with total OPS near 0.730,
`homerun` ≈ 1.2, `single_one` ≈ 4.0, `walk` ≈ 3.0 in the base ratings). Apply all four
tier boosts cumulatively, distributing the total 4.0-chance budget across positive-outcome
columns (HR, singles, walk) with equal reductions from out columns. Calculate the resulting
absolute and relative probability change per D20 roll outcome.

Design target: the full T4 evolution shifts approximately 3.7% of all outcomes from outs to
positive results (4.0 / 108 = 0.037). The shift should be perceptible to a player reviewing
their card stats but should not fundamentally alter the card's tier or role. A Bronze batter
does not become a Gold batter through evolution — they become an evolved Bronze batter.

Worked example for validation reference:
- Pre-evolution: `homerun = 1.2` → probability per D20 = 1.2 / 108 ≈ 1.11%
- Post T4 with +0.5 to homerun per tier (4 tiers × 0.5 = +2.0 total): `homerun = 3.2`
  → probability per D20 = 3.2 / 108 ≈ 2.96% — an increase of ~1.85 percentage points
- Across all positive outcomes: total shift = 4.0 / 108 ≈ 3.7%

**Expected Outcome:**

The cumulative 4.0-chance shift produces a ~3.7% total movement from negative to positive
outcomes. No single outcome column increases by more than 2.5 chances across the full T4
journey under any profile. The card remains recognizably Bronze — it does not cross the Gold
OPS threshold (0.700 for 2024/2025 thresholds) unless it was already near the boundary.

**Risk If Failed:**

If the shift is too large, evolution becomes a rarity bypass — players grind low-rarity cards
to simulate an upgrade they cannot earn through pack pulls. If the shift is too small, the
system feels unrewarding and players lose motivation to complete tiers. Either miscalibration
undermines the core design intent.

**Files Involved:**

- `docs/prd-evolution/05-rating-boosts.md` — section 5.2 (boost budgets), section 5.3 (profiles)
- `rarity_thresholds.py` — OPS boundary values used to assess whether evolution crosses a rarity
  threshold as a side effect (it should not for mid-range cards)
- Phase 2: `pd_cards/evo/boost_profiles.py` — boost distribution logic

---

### T4-3: T4 rarity upgrade — pipeline collision risk

**Status:** Pending — Phase 2

**Scenario:**

The Refractor T4 rarity upgrade (`player.rarity_id` incremented by one ladder step) and the
live-series `post_player_updates()` rarity assignment (OPS-threshold-based, in
`batters/creation.py`) both write to the same `rarity_id` field on the player record. A
collision occurs when both run against the same player:

1. Player completes Refractor T4. Evolution system upgrades rarity: Bronze (4) → Silver (3).
   `evolution_card_state.final_rarity_id = 3` is written as an audit record.
2. Live-series update runs two weeks later. `post_player_updates()` recalculates OPS → maps to
   Bronze (4) → writes `rarity_id = 4` to the player record.
3. The T4 rarity upgrade is silently overwritten. The player's card reverts to Bronze. The
   `evolution_card_state` record still shows `final_rarity_id = 3` but the live card is Bronze.

This is a conflict between two independent systems both writing to the same field without
awareness of each other. The current live-series pipeline has no concept of evolution state.

Proposed resolution strategies (document and evaluate; do not implement during Phase 2 spec):
- **Guard clause in `post_player_updates()`:** Before writing `rarity_id`, check
  `evolution_card_state.final_rarity_id` for the player. If an evolution upgrade is on record,
  apply `max(ops_rarity, final_rarity_id_ladder_position)` — never downgrade past the T4 result.
- **Separate evolution rarity field:** Add `evolution_rarity_bump` (int, default 0) to the
  card model. The game engine resolves effective rarity as `base_rarity + bump`. Live-series
  updates only touch `base_rarity`; the bump is immutable once T4 is reached.
- **Deferred rarity upgrade:** T4 does not write `rarity_id` immediately. Instead, it sets a
  flag on `evolution_card_state`. `post_player_updates()` checks the flag and applies the bump
  after its own rarity calculation, ensuring the evolution upgrade layers on top of the current
  OPS-derived rarity rather than competing with it.

**Expected Outcome:**

Phase 2 must implement one of these strategies (or an alternative that provides equivalent
protection). The collision scenario must be explicitly tested: evolve a Bronze card to T4,
run a live-series update that maps the same player to Bronze, confirm the displayed rarity is
Silver or higher — not Bronze.

**Risk If Failed:**

Live-series updates silently revert T4 rarity upgrades. Players invest significant game time
reaching T4, receive the visual rarity upgrade, then lose it after the next live-series run
with no explanation. This is one of the highest-trust violations the system can produce — a
reward that disappears invisibly.

**Files Involved:**

- `batters/creation.py` — `post_player_updates()` (lines ~304–480)
- `pitchers/creation.py` — equivalent `post_player_updates()` for pitchers
- `docs/prd-evolution/05-rating-boosts.md` — section 5.4 (rarity upgrade at T4), note on live
  series interaction
- Phase 2: `pd_cards/evo/tier_completion.py` (to be created) — T4 completion handler
- Database: `evolution_card_state` table, `final_rarity_id` column

---

### T4-4: T4 rarity cap for HoF cards

**Status:** Pending — Phase 2

**Scenario:**

A player card currently at Hall of Fame rarity (`rarity_id = 99`) completes Refractor T4. The
design specifies: HoF cards receive the T4 rating boost deltas (1.0 chance shift) but do not
receive a rarity upgrade. The rarity stays at 99.

The implementation must handle this without producing an invalid rarity value. The rarity ID
sequence in `rarity_thresholds.py` is non-contiguous — the IDs are:

```
5 (Common) → 4 (Bronze) → 3 (Silver) → 2 (Gold) → 1 (Diamond) → 99 (Hall of Fame)
```

A naive `rarity_id + 1` would produce `100`, which is not a valid rarity. A lookup-table
approach on the ordered ladder must be used instead. At `99` (HoF), the ladder returns `99`
(no-op). Additionally, Diamond (1) cards that complete T4 should upgrade to HoF (99), not to
`rarity_id = 0` or any other invalid value.

**Expected Outcome:**

- `rarity_id = 99` (HoF): T4 boost applied, rarity unchanged at 99.
- `rarity_id = 1` (Diamond): T4 boost applied, rarity upgrades to 99 (HoF).
- `rarity_id = 2` (Gold): T4 boost applied, rarity upgrades to 1 (Diamond).
- `rarity_id = 3` (Silver): T4 boost applied, rarity upgrades to 2 (Gold).
- `rarity_id = 4` (Bronze): T4 boost applied, rarity upgrades to 3 (Silver).
- `rarity_id = 5` (Common): T4 boost applied, rarity upgrades to 4 (Bronze).
- No card ever receives `rarity_id` outside the set {1, 2, 3, 4, 5, 99}.

**Risk If Failed:**

An invalid rarity ID (e.g., 0, 100, or None) propagates into the game engine and Discord bot
display layer. Cards with invalid rarities may render incorrectly, break sort/filter operations
in pack-opening UX, or cause exceptions in code paths that switch on rarity values.

**Files Involved:**

- `rarity_thresholds.py` — authoritative rarity ID definitions
- `docs/prd-evolution/05-rating-boosts.md` — section 5.4 (HoF cap behavior)
- Phase 2: `pd_cards/evo/tier_completion.py` — rarity ladder lookup, T4 completion handler
- Database: `evolution_card_state.final_rarity_id`

---

### T4-5: RP T1 achievability in realistic timeframe

**Status:** Pending — Phase 2

**Scenario:**

The Relief Pitcher track formula is `IP + K` with a T1 threshold of 3. The design intent is
"almost any active reliever hits this" in approximately 2 appearances (from `04-milestones.md`
section 4.2). The scenario to validate: a reliever who throws 1.2 IP (4 outs) with 1 K in an
appearance scores `1.33 + 1 = 2.33` — below T1. This reliever needs another appearance before
reaching T1.

The validation question is whether this is a blocking problem. If typical active RP usage
(5+ team game appearances) reliably produces T1 within a few sessions of play, the design is
sound. If a reliever can appear 4–5 times and still not reach T1 due to short, low-strikeout
outings (e.g., a pure groundball closer who throws 1.0 IP / 0 K per outing), the threshold
may be too high for the RP role to feel rewarding.

Reference calibration data from Season 10 (via `evo_milestone_simulator.py`): ~94% of all
relievers reached T1 under the IP+K formula with the threshold of 3. However, this is based on
a full or near-full season of data. The question is whether early-season RP usage (first 3–5
team games) produces T1 reliably.

Worked example for a pure-groundball closer:
- 5 appearances × (1.0 IP + 0 K) = 5.0 — reaches T1 (threshold 3) after appearance 3
- 5 appearances × (0.2 IP + 0 K) = 1.0 — does not reach T1 after 5 appearances

The second case (mop-up reliever with minimal usage) is expected to not reach T1 quickly, and
the design accepts this. What is NOT acceptable: a dedicated closer or setup man with 2+ IP per
session failing to reach T1 after 5+ appearances.

**Expected Outcome:**

A reliever averaging 1.0+ IP per appearance reaches T1 after 3 appearances. A reliever
averaging 0.5+ IP per appearance reaches T1 after 5–6 appearances. A reliever with fewer than
3 total appearances in a season is not expected to reach T1 — this is acceptable. The ~94%
Season 10 T1 rate confirms the threshold is calibrated correctly for active relievers.

**Risk If Failed:**

If active relievers (regular bullpen roles) cannot reach T1 within 5–10 team games, the
Refractor system is effectively dead for RP cards from launch. Players who pick up RP cards
expecting progression will see no reward for multiple play sessions, creating a negative first
impression of the entire system.

**Files Involved:**

- `docs/prd-evolution/04-milestones.md` — section 4.2 (RP track thresholds and design intent),
  section 4.3 (Season 10 calibration data)
- `scripts/evo_milestone_simulator.py` — `formula_rp_ip_k`, `simulate_tiers` — re-run against
  current season data to validate T1 achievability in early-season usage windows
- Database: `evolution_track` table — threshold values (admin-tunable, no code change required
  if recalibration is needed)

---

### T4-6: SP/RP T4 parity with batters

**Status:** Pending — Phase 2

**Scenario:**

The T4 thresholds are:

| Position | T4 Threshold | Formula |
|---|---|---|
| Batter | 896 | PA + (TB x 2) |
| Starting Pitcher | 240 | IP + K |
| Relief Pitcher | 70 | IP + K |

These were calibrated against Season 10 production data using `evo_milestone_simulator.py`.
The calibration target was approximately 3% of active players reaching T4 over a full season
across all position types. The validation here is that this parity holds: one position type
does not trivially farm Superfractors while another cannot reach T2 without extraordinary
performance.

The specific risk: SP T4 requires 240 IP+K across the full season. Top Season 10 SPs (Harang:
163, deGrom: 143) were on pace for T4 at the time of measurement but had not crossed 240 yet.
If the final-season data shows a spike (e.g., 10–15% of SPs reaching T4 vs. 3% of batters),
the SP threshold needs adjustment. Conversely, if no reliever reaches T4 in a full season
where 94% reach T1, the RP T4 threshold of 70 may be achievable only by top closers in
extreme usage scenarios.

Validation requires re-running `evo_milestone_simulator.py --season <current>` with the final
season data for all three position types and comparing T4 reach percentages. Accepted tolerance:
T4 reach rate within 2x across position types (e.g., if batters are at 3%, SP and RP should be
between 1.5% and 6%).

**Expected Outcome:**

All three position types produce T4 rates between 1% and 6% over a full season of active play.
No position type produces T4 rates above 10% (trivially farmable) or below 0.5% (effectively
unachievable). SP and RP T4 rates should be comparable because their thresholds were designed
together with the same 3% target in mind.

**Risk If Failed:**

If SP is easy (T4 in half a season) while RP is hard (T4 only for elite closers), then SP card
owners extract disproportionate value from the system. The Refractor system's balance premise
— "same tier, same reward, regardless of position" — breaks down, undermining player confidence
in the fairness of the progression.

**Files Involved:**

- `docs/prd-evolution/04-milestones.md` — section 4.3 (Season 10 calibration table)
- `scripts/evo_milestone_simulator.py` — primary validation tool; run with `--all-formulas
  --pitchers-only` and `--batters-only` flags against final season data
- Database: `evolution_track` table — thresholds are admin-tunable; recalibration does not
  require a code deployment

---

### T4-7: Cross-season stat accumulation — design confirmation

**Status:** Pending — Phase 2

**Scenario:**

The milestone evaluator (Phase 1, already implemented) queries `BattingSeasonStats` and
`PitchingSeasonStats` and SUMs the formula metric across all rows for a given
`(player_id, team_id)` pair, regardless of season number. This means a player's Refractor
progress is cumulative across seasons: if a player reaches 400 batter points in Season 10 and
another 400 in Season 11, their total is 800 — within range of T4 (threshold: 896).

This design must be confirmed as intentional before Phase 2 is implemented, because it has
significant downstream implications:

1. **Progress does not reset between seasons.** A player who earns a card across multiple
   seasons continues progressing the same Refractor state. Season boundaries are invisible to
   the evaluator.
2. **New teams start from zero.** If a player trades away a card and acquires a new copy of the
   same player, the new card's `evolution_card_state` row starts at T0. The stat accumulation
   query is scoped to `(player_id, team_id)`, so historical stats from the previous owner are
   not inherited.
3. **Live-series stat updates do not retroactively change progress.** The evaluator reads
   finalized season stat rows. If a player's Season 10 stats are adjusted via a data correction,
   the evaluator will pick up the change on the next evaluation run — progress could shift
   backward if a data correction removes a game's stats.
4. **The "full season" targets in the design docs (e.g., "T4 requires ~120 games") assume
   cumulative multi-season play, not a single season.** At ~7.5 batter points per game, T4 of
   896 requires approximately 120 in-game appearances. A player who plays 40 games per season
   across three seasons reaches T4 in their third season.

This is the confirmed intended design per `04-milestones.md`: "Cumulative within a season —
progress never resets mid-season." The document does not explicitly state "cumulative across
seasons," but the evaluator implementation (SUM across all rows, no season filter) makes this
behavior implicit. This test case exists to surface that ambiguity and require an explicit
design decision before Phase 2 ships.

**Expected Outcome:**

Before Phase 2 implementation begins, the design intent must be explicitly confirmed in writing
(update `04-milestones.md` section 4.1 with a cross-season statement) or the evaluator query
must be updated to add a season boundary. The options are:

- **Option A (current behavior — accumulate across seasons):** Document explicitly. The
  Refractor journey can span multiple seasons. Long-term card holders are rewarded for loyalty.
- **Option B (reset per season):** Add a season filter to the evaluator query. Refractor
  progress resets at season start. T4 is achievable within a single full season. Cards earned
  mid-season have a natural catch-up disadvantage.

This spec takes no position on which option is correct. It records that the choice exists,
that the current implementation defaults to Option A, and that Phase 2 must not be built on
an unexamined assumption about which option is in effect.

**Risk If Failed:**

If Option A is unintentional and players discover their Refractor progress carries over across
seasons before it is documented as a feature, they will optimize around it in ways the design
did not anticipate (e.g., holding cards across seasons purely to farm Refractor tiers). If
Option B is unintentional and progress resets each season without warning, players who invested
heavily in T3 at season end will be angry when their progress disappears.

**Files Involved:**

- `docs/prd-evolution/04-milestones.md` — section 4.1 (design principles) — **requires update
  to state the cross-season policy explicitly**
- Phase 1 (implemented): `pd_cards/evo/evaluator.py` — stat accumulation query; inspect the
  WHERE clause for any season filter
- Database: `BattingSeasonStats`, `PitchingSeasonStats` — confirm schema includes `season`
  column and whether the evaluator query filters on it
- Database: `evolution_card_state` — confirm there is no season-reset logic in the state
  management layer

---

## Summary Status

| ID | Title | Status |
|---|---|---|
| T4-1 | 108-sum preservation under profile-based boosts | Pending — Phase 2 |
| T4-2 | D20 probability shift at T4 | Pending — Phase 2 |
| T4-3 | T4 rarity upgrade — pipeline collision risk | Pending — Phase 2 |
| T4-4 | T4 rarity cap for HoF cards | Pending — Phase 2 |
| T4-5 | RP T1 achievability in realistic timeframe | Pending — Phase 2 |
| T4-6 | SP/RP T4 parity with batters | Pending — Phase 2 |
| T4-7 | Cross-season stat accumulation — design confirmation | Pending — Phase 2 |

All cases are unblocked pending Phase 2 implementation. T4-7 requires a design decision before
any Phase 2 code is written. T4-3 requires a resolution strategy to be selected before the T4
completion handler is implemented.