Commit Graph

210 Commits

Author SHA1 Message Date
2b955dd8f7 Merge pull request 'fix: resolve unreachable duplicate elif 'DO*' branch in result_string() (#6)' (#39) from ai/paper-dynasty-card-creation#6 into main 2026-03-23 03:51:33 +00:00
cbfcba5e26 Merge branch 'main' into ai/paper-dynasty-card-creation#6 2026-03-23 03:50:40 +00:00
006b48e60f Merge pull request 'fix: use player_id instead of key_bbref in create_pit_position() (#7)' (#38) from ai/paper-dynasty-card-creation#7 into main 2026-03-23 03:50:38 +00:00
5e135ff554 Merge branch 'main' into ai/paper-dynasty-card-creation#7 2026-03-23 03:50:35 +00:00
602151fb16 Merge pull request 'Remove hardcoded secrets, load API token from env' (#29) from fix/2-3-security-hardcoded-secrets into main 2026-03-23 03:50:07 +00:00
6c20f93901 Merge branch 'main' into fix/2-3-security-hardcoded-secrets 2026-03-23 03:50:00 +00:00
Cal Corum
b52c5418db fix: resolve unreachable duplicate elif 'DO*' branch in result_string() (#6)
The second `elif "DO*" in data_string` was dead code — the first always
matched, so `spaces -= 2` for the DO** variant was silently skipped.
Fix: check "DO**" first (spaces -= 2), then "DO*" (spaces -= 1).

Closes #6

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 23:33:06 -05:00
Cal Corum
1d96223c78 fix: use player_id instead of key_bbref in create_pit_position() (#7)
Closes #7

The fallback branch of create_pit_position() used `int(df_data["key_bbref"])`
which always raises ValueError for string IDs like 'verlaju01'. The exception
was silently swallowed, causing pitchers without defensive stats to receive no
position record at all.

Fix: use `int(float(df_data["player_id"]))` to match the pattern used in
create_pitching_card() on the same file.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 23:03:43 -05:00
Cal Corum
e3220bf337 Remove hardcoded secrets, load API token from environment
- Replace hardcoded PD API bearer token in db_calls.py with dotenv/env var
- Delete scripts/supabase_doodling.py (dead scratch file with hardcoded Supabase JWT)
- Add python-dotenv dependency and .env.example template
- Consolidate check_prod_missing_ratings.py to import AUTH_TOKEN from db_calls
- Hard fail if PD_API_TOKEN is missing to prevent silent auth failures

Fixes #2, Fixes #3

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 12:38:01 -05:00
cal
f1ca14791d Merge pull request 'feat: render pipeline optimization (Phase 0)' (#28) from feature/render-pipeline-optimization into main
Reviewed-on: #28
2026-03-17 15:58:39 +00:00
Cal Corum
81622cceb3 docs: update Phase 0 status with PR references
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 10:13:18 -05:00
Cal Corum
8bddf31bf6 feat: configurable API URL for local high-concurrency card rendering
Allow upload scripts to target a local API server instead of the remote
production server, enabling 32x+ concurrency for dramatically faster
full-cardset uploads (~30-45s vs ~2-3min for 800 cards).

- pd_cards/core/upload.py: add api_url param to upload_cards_to_s3(),
  refresh_card_images(), and check_card_images()
- pd_cards/commands/upload.py: add --api-url CLI option to upload s3
- check_cards_and_upload.py: read PD_API_URL env var with prod fallback
- Update CLAUDE.md, CLI reference, and Phase 0 project plan docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-16 10:27:16 -05:00
cal
6e20a98226 Merge pull request 'feat: Phase 0 concurrent upload pipeline and benchmarks' (#27) from feature/render-pipeline-optimization into main
Reviewed-on: #27
2026-03-13 15:56:18 +00:00
Cal Corum
ed1daa20b0 fix: use get_running_loop() instead of deprecated get_event_loop()
get_event_loop() is deprecated in Python 3.10+ when called inside
a running coroutine. get_running_loop() is the correct replacement.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 08:50:51 -05:00
Cal Corum
979f3080d5 feat: concurrent upload pipeline and benchmarks (Phase 0)
- Replace sequential upload loop with asyncio.gather + Semaphore(8) (WP-04)
- Offload synchronous boto3 S3 calls to thread pool executor
- Increase fetch_card_image timeout from 6s to 10s
- Add --concurrency/-j CLI flag to pd-cards upload
- Add progress reporting every 20 completions
- Individual card failures no longer abort batch
- Apply same concurrency pattern to legacy check_cards_and_upload.py (WP-05)
- Add benchmark script for render pipeline measurements (WP-00)

Target: 800-card upload from ~40 min to <5 min (with server-side
persistent browser deployed).

Refs: #87, #91, #92

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 23:53:56 -05:00
cal
336014b689 Merge pull request 'Fix hold rating formula and format codebase' (#23) from fix/hold-rating-and-formatting into main
Reviewed-on: #23
2026-03-10 15:47:58 +00:00
Cal Corum
c2f7181f5b Remove unused defenders/creation.py stub
Single-line file with invalid syntax, not imported anywhere.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:24:33 -05:00
Cal Corum
0a17745389 Run black and ruff across entire codebase
Standardize formatting with black and apply ruff auto-fixes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:24:33 -05:00
Cal Corum
7fd691690c Fix pitcher hold rating: change pickoffs from override to bonus modifier
Pickoffs were using min(pick_cap, hold_num) which let high pickoff counts
completely override bad CS%, giving 31% of pitchers a -3 hold rating.
Now pickoffs act as a 1-3 point bonus on top of the CS%-based rating.
Pitchers with no CS data default to +2 (capped at -1 with pickoff bonus)
instead of the old +9.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:24:24 -05:00
cal
a4e56f6062 Merge pull request 'Fix SLG formula in extracted card rating models' (#22) from feature/fullcard-migration into main
Reviewed-on: #22
2026-03-03 21:59:06 +00:00
Cal Corum
931416a7c7 Add FullCard migration status doc for session continuity
Documents current branch state, what's built, what's left,
known bugs, and decision points so the next session can pick
up without re-investigating.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 23:58:55 -06:00
Cal Corum
db3822565c Add offense_col resolver for retrosheet pipeline to fix 883 silent KeyErrors
The FullCard migration requires offense_col and player_id on each player's
DataFrame row. The retrosheet pipeline calculates ratings before posting,
so both fields were missing — causing silent card layout builder failures.

Adds a three-tier resolution: CSV cache → API bulk fetch → deterministic
hash fallback. Also includes player_id fallback in both calcs modules.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 15:37:59 -06:00
Cal Corum
32cadb1c49 Fix two bugs in pitcher card builder dispatch logic
Fix duplicate elif branch in HR overflow cascade that prevented
single_one from receiving excess chances, and reorder single_two
secondary dispatch to check flyout full_name before groundout
short_name to prevent false 'B' matches on fly ball results.
Also add missing new_ratings.flyout_cf_b increment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-26 12:04:51 -06:00
Cal Corum
2bf3a6cee7 Fix SLG formula drift in extracted rating models
The extracted batting and pitching models used malformed SLG equations that double-counted and omitted outcomes, skewing slash lines. Align formulas with canonical weighting and add regression tests to prevent recurrence.

Co-Authored-By: Claude GPT-5.3-Codex <noreply@anthropic.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-26 07:47:15 -06:00
Cal Corum
39c652e55c Extract BattingCardRatingsModel and PitchingCardRatingsModel into models.py files
Move each ratings model class (and, for batters, the helper functions it
depends on) into a dedicated models.py so that calcs_*.py can import from
card_builder.py at module level without circular imports.

- batters/models.py: BattingCardRatingsModel + bp_singles, wh_singles,
  one_singles, bp_homeruns, triples, two_doubles, hit_by_pitch, strikeouts,
  flyout_a, flyout_bq, flyout_b, groundball_a, groundball_c
- pitchers/models.py: PitchingCardRatingsModel (no helper deps needed)
- batters/calcs_batter.py: imports model + build_batter_full_cards at top
- pitchers/calcs_pitcher.py: imports model + build_pitcher_full_cards at top
- batters/card_builder.py: imports from batters.models
- pitchers/card_builder.py: imports from pitchers.models
- tests/test_batter_calcs.py: import bp_singles, wh_singles from batters.models

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-25 16:42:51 -06:00
Cal Corum
a72abc01a3 Add FullCard/CardColumn/CardResult models and card builder pipeline
- card_layout.py: Port PlayResult, PLAY_RESULTS, EXACT_CHANCES, get_chances(),
  CardResult, CardColumn, FullCard, FullBattingCard, FullPitchingCard from
  database/app/card_creation.py. card_output() uses col_* key names.
  get_chances() always returns Decimal to avoid float/Decimal type errors.

- batters/card_builder.py: Port get_batter_card_data() algorithm as
  build_batter_full_cards(ratings_vl, ratings_vr, offense_col, player_id, hand).
  assign_bchances() returns float tuples for compatibility with float-based
  BattingCardRatingsModel fields.

- pitchers/card_builder.py: Port get_pitcher_card_data() algorithm as
  build_pitcher_full_cards(). assign_pchances() returns float tuples.
  Includes card.add_fatigue() at end of each card iteration.

- batters/calcs_batter.py: Integrate card builder in get_batter_ratings().
  After computing raw ratings, call build_batter_full_cards() and merge
  9 col_* rendered column fields into each ratings dict. Lazy import to
  avoid circular dependency.

- pitchers/calcs_pitcher.py: Same integration for get_pitcher_ratings().

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-02-25 16:21:26 -06:00
cal
7c967b6119 Merge pull request 'Add card generation pipeline agents' (#1) from add-card-pipeline-agents into main
Reviewed-on: #1
2026-02-16 20:18:37 +00:00
Cal Corum
fe91de905a Add card generation pipeline agents and refresh scouting data
- Add retrosheet-card-update agent (8-step pipeline with validation gates)
- Add live-series-card-update agent (7-step pipeline with PotM support)
- Both agents: dev/prod S3 guard, environment verification, groundball_b validation
- Restore db_calls.py to production (alt_database = None)
- Refresh scouting reports (6303 batters, 7164 pitchers)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-16 14:16:50 -06:00
Cal Corum
923edd0eeb Update 2005 Live cardset through mid-August (73% season)
790 players (397 batters, 393 pitchers) processed from Retrosheet data
through 2005-08-15 with 0.728 season percentage. Includes updated scouting
reports, card deltas, and FanGraphs scrape script.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:56:19 -06:00
Cal Corum
9e8dc4ef96 Rename Sippie Swartzel to Sippie Swartzel Sr 2026-01-25 22:50:34 -06:00
Cal Corum
1de8b1db2f Add custom card profiles, S3 upload with timestamp cache-busting, and CLI enhancements
- Add Sippie Swartzel custom batter profile (0.820 OPS, SS/RF, no HR power)
- Update Kalin Young profile (0.891 OPS, All-Star rarity)
- Update Admiral Ball Traits profile with innings field
- Fix S3 cache-busting to include Unix timestamp for same-day updates
- Add pd_cards/core/upload.py and scouting.py modules
- Add custom card submission scripts and documentation
- Add uv.lock for dependency tracking
2026-01-25 21:57:35 -06:00
Cal Corum
9a121d370f Add card builder architecture redesign documentation
Documents proposed architecture for moving card-building logic upstream
to Python, including:
- Executive summary with problem statement and migration path
- CardBuilder sketch with contract system for pluggable placement strategies
- Support for different card "personalities" (Standard, Clutch, Power Heavy, etc.)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-22 10:56:55 -06:00
Cal Corum
a2f4d02b18 Add scouting upload CLI command
- Add `pd-cards scouting upload` command to upload scouting CSVs to database server via SCP
- Update CLAUDE.md with critical warning: scouting must always run for ALL cardsets
- Document full workflow: `pd-cards scouting all && pd-cards scouting upload`

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:17:14 -06:00
Cal Corum
88acd5308b Update Will The Thrill profile to 0.883 combined OPS
- Sync ratings with production database values
- Increase target OPS from 0.825 to 0.880 (actual: 0.883)
- Update baserunning: steal_high 14, steal_jump 0.33, running 13
- Adjust defensive profile: LF error 5, arm 0

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 19:22:40 -06:00
Cal Corum
48ec7375a1 Normalize franchise values in card generation
- Add FRANCHISE_NORMALIZE dict and helper function
- Update FRANCHISE_LIST to return city-agnostic values
- Update mlbteam_and_franchise() to normalize franchise

Ensures new cards use normalized franchise format for AI roster matching

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 12:01:12 -06:00
Cal Corum
78eb8862e1 Update scouting reports with latest card data
Regenerated scouting CSVs for all cardsets (6211 batters, 7070 pitchers)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-23 09:17:29 -06:00
Cal Corum
fe0ab5e1bd Add --create flag to custom submit command
- Add --create/-c flag to create new players directly from YAML profiles
- Skip MLBPlayer creation (not needed for custom players)
- Auto-populate required API fields (cost, rarity, mlbclub, etc.)
- Update YAML profile with player_id and card_id after creation
- Add Adm Ball Traits custom player profile

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 14:23:20 -06:00
Cal Corum
0fe549449d Add recency bias CLI flags to retrosheet command
- Add --last-week-ratio, --last-twoweeks-ratio, --last-month-ratio flags
- Auto-enable 0.2 recency bias for last 2 weeks on Live series after May 30
- Fix main() call to pass empty args list (legacy parameter required)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-21 16:38:12 -06:00
Cal Corum
5b75a3d38f Implement CLI wrappers for live-series, retrosheet, scouting, upload
Migrated all major card creation workflows to pd-cards CLI:

live-series:
- update: Full FanGraphs/BBRef card generation with CLI options
- status: Show cardset status from database

retrosheet:
- process: Historical Retrosheet data processing
- arms: Generate outfield arm ratings from play-by-play
- validate: Check for position anomalies in cardsets
- defense: Fetch defensive stats from Baseball Reference

scouting:
- batters: Generate batting scouting reports
- pitchers: Generate pitching scouting reports
- all: Generate all reports at once

upload:
- s3: Upload card images to AWS S3
- check: Validate cards without uploading
- refresh: Re-generate and re-upload card images

Updated CLAUDE.md with comprehensive CLI documentation.
Legacy scripts remain available but CLI is now the primary interface.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-18 16:39:38 -06:00
Cal Corum
699d26468f Add YAML profiles for remaining custom players
Created profiles for 4 custom batters:
- Sphealthamus Spheal: L/1B, 0.850 target OPS, extreme platoon splits
- Luan Arroto: L/1B+RF, 0.850 target OPS, player_id=13156
- Will The Thrill: R/LF+2B, 0.825 target OPS, contact hitter
- Valerie Theolia: L/C, 0.855 target OPS, catcher with A bunting

All profiles have:
- Valid structure (positions, baserunning, ratings)
- Ratings summing to exactly 108.0 per split
- Detailed comments explaining player characteristics

Note: OPS values need calibration using archetype calculator
or manual tuning to match exact targets.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-18 16:26:15 -06:00
Cal Corum
8ac89cfcd8 Add pitcher support to custom cards CLI
- Add pitcher detection via player_type field or ratings schema
- Separate calc_ops and verify_total for batters vs pitchers
- Pitcher template with correct schema (double_cf, xcheck fields, etc.)
- Combined OPS formula: max() for pitchers, min() for batters
- Add --type option to 'pd-cards custom new' command
- Migrate Tony Smehrik to YAML pitcher profile

Pitcher schema differences from batters:
- double_cf instead of double_pull
- flyout_cf_b instead of flyout_a/flyout_bq
- No groundout_c
- xcheck_* fields (29 chances for fielder plays)
- pitching block for starter/relief/closer ratings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-18 16:17:17 -06:00
Cal Corum
2e28d29ced Add pd-cards CLI skeleton with Typer
Introduces new pd-cards CLI tool for all card creation workflows:
- custom: manage fictional character cards via YAML profiles
- live-series: live season card updates (stub)
- retrosheet: historical data processing (stub)
- scouting: scouting report generation (stub)
- upload: S3 card image upload (stub)

Key features:
- Typer-based CLI with auto-generated help and shell completion
- YAML profiles for custom characters (replaces per-character Python scripts)
- Preview, submit, new, and list commands for custom cards
- First character migrated: Kalin Young

Install with: uv pip install -e .
Run with: pd-cards --help

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-18 16:08:32 -06:00
Cal Corum
1c39f7f8b3 Update scouting reports and reset retrosheet config to Live defaults
- Regenerated scouting CSVs with May PotM players (13287-13294)
- Reset retrosheet_data.py from May PotM back to Live series config

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-14 10:09:09 -06:00
Cal Corum
b10cfcfd09 Fix check_cards_and_upload.py to regenerate cards from PD API
- Changed card URL generation to fetch from PD API endpoint
  (/v2/players/{id}/battingcard) instead of existing S3 URL
- This ensures database changes (like cardpositions) are reflected
  in regenerated card images
- Added fix_cardpositions.py utility for regenerating batter positions
  without re-running full retrosheet_data.py script

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-07 17:05:47 -06:00
Cal Corum
9d9c507e84 Update Sphealy custom card to 0.850 OPS and cost 188
Increased target OPS from 0.820 to 0.850 with adjusted stat splits:
- vs RHP: .260/.340/.495 (power profile)
- vs LHP: .260/.375/.420 (patient/OBP profile)
- Cost updated from 85 to 188

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 10:53:27 -06:00
Cal Corum
c2ec561c82 Add Luan Arroto custom card scripts
- luan_arroto_final.py: Card definition with 0.850 OPS target
- submit_luan_arroto.py: Initial card creation script
- update_luan_arroto.py: Update existing card ratings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 16:26:07 -06:00
Cal Corum
92256cb29c Update scouting data and card creation scripts
- Regenerate scouting CSVs with latest player ratings
- Update archetype calculator with BP-HR whole number rule
- Refresh retrosheet normalized data
- Minor script updates for Kalin Young card creation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 16:25:42 -06:00
Cal Corum
cc5f93eb66 Fix critical asterisk regression in player names
CRITICAL BUG FIX: Removed code that was appending asterisks to left-handed
players' names and hash symbols to switch hitters' names in production.

## Changes

### Core Fix (retrosheet_data.py)
- Removed name_suffix code from new_player_payload() (lines 1103-1108)
- Players names now stored cleanly without visual indicators
- Affected 20 left-handed batters in 2005 Live cardset

### New Utility Scripts
- fix_player_names.py: PATCH player names to remove symbols (uses 'name' param)
- check_player_names.py: Verify all players for asterisks/hashes
- regenerate_lefty_cards.py: Update image URLs with cache-busting dates
- upload_lefty_cards_to_s3.py: Fetch fresh cards and upload to S3

### Documentation (CRITICAL - READ BEFORE WORKING WITH CARDS)
- docs/LESSONS_LEARNED_ASTERISK_REGRESSION.md: Comprehensive guide
  * API parameter is 'name' NOT 'p_name'
  * Card generation caching requires timestamp cache-busting
  * S3 keys must not include query parameters
  * Player names only in 'players' table
  * Never append visual indicators to stored data

- CLAUDE.md: Added critical warnings section at top

## Key Learnings
1. API param for player name is 'name', not 'p_name'
2. Cards are cached - use timestamp in ?d= parameter
3. S3 keys != S3 URLs (no query params in keys)
4. Fix data BEFORE generating/uploading cards
5. Visual indicators belong in UI, not database

## Impact
- Fixed 20 player records in production
- Regenerated and uploaded 20 clean cards to S3
- Documented to prevent future regressions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-24 14:38:04 -06:00
Cal Corum
4be418d6f0 Card Creation Automation 2025-11-23 01:28:33 -06:00
Cal Corum
de253eb561 Add batch migration script for uploading all card images to AWS S3
Creates migrate_all_cards_to_s3.py to migrate historical card images from
Paper Dynasty API to S3 bucket. Key features:

- Processes all cardsets automatically (12,966 player cards across 29 cardsets)
- Detects and skips URLs already pointing to AWS S3
- Dry-run mode for previewing changes before execution
- Flexible filtering by cardset ID ranges or exclusion lists
- Per-cardset and global statistics tracking
- Updates player records with new S3 URLs after upload

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-16 08:42:40 -06:00