Commit Graph

16 Commits

Author SHA1 Message Date
6c20f93901 Merge branch 'main' into fix/2-3-security-hardcoded-secrets 2026-03-23 03:50:00 +00:00
Cal Corum
e3220bf337 Remove hardcoded secrets, load API token from environment
- Replace hardcoded PD API bearer token in db_calls.py with dotenv/env var
- Delete scripts/supabase_doodling.py (dead scratch file with hardcoded Supabase JWT)
- Add python-dotenv dependency and .env.example template
- Consolidate check_prod_missing_ratings.py to import AUTH_TOKEN from db_calls
- Hard fail if PD_API_TOKEN is missing to prevent silent auth failures

Fixes #2, Fixes #3

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 12:38:01 -05:00
Cal Corum
979f3080d5 feat: concurrent upload pipeline and benchmarks (Phase 0)
- Replace sequential upload loop with asyncio.gather + Semaphore(8) (WP-04)
- Offload synchronous boto3 S3 calls to thread pool executor
- Increase fetch_card_image timeout from 6s to 10s
- Add --concurrency/-j CLI flag to pd-cards upload
- Add progress reporting every 20 completions
- Individual card failures no longer abort batch
- Apply same concurrency pattern to legacy check_cards_and_upload.py (WP-05)
- Add benchmark script for render pipeline measurements (WP-00)

Target: 800-card upload from ~40 min to <5 min (with server-side
persistent browser deployed).

Refs: #87, #91, #92

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 23:53:56 -05:00
Cal Corum
0a17745389 Run black and ruff across entire codebase
Standardize formatting with black and apply ruff auto-fixes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-08 14:24:33 -05:00
Cal Corum
923edd0eeb Update 2005 Live cardset through mid-August (73% season)
790 players (397 batters, 393 pitchers) processed from Retrosheet data
through 2005-08-15 with 0.728 season percentage. Includes updated scouting
reports, card deltas, and FanGraphs scrape script.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-15 18:56:19 -06:00
Cal Corum
2081a8f0ac Custom card creation work 2025-11-14 09:51:04 -06:00
Cal Corum
5d7a0dd74b CLAUDE: Fix outfield position assignment bug and add validation script
Fixed critical bug where all outfielders were incorrectly assigned as DH
due to defense CSV column mismatch in retrosheet_data.py:

- Lines 889, 926: Changed column check from 'in row' to 'in pos_df.columns'
  to correctly detect bis_runs_total availability
- Line 947: Fixed fallback from non-existent 'tz_runs_outfield' to
  'tz_runs_total' which actually exists in Baseball Reference CSVs

Impact:
- Before: 57 DH players, 0 outfield positions
- After: 3 DH players, 62 outfielders (23 RF, 20 CF, 19 LF)

Added scripts/check_positions.sh:
- Validates position distribution after card generation
- Flags anomalous DH counts (>5 or >10%)
- Verifies outfield positions exist in cardpositions table
- Provides quick smoke test for defensive calculations

Updated CLAUDE.md:
- Added Position Validation section with check_positions.sh usage
- Documented outfield position bug in Common Issues & Solutions
- Included code examples and verification steps

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-09 11:38:36 -06:00
Cal Corum
6de22f7f1f Code stabilization & retrosheet functionality 2025-11-08 16:57:35 -06:00
Cal Corum
f9411c5e55 CLAUDE: Add utility scripts for defense CSV column renaming
Added one-time utility scripts used to prepare 2005 defense CSV files
for compatibility with retrosheet_data.py.

Scripts:
- rename_defense_columns.py: Renamed initial batch of defense columns
  - RF/9 → range_factor_per_nine
  - RF/G → range_factor_per_game
  - DP → DP_def, E → E_def, Ch → chances, Inn → Inn_def
  - CS% → caught_stealing_perc, PO → pickoffs
  - Name-additional → key_bbref

- rename_additional_defense_columns.py: Second batch of column renames
  - Fld% → fielding_perc
  - Rtot → tz_runs_total, Rtot/yr → tz_runs_total_per_season
  - Rtz → tz_runs_field, Rdp → tz_runs_infield

- undo_po_rename.py: Reverted PO → pickoffs for position players
  - Kept 'pickoffs' for defense_p.csv (pitchers)
  - Changed back to 'PO' for all other positions (c, 1b, 2b, etc.)

- test_retrosheet_integration.py: Integration test for retrosheet_transformer
  - Validates batting and pitching stats loading
  - Tests date range filtering
  - Verifies player counts

These scripts have already been executed and the defense files are
properly formatted. Kept for historical reference and documentation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-08 16:15:37 -06:00
Cal Corum
c89e1eb507 Claude introduction & Live Series Update 2025-07-22 09:24:34 -05:00
Cal Corum
71b2fdbeba Refactor refresh_cards 2025-02-09 01:17:58 -06:00
Cal Corum
25d4d9a63c Migrate to rotating file logger 2024-11-10 14:42:12 -06:00
Cal Corum
ac544965ae Migrated args to constants 2024-11-02 22:50:54 -05:00
Cal Corum
863d906657 Script to change pos_1=P to proper SP/RP/CP 2024-11-02 22:50:39 -05:00
Cal Corum
e396b50230 Pitching defense done
Pitching cards done
2024-10-27 00:42:51 -05:00
Cal Corum
639e032586 Moving older scripts into holding cell 2024-10-17 09:28:02 -05:00