paper-dynasty-card-creation/scripts
Cal Corum 979f3080d5 feat: concurrent upload pipeline and benchmarks (Phase 0)
- Replace sequential upload loop with asyncio.gather + Semaphore(8) (WP-04)
- Offload synchronous boto3 S3 calls to thread pool executor
- Increase fetch_card_image timeout from 6s to 10s
- Add --concurrency/-j CLI flag to pd-cards upload
- Add progress reporting every 20 completions
- Individual card failures no longer abort batch
- Apply same concurrency pattern to legacy check_cards_and_upload.py (WP-05)
- Add benchmark script for render pipeline measurements (WP-00)

Target: 800-card upload from ~40 min to <5 min (with server-side
persistent browser deployed).

Refs: #87, #91, #92

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 23:53:56 -05:00
..
legacy Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
add_second_cards.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
Batting_vLHP_BattedBalls.csv Update 2005 Live cardset through mid-August (73% season) 2026-02-15 18:56:19 -06:00
Batting_vLHP_Standard.csv Update 2005 Live cardset through mid-August (73% season) 2026-02-15 18:56:19 -06:00
Batting_vRHP_BattedBalls.csv Update 2005 Live cardset through mid-August (73% season) 2026-02-15 18:56:19 -06:00
Batting_vRHP_Standard.csv Update 2005 Live cardset through mid-August (73% season) 2026-02-15 18:56:19 -06:00
benchmark_render.sh feat: concurrent upload pipeline and benchmarks (Phase 0) 2026-03-12 23:53:56 -05:00
benchmark_results.txt feat: concurrent upload pipeline and benchmarks (Phase 0) 2026-03-12 23:53:56 -05:00
check_positions.sh CLAUDE: Fix outfield position assignment bug and add validation script 2025-11-09 11:38:36 -06:00
custom_card_creation.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
fangraphs_scrape.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
fix_pitcher_pos.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
Pitching_vLHH_BattedBalls.csv Update 2005 Live cardset through mid-August (73% season) 2026-02-15 18:56:19 -06:00
Pitching_vLHH_Standard.csv Update 2005 Live cardset through mid-August (73% season) 2026-02-15 18:56:19 -06:00
Pitching_vRHH_BattedBalls.csv Update 2005 Live cardset through mid-August (73% season) 2026-02-15 18:56:19 -06:00
Pitching_vRHH_Standard.csv Update 2005 Live cardset through mid-August (73% season) 2026-02-15 18:56:19 -06:00
position_updates.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
raw play tracking.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
README.md Code stabilization & retrosheet functionality 2025-11-08 16:57:35 -06:00
rename_additional_defense_columns.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
rename_defense_columns.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
scouting_updates.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
supabase_doodling.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
test_retrosheet_integration.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
undo_po_rename.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
update_scouting.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00
upload_scouting.py Run black and ruff across entire codebase 2026-03-08 14:24:33 -05:00

Scripts Directory

This directory contains utility scripts for Paper Dynasty card management and data operations.

Scripts

upload_scouting.py

Generates and uploads scouting reports to the remote server.

What it does:

  1. Runs scouting_batters.py to generate batting scouting reports
  2. Runs scouting_pitchers.py to generate pitching scouting reports
  3. Uploads all 4 CSV files (batting-basic, batting-ratings, pitching-basic, pitching-ratings) to sba-db server via SCP

How to run:

cd scripts
python upload_scouting.py

add_second_cards.py

Adds secondary card images (image2) to player records for players who have both batting and pitching cards.

What it does:

  • Pulls all batting and pitching cards from the database
  • For players missing their alternate card type in image2, adds the appropriate card URL
  • Updates player records with the secondary card image

How to run:

python add_second_cards.py

custom_card_creation.py

Creates custom cardsets with specific configuration options.

What it does:

  • Generates cards for a specified custom cardset
  • Allows control over batting/pitching card generation
  • Handles player position updates based on card data
  • Uses command-line arguments for configuration

Arguments:

  • cardset_name: Name of the cardset to create
  • season: Season year (optional, defaults to first 4 chars of cardset name)
  • player_description: Description shown on cards (optional)
  • post_batters: Whether to post batting cards (default: true)
  • post_pitchers: Whether to post pitching cards (default: true)
  • post_players: Whether to post player updates (default: true)

How to run:

python custom_card_creation.py cardset_name="2024 Custom" season=2024

fix_pitcher_pos.py

Updates pitcher positions (SP/RP/CP) based on their pitching card ratings.

What it does:

  • Queries all players with position P
  • Checks their starter_rating and closer_rating from pitching cards
  • Updates positions: SR >= 4 → SP, otherwise → RP, CR present → CP

How to run:

python fix_pitcher_pos.py

position_updates.py

Comprehensive player position updates for a specified cardset.

What it does:

  • Updates all player positions based on their cards in a specific cardset
  • Handles pitchers (SP/RP/CP logic based on ratings)
  • Handles fielders (preserves existing positions)
  • Only updates positions that have changed

Arguments:

  • cardset_name: Name of the cardset to update
  • post_players: Whether to actually post updates (default: false)

How to run:

python position_updates.py cardset_name="2025 Live" post_players=true

scouting_updates.py

Triggers scouting report calculations on the Paper Dynasty API.

What it does:

  • Runs 4 sequential API calls to calculate card ratings:
    1. Basic batting ratings
    2. Full batting scouting ratings
    3. Basic pitching ratings
    4. Full pitching scouting ratings
  • Times each operation and reports total runtime

How to run:

python scouting_updates.py

supabase_doodling.py

Development/testing script for Supabase operations (not for production use).


raw play tracking.py

Legacy script (purpose unclear from filename - check contents before use).


Legacy Scripts

The legacy/ subdirectory contains older scripts that are no longer actively used but kept for reference.

Notes

  • Most scripts require authentication credentials configured in db_calls.py
  • Scripts typically use async/await patterns and should be run with asyncio.run()
  • Check individual script headers for additional documentation
  • Scripts in this directory should be run from the scripts directory or with appropriate path handling