feat: Phase 0 concurrent upload pipeline and benchmarks #27

Merged
cal merged 2 commits from feature/render-pipeline-optimization into main 2026-03-13 15:56:19 +00:00

2 Commits

Author SHA1 Message Date
Cal Corum
ed1daa20b0 fix: use get_running_loop() instead of deprecated get_event_loop()
get_event_loop() is deprecated in Python 3.10+ when called inside
a running coroutine. get_running_loop() is the correct replacement.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 08:50:51 -05:00
Cal Corum
979f3080d5 feat: concurrent upload pipeline and benchmarks (Phase 0)
- Replace sequential upload loop with asyncio.gather + Semaphore(8) (WP-04)
- Offload synchronous boto3 S3 calls to thread pool executor
- Increase fetch_card_image timeout from 6s to 10s
- Add --concurrency/-j CLI flag to pd-cards upload
- Add progress reporting every 20 completions
- Individual card failures no longer abort batch
- Apply same concurrency pattern to legacy check_cards_and_upload.py (WP-05)
- Add benchmark script for render pipeline measurements (WP-00)

Target: 800-card upload from ~40 min to <5 min (with server-side
persistent browser deployed).

Refs: #87, #91, #92

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 23:53:56 -05:00