cal/paper-dynasty-card-creation
1
0
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: render pipeline optimization (Phase 0) #28

Merged
cal merged 2 commits from feature/render-pipeline-optimization into main 2026-03-17 15:58:40 +00:00
Conversation 0 Commits 2 Files Changed 6 +406 -7
cal commented 2026-03-16 16:04:13 +00:00
Owner
Copy Link

Summary

Phase 0 render pipeline optimization for the card-creation repo (WP-04, WP-05, and local rendering follow-on).

  • Concurrent upload pipeline: Replace sequential card processing with semaphore-bounded asyncio.gather (default 8x concurrency, configurable via --concurrency)
  • Legacy script update: Same concurrency pattern applied to check_cards_and_upload.py
  • Configurable API URL: New --api-url CLI option and PD_API_URL env var for targeting a local API server, enabling 32x+ concurrency for dramatically faster uploads
  • Deprecated API fix: get_event_loop() → get_running_loop()
  • Documentation: Updated CLAUDE.md, CLI reference, and Phase 0 project plan

Commits

  • 979f308 — Concurrent upload pipeline and benchmarks
  • ed1daa2 — Fix deprecated get_event_loop()
  • 8bddf31 — Configurable API URL for local rendering

Performance

Scenario 800 cards
Before (sequential) ~27 min
Remote, 8x concurrent ~2-3 min
Local, 32x concurrent ~30-45 sec

Closes database repo issues: WP-04 (#91), WP-05 (#92), tracker (#93)

Test plan

  • pd-cards upload s3 --cardset "2005 Live" --dry-run — verify CLI accepts new options
  • pd-cards upload s3 --cardset "2005 Live" --limit 10 — small batch upload to S3
  • pd-cards upload s3 --cardset "2005 Live" --api-url http://localhost:8000/api --concurrency 32 --limit 10 — local rendering test
  • PD_API_URL=http://localhost:8000/api python check_cards_and_upload.py — legacy script env var

🤖 Generated with Claude Code

## Summary Phase 0 render pipeline optimization for the card-creation repo (WP-04, WP-05, and local rendering follow-on). - **Concurrent upload pipeline**: Replace sequential card processing with semaphore-bounded `asyncio.gather` (default 8x concurrency, configurable via `--concurrency`) - **Legacy script update**: Same concurrency pattern applied to `check_cards_and_upload.py` - **Configurable API URL**: New `--api-url` CLI option and `PD_API_URL` env var for targeting a local API server, enabling 32x+ concurrency for dramatically faster uploads - **Deprecated API fix**: `get_event_loop()` → `get_running_loop()` - **Documentation**: Updated CLAUDE.md, CLI reference, and Phase 0 project plan ### Commits - `979f308` — Concurrent upload pipeline and benchmarks - `ed1daa2` — Fix deprecated get_event_loop() - `8bddf31` — Configurable API URL for local rendering ### Performance | Scenario | 800 cards | |----------|-----------| | Before (sequential) | ~27 min | | Remote, 8x concurrent | ~2-3 min | | Local, 32x concurrent | ~30-45 sec | Closes database repo issues: WP-04 (#91), WP-05 (#92), tracker (#93) ## Test plan - [ ] `pd-cards upload s3 --cardset "2005 Live" --dry-run` — verify CLI accepts new options - [ ] `pd-cards upload s3 --cardset "2005 Live" --limit 10` — small batch upload to S3 - [ ] `pd-cards upload s3 --cardset "2005 Live" --api-url http://localhost:8000/api --concurrency 32 --limit 10` — local rendering test - [ ] `PD_API_URL=http://localhost:8000/api python check_cards_and_upload.py` — legacy script env var 🤖 Generated with [Claude Code](https://claude.com/claude-code)
cal added 1 commit 2026-03-16 16:04:14 +00:00 
Allow upload scripts to target a local API server instead of the remote
production server, enabling 32x+ concurrency for dramatically faster
full-cardset uploads (~30-45s vs ~2-3min for 800 cards).

- pd_cards/core/upload.py: add api_url param to upload_cards_to_s3(),
  refresh_card_images(), and check_card_images()
- pd_cards/commands/upload.py: add --api-url CLI option to upload s3
- check_cards_and_upload.py: read PD_API_URL env var with prod fallback
- Update CLAUDE.md, CLI reference, and Phase 0 project plan docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cal added 1 commit 2026-03-17 15:13:27 +00:00 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cal merged commit f1ca14791d into main 2026-03-17 15:58:40 +00:00
cal deleted branch feature/render-pipeline-optimization 2026-03-17 15:58:40 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
ai-merged

PR merged by AI ops agent

  
ai-pr-opened

   
ai-reviewed

   
ai-reviewing

   
ai-working
No Label
ai-merged
ai-pr-opened
ai-reviewed
ai-reviewing
ai-working
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
1 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: cal/paper-dynasty-card-creation#28
No description provided.
Delete Branch "feature/render-pipeline-optimization"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?