cal/major-domo-database
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity
79a559088a
major-domo-database/migration_issues_tracker.md
Cal Corum 79a559088a CLAUDE: Phase 1 PostgreSQL migration fixes complete 
- Fixed 4 critical schema issues blocking migration
- Resolved integer overflow by converting Discord IDs to strings
- Fixed VARCHAR length limits for Google Photos URLs
- Made injury_count field nullable for NULL values
- Successfully migrating 7/30 tables (5,432+ records)

Issues resolved:
- CONSTRAINT-CURRENT-INJURY_COUNT-001: Made nullable
- DATA_QUALITY-PLAYER-NAME-001: Increased VARCHAR limits to 1000
- MIGRATION_LOGIC-TEAM-INTEGER-001: Discord IDs now strings
- MIGRATION_LOGIC-DRAFTDATA-INTEGER-001: Channel IDs now strings

New issues discovered for Phase 2:
- CONSTRAINT-CURRENT-BSTATCOUNT-001: NULL stats count
- CONSTRAINT-TEAM-AUTO_DRAFT-001: NULL auto draft flag

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-18 18:09:45 -05:00

12 KiB
Raw Blame History

Migration Issues Tracker

Summary Dashboard

Last Updated: 2025-08-18 17:53:23
Test Run: #2 (Phase 1 Schema Fixes)
Total Issues: 27 (3 new discovered)
Resolved: 4
In Progress: 0
Remaining: 23

Status Overview

  • 🔴 Critical: 7 issues (3 new NULL constraints discovered)
  • 🟡 High: 12 issues (data integrity concerns)
  • 🟢 Medium: 4 issues (data quality issues)
  • ⚪ Low: 0 issues

Phase 1 Progress

  • ✅ 4 Critical Issues Resolved
  • ✅ 7 Tables Now Migrating Successfully: manager, division, sbaplayer, battingcareer, pitchingcareer, fieldingcareer, draftdata
  • ✅ Zero Integer Overflow Errors
  • ✅ Zero String Length Errors

✅ RESOLVED ISSUES (Phase 1)

CONSTRAINT-CURRENT-INJURY_COUNT-001 ✅

  • Resolution: Made injury_count field nullable (null=True)
  • Date Resolved: 2025-08-18
  • Solution Applied: Schema change in db_engine.py line 170
  • Test Result: ✅ NULL values now accepted

DATA_QUALITY-PLAYER-NAME-001 ✅

  • Resolution: Increased image and image2 field limits to max_length=1000
  • Date Resolved: 2025-08-18
  • Root Cause: Google Photos URLs up to 801 characters
  • Solution Applied: Schema change in db_engine.py lines 849-850
  • Test Result: ✅ Long URLs now accepted

MIGRATION_LOGIC-TEAM-INTEGER-001 ✅

  • Resolution: Converted Discord snowflake IDs to strings
  • Date Resolved: 2025-08-18
  • Root Cause: Discord IDs (1018721510111838309) exceed INTEGER range
  • Solution Applied: gmid/gmid2 now CharField(max_length=20)
  • Best Practice: Discord IDs should always be strings, not integers
  • Test Result: ✅ Large Discord IDs now handled properly

MIGRATION_LOGIC-DRAFTDATA-INTEGER-001 ✅

  • Resolution: Converted Discord channel IDs to strings
  • Date Resolved: 2025-08-18
  • Root Cause: Channel IDs exceed INTEGER range
  • Solution Applied: result_channel/ping_channel now CharField(max_length=20)
  • Test Result: ✅ draftdata table migrated successfully (1 record)

🔴 Critical Issues (Migration Blockers)

CONSTRAINT-CURRENT-BSTATCOUNT-001 🆕

  • Priority: CRITICAL
  • Table: current
  • Error: null value in column "bstatcount" violates not-null constraint
  • Impact: Blocks current table migration (11 records affected)
  • Status: IDENTIFIED
  • Solution: Make field nullable or set default value

CONSTRAINT-CURRENT-PSTATCOUNT-001 🆕

  • Priority: CRITICAL (Predicted)
  • Table: current
  • Error: null value in column "pstatcount" violates not-null constraint (likely)
  • Impact: Blocks current table migration (11 records affected)
  • Status: PREDICTED
  • Solution: Make field nullable or set default value

CONSTRAINT-TEAM-AUTO_DRAFT-001 🆕

  • Priority: CRITICAL
  • Table: team
  • Error: null value in column "auto_draft" violates not-null constraint
  • Impact: Blocks team table migration (546 records affected)
  • Status: IDENTIFIED
  • Solution: Make field nullable or set default value (False)

SCHEMA-CUSTOMCOMMANDCREATOR-MISSING-001

  • Priority: CRITICAL
  • Table: customcommandcreator
  • Error: no such table: customcommandcreator
  • Impact: Table doesn't exist in SQLite source
  • Status: IDENTIFIED
  • Solution: Skip table or create empty table

SCHEMA-CUSTOMCOMMAND-MISSING-001

  • Priority: CRITICAL
  • Table: customcommand
  • Error: no such table: customcommand
  • Impact: Table doesn't exist in SQLite source
  • Status: IDENTIFIED
  • Solution: Skip table or create empty table

CONSTRAINT-DECISION-TEAM_ID-001

  • Priority: CRITICAL
  • Table: decision
  • Error: null value in column "team_id" violates not-null constraint
  • Impact: Blocks decision table migration (20,309 records affected)
  • Status: IDENTIFIED
  • Solution: Handle NULL team_id values

FOREIGN_KEY-STRATPLAY-GAME_ID-001

  • Priority: CRITICAL
  • Table: stratplay
  • Error: violates foreign key constraint "stratplay_game_id_fkey"
  • Impact: Blocks stratplay table migration (192,790 records affected)
  • Status: IDENTIFIED
  • Solution: Migrate stratgame table first or fix referential integrity

🟡 High Priority Issues (Data Integrity)

FOREIGN_KEY-BATTINGSEASON-PLAYER_ID-001

  • Priority: HIGH
  • Table: battingseason
  • Error: violates foreign key constraint "battingseason_player_id_fkey"
  • Impact: References missing player records (4,878 records affected)
  • Status: IDENTIFIED
  • Solution: Migrate players first or clean orphaned records

FOREIGN_KEY-PITCHINGSEASON-PLAYER_ID-001

  • Priority: HIGH
  • Table: pitchingseason
  • Error: violates foreign key constraint "pitchingseason_player_id_fkey"
  • Impact: References missing player records (2,810 records affected)
  • Status: IDENTIFIED
  • Solution: Migrate players first or clean orphaned records

FOREIGN_KEY-FIELDINGSEASON-PLAYER_ID-001

  • Priority: HIGH
  • Table: fieldingseason
  • Error: violates foreign key constraint "fieldingseason_player_id_fkey"
  • Impact: References missing player records (8,981 records affected)
  • Status: IDENTIFIED
  • Solution: Migrate players first or clean orphaned records

FOREIGN_KEY-RESULT-TEAM_ID-001

  • Priority: HIGH
  • Table: result
  • Error: Foreign key constraint violations (estimated)
  • Impact: Result records may reference missing teams
  • Status: IDENTIFIED
  • Solution: Ensure teams migrate before results

FOREIGN_KEY-SCHEDULE-TEAM_ID-001

  • Priority: HIGH
  • Table: schedule
  • Error: Foreign key constraint violations (estimated)
  • Impact: Schedule records may reference missing teams
  • Status: IDENTIFIED
  • Solution: Ensure teams migrate before schedules

FOREIGN_KEY-TRANSACTION-PLAYER_ID-001

  • Priority: HIGH
  • Table: transaction
  • Error: Foreign key constraint violations (estimated)
  • Impact: Transaction records may reference missing players/teams
  • Status: IDENTIFIED
  • Solution: Fix dependency order

FOREIGN_KEY-BATTINGSTAT-PLAYER_ID-001

  • Priority: HIGH
  • Table: battingstat
  • Error: Foreign key constraint violations (estimated)
  • Impact: Stats records may reference missing players
  • Status: IDENTIFIED
  • Solution: Migrate players/teams first

FOREIGN_KEY-PITCHINGSTAT-PLAYER_ID-001

  • Priority: HIGH
  • Table: pitchingstat
  • Error: Foreign key constraint violations (estimated)
  • Impact: Stats records may reference missing players
  • Status: IDENTIFIED
  • Solution: Migrate players/teams first

FOREIGN_KEY-STANDINGS-TEAM_ID-001

  • Priority: HIGH
  • Table: standings
  • Error: Foreign key constraint violations (estimated)
  • Impact: Standings records may reference missing teams
  • Status: IDENTIFIED
  • Solution: Migrate teams first

FOREIGN_KEY-DRAFTPICK-TEAM_ID-001

  • Priority: HIGH
  • Table: draftpick
  • Error: Foreign key constraint violations (estimated)
  • Impact: Draft records may reference missing teams/players
  • Status: IDENTIFIED
  • Solution: Fix dependency order

FOREIGN_KEY-DRAFTLIST-TEAM_ID-001

  • Priority: HIGH
  • Table: draftlist
  • Error: Foreign key constraint violations (estimated)
  • Impact: Draft list records may reference missing teams/players
  • Status: IDENTIFIED
  • Solution: Fix dependency order

FOREIGN_KEY-AWARD-MANAGER_ID-001

  • Priority: HIGH
  • Table: award
  • Error: Foreign key constraint violations (estimated)
  • Impact: Award records may reference missing managers/players
  • Status: IDENTIFIED
  • Solution: Migrate managers/players first

🟢 Medium Priority Issues (Data Quality)

FOREIGN_KEY-DICEROLL-TEAM_ID-001

  • Priority: MEDIUM
  • Table: diceroll
  • Error: Foreign key constraint violations (estimated)
  • Impact: Dice roll records may reference missing teams
  • Status: IDENTIFIED
  • Solution: Migrate teams first or allow NULL teams

FOREIGN_KEY-KEEPER-TEAM_ID-001

  • Priority: MEDIUM
  • Table: keeper
  • Error: Foreign key constraint violations (estimated)
  • Impact: Keeper records may reference missing teams/players
  • Status: IDENTIFIED
  • Solution: Fix dependency order

FOREIGN_KEY-INJURY-PLAYER_ID-001

  • Priority: MEDIUM
  • Table: injury
  • Error: Foreign key constraint violations (estimated)
  • Impact: Injury records may reference missing players
  • Status: IDENTIFIED
  • Solution: Migrate players first

FOREIGN_KEY-STRATGAME-TEAM_ID-001

  • Priority: MEDIUM
  • Table: stratgame
  • Error: Foreign key constraint violations (estimated)
  • Impact: Game records may reference missing teams/managers
  • Status: IDENTIFIED
  • Solution: Migrate teams/managers first

📊 Resolution Strategy

Phase 1: Schema Fixes (Fix PostgreSQL Schema)

  • CONSTRAINT-CURRENT-INJURY_COUNT-001: Make injury_count nullable or set default
  • DATA_QUALITY-PLAYER-NAME-001: Increase VARCHAR limits
  • MIGRATION_LOGIC-TEAM-INTEGER-001: Use BIGINT for large integers
  • MIGRATION_LOGIC-DRAFTDATA-INTEGER-001: Use BIGINT for large integers

Phase 2: Missing Tables (Handle Non-existent Tables)

  • SCHEMA-CUSTOMCOMMANDCREATOR-MISSING-001: Skip gracefully
  • SCHEMA-CUSTOMCOMMAND-MISSING-001: Skip gracefully

Phase 3: Data Cleaning (Fix SQLite Data)

  • CONSTRAINT-DECISION-TEAM_ID-001: Handle NULL team_id values
  • Clean orphaned records before migration

Phase 4: Dependency Order (Fix Migration Logic)

  • Migrate base tables first: Current, Manager, Division, SbaPlayer
  • Then dependent tables: Team, Player
  • Finally stats and transaction tables
  • Disable foreign key checks during migration if needed

Phase 5: Validation

  • Run full migration test
  • Validate all record counts
  • Check referential integrity
  • Performance testing

📈 Progress Tracking

Test Run History

Run # Date Issues Found Issues Fixed Status Notes
1 2025-08-18 16:52 24 0 Discovery Complete Initial discovery run
2 2025-08-18 17:53 3 new 4 Phase 1 Complete Schema fixes successful
3 Planned Phase 2: NULL constraints

Test Run #2 Details (Phase 1)

Duration: ~3 minutes
Focus: Critical schema issues
Approach: Fixed 4 blocking schema problems

Issues Resolved:

  1. ✅ CONSTRAINT-CURRENT-INJURY_COUNT-001 → Made nullable
  2. ✅ DATA_QUALITY-PLAYER-NAME-001 → Increased VARCHAR limits
  3. ✅ MIGRATION_LOGIC-TEAM-INTEGER-001 → Discord IDs to strings
  4. ✅ MIGRATION_LOGIC-DRAFTDATA-INTEGER-001 → Channel IDs to strings

New Issues Found:

  1. 🆕 CONSTRAINT-CURRENT-BSTATCOUNT-001 → NULL stats count
  2. 🆕 CONSTRAINT-CURRENT-PSTATCOUNT-001 → NULL stats count (predicted)
  3. 🆕 CONSTRAINT-TEAM-AUTO_DRAFT-001 → NULL auto draft flag

Migration Results:

  • ✅ 7 tables migrated successfully (vs 0 in Run #1)
  • ✅ 5,432 records migrated (manager, division, sbaplayer, careers, draftdata)
  • ✅ No integer overflow errors
  • ✅ No string length errors
  • ⚠️ 3 new NULL constraint issues discovered

Next Actions (Phase 2)

  1. Immediate: Fix 3 new NULL constraint issues discovered in Phase 1
    • CONSTRAINT-CURRENT-BSTATCOUNT-001: Make bstatcount nullable
    • CONSTRAINT-CURRENT-PSTATCOUNT-001: Make pstatcount nullable
    • CONSTRAINT-TEAM-AUTO_DRAFT-001: Make auto_draft nullable or set default False
  2. Then: Handle missing tables gracefully (custom command tables)
  3. Next: Fix dependency order and foreign key issues
  4. Finally: Data cleaning and validation

Success Metrics (Current Status)

  • Tables Successfully Migrating: 7/30 (23%) ⬆️ from 0/30
  • Records Successfully Migrated: ~5,432 ⬆️ from 0
  • Critical Issues Resolved: 4/8 (50%) ⬆️ from 0/8
  • Schema Issues: ✅ Resolved (integers, string lengths)
  • NULL Constraints: ⚠️ 3 new issues discovered

This tracker will be updated after each test run to monitor progress toward successful migration.