79a559088a
- Fixed 4 critical schema issues blocking migration - Resolved integer overflow by converting Discord IDs to strings - Fixed VARCHAR length limits for Google Photos URLs - Made injury_count field nullable for NULL values - Successfully migrating 7/30 tables (5,432+ records) Issues resolved: - CONSTRAINT-CURRENT-INJURY_COUNT-001: Made nullable - DATA_QUALITY-PLAYER-NAME-001: Increased VARCHAR limits to 1000 - MIGRATION_LOGIC-TEAM-INTEGER-001: Discord IDs now strings - MIGRATION_LOGIC-DRAFTDATA-INTEGER-001: Channel IDs now strings New issues discovered for Phase 2: - CONSTRAINT-CURRENT-BSTATCOUNT-001: NULL stats count - CONSTRAINT-TEAM-AUTO_DRAFT-001: NULL auto draft flag 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
334 lines
12 KiB
Markdown
334 lines
12 KiB
Markdown
# Migration Issues Tracker
|
|
|
|
## Summary Dashboard
|
|
|
|
**Last Updated**: 2025-08-18 17:53:23
|
|
**Test Run**: #2 (Phase 1 Schema Fixes)
|
|
**Total Issues**: 27 (3 new discovered)
|
|
**Resolved**: 4
|
|
**In Progress**: 0
|
|
**Remaining**: 23
|
|
|
|
### Status Overview
|
|
- 🔴 **Critical**: 7 issues (3 new NULL constraints discovered)
|
|
- 🟡 **High**: 12 issues (data integrity concerns)
|
|
- 🟢 **Medium**: 4 issues (data quality issues)
|
|
- ⚪ **Low**: 0 issues
|
|
|
|
### Phase 1 Progress
|
|
- ✅ **4 Critical Issues Resolved**
|
|
- ✅ **7 Tables Now Migrating Successfully**: manager, division, sbaplayer, battingcareer, pitchingcareer, fieldingcareer, draftdata
|
|
- ✅ **Zero Integer Overflow Errors**
|
|
- ✅ **Zero String Length Errors**
|
|
|
|
---
|
|
|
|
## ✅ **RESOLVED ISSUES** (Phase 1)
|
|
|
|
### CONSTRAINT-CURRENT-INJURY_COUNT-001 ✅
|
|
- **Resolution**: Made `injury_count` field nullable (`null=True`)
|
|
- **Date Resolved**: 2025-08-18
|
|
- **Solution Applied**: Schema change in db_engine.py line 170
|
|
- **Test Result**: ✅ NULL values now accepted
|
|
|
|
### DATA_QUALITY-PLAYER-NAME-001 ✅
|
|
- **Resolution**: Increased `image` and `image2` field limits to `max_length=1000`
|
|
- **Date Resolved**: 2025-08-18
|
|
- **Root Cause**: Google Photos URLs up to 801 characters
|
|
- **Solution Applied**: Schema change in db_engine.py lines 849-850
|
|
- **Test Result**: ✅ Long URLs now accepted
|
|
|
|
### MIGRATION_LOGIC-TEAM-INTEGER-001 ✅
|
|
- **Resolution**: Converted Discord snowflake IDs to strings
|
|
- **Date Resolved**: 2025-08-18
|
|
- **Root Cause**: Discord IDs (1018721510111838309) exceed INTEGER range
|
|
- **Solution Applied**: `gmid`/`gmid2` now `CharField(max_length=20)`
|
|
- **Best Practice**: Discord IDs should always be strings, not integers
|
|
- **Test Result**: ✅ Large Discord IDs now handled properly
|
|
|
|
### MIGRATION_LOGIC-DRAFTDATA-INTEGER-001 ✅
|
|
- **Resolution**: Converted Discord channel IDs to strings
|
|
- **Date Resolved**: 2025-08-18
|
|
- **Root Cause**: Channel IDs exceed INTEGER range
|
|
- **Solution Applied**: `result_channel`/`ping_channel` now `CharField(max_length=20)`
|
|
- **Test Result**: ✅ draftdata table migrated successfully (1 record)
|
|
|
|
---
|
|
|
|
## 🔴 Critical Issues (Migration Blockers)
|
|
|
|
### CONSTRAINT-CURRENT-BSTATCOUNT-001 🆕
|
|
- **Priority**: CRITICAL
|
|
- **Table**: current
|
|
- **Error**: `null value in column "bstatcount" violates not-null constraint`
|
|
- **Impact**: Blocks current table migration (11 records affected)
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Make field nullable or set default value
|
|
|
|
### CONSTRAINT-CURRENT-PSTATCOUNT-001 🆕
|
|
- **Priority**: CRITICAL (Predicted)
|
|
- **Table**: current
|
|
- **Error**: `null value in column "pstatcount" violates not-null constraint` (likely)
|
|
- **Impact**: Blocks current table migration (11 records affected)
|
|
- **Status**: PREDICTED
|
|
- **Solution**: Make field nullable or set default value
|
|
|
|
### CONSTRAINT-TEAM-AUTO_DRAFT-001 🆕
|
|
- **Priority**: CRITICAL
|
|
- **Table**: team
|
|
- **Error**: `null value in column "auto_draft" violates not-null constraint`
|
|
- **Impact**: Blocks team table migration (546 records affected)
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Make field nullable or set default value (False)
|
|
|
|
### SCHEMA-CUSTOMCOMMANDCREATOR-MISSING-001
|
|
- **Priority**: CRITICAL
|
|
- **Table**: customcommandcreator
|
|
- **Error**: `no such table: customcommandcreator`
|
|
- **Impact**: Table doesn't exist in SQLite source
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Skip table or create empty table
|
|
|
|
### SCHEMA-CUSTOMCOMMAND-MISSING-001
|
|
- **Priority**: CRITICAL
|
|
- **Table**: customcommand
|
|
- **Error**: `no such table: customcommand`
|
|
- **Impact**: Table doesn't exist in SQLite source
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Skip table or create empty table
|
|
|
|
### CONSTRAINT-DECISION-TEAM_ID-001
|
|
- **Priority**: CRITICAL
|
|
- **Table**: decision
|
|
- **Error**: `null value in column "team_id" violates not-null constraint`
|
|
- **Impact**: Blocks decision table migration (20,309 records affected)
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Handle NULL team_id values
|
|
|
|
### FOREIGN_KEY-STRATPLAY-GAME_ID-001
|
|
- **Priority**: CRITICAL
|
|
- **Table**: stratplay
|
|
- **Error**: `violates foreign key constraint "stratplay_game_id_fkey"`
|
|
- **Impact**: Blocks stratplay table migration (192,790 records affected)
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate stratgame table first or fix referential integrity
|
|
|
|
---
|
|
|
|
## 🟡 High Priority Issues (Data Integrity)
|
|
|
|
### FOREIGN_KEY-BATTINGSEASON-PLAYER_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: battingseason
|
|
- **Error**: `violates foreign key constraint "battingseason_player_id_fkey"`
|
|
- **Impact**: References missing player records (4,878 records affected)
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate players first or clean orphaned records
|
|
|
|
### FOREIGN_KEY-PITCHINGSEASON-PLAYER_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: pitchingseason
|
|
- **Error**: `violates foreign key constraint "pitchingseason_player_id_fkey"`
|
|
- **Impact**: References missing player records (2,810 records affected)
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate players first or clean orphaned records
|
|
|
|
### FOREIGN_KEY-FIELDINGSEASON-PLAYER_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: fieldingseason
|
|
- **Error**: `violates foreign key constraint "fieldingseason_player_id_fkey"`
|
|
- **Impact**: References missing player records (8,981 records affected)
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate players first or clean orphaned records
|
|
|
|
### FOREIGN_KEY-RESULT-TEAM_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: result
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Result records may reference missing teams
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Ensure teams migrate before results
|
|
|
|
### FOREIGN_KEY-SCHEDULE-TEAM_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: schedule
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Schedule records may reference missing teams
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Ensure teams migrate before schedules
|
|
|
|
### FOREIGN_KEY-TRANSACTION-PLAYER_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: transaction
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Transaction records may reference missing players/teams
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Fix dependency order
|
|
|
|
### FOREIGN_KEY-BATTINGSTAT-PLAYER_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: battingstat
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Stats records may reference missing players
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate players/teams first
|
|
|
|
### FOREIGN_KEY-PITCHINGSTAT-PLAYER_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: pitchingstat
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Stats records may reference missing players
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate players/teams first
|
|
|
|
### FOREIGN_KEY-STANDINGS-TEAM_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: standings
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Standings records may reference missing teams
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate teams first
|
|
|
|
### FOREIGN_KEY-DRAFTPICK-TEAM_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: draftpick
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Draft records may reference missing teams/players
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Fix dependency order
|
|
|
|
### FOREIGN_KEY-DRAFTLIST-TEAM_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: draftlist
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Draft list records may reference missing teams/players
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Fix dependency order
|
|
|
|
### FOREIGN_KEY-AWARD-MANAGER_ID-001
|
|
- **Priority**: HIGH
|
|
- **Table**: award
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Award records may reference missing managers/players
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate managers/players first
|
|
|
|
---
|
|
|
|
## 🟢 Medium Priority Issues (Data Quality)
|
|
|
|
### FOREIGN_KEY-DICEROLL-TEAM_ID-001
|
|
- **Priority**: MEDIUM
|
|
- **Table**: diceroll
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Dice roll records may reference missing teams
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate teams first or allow NULL teams
|
|
|
|
### FOREIGN_KEY-KEEPER-TEAM_ID-001
|
|
- **Priority**: MEDIUM
|
|
- **Table**: keeper
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Keeper records may reference missing teams/players
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Fix dependency order
|
|
|
|
### FOREIGN_KEY-INJURY-PLAYER_ID-001
|
|
- **Priority**: MEDIUM
|
|
- **Table**: injury
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Injury records may reference missing players
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate players first
|
|
|
|
### FOREIGN_KEY-STRATGAME-TEAM_ID-001
|
|
- **Priority**: MEDIUM
|
|
- **Table**: stratgame
|
|
- **Error**: Foreign key constraint violations (estimated)
|
|
- **Impact**: Game records may reference missing teams/managers
|
|
- **Status**: IDENTIFIED
|
|
- **Solution**: Migrate teams/managers first
|
|
|
|
---
|
|
|
|
## 📊 Resolution Strategy
|
|
|
|
### Phase 1: Schema Fixes (Fix PostgreSQL Schema)
|
|
- [ ] CONSTRAINT-CURRENT-INJURY_COUNT-001: Make injury_count nullable or set default
|
|
- [ ] DATA_QUALITY-PLAYER-NAME-001: Increase VARCHAR limits
|
|
- [ ] MIGRATION_LOGIC-TEAM-INTEGER-001: Use BIGINT for large integers
|
|
- [ ] MIGRATION_LOGIC-DRAFTDATA-INTEGER-001: Use BIGINT for large integers
|
|
|
|
### Phase 2: Missing Tables (Handle Non-existent Tables)
|
|
- [ ] SCHEMA-CUSTOMCOMMANDCREATOR-MISSING-001: Skip gracefully
|
|
- [ ] SCHEMA-CUSTOMCOMMAND-MISSING-001: Skip gracefully
|
|
|
|
### Phase 3: Data Cleaning (Fix SQLite Data)
|
|
- [ ] CONSTRAINT-DECISION-TEAM_ID-001: Handle NULL team_id values
|
|
- [ ] Clean orphaned records before migration
|
|
|
|
### Phase 4: Dependency Order (Fix Migration Logic)
|
|
- [ ] Migrate base tables first: Current, Manager, Division, SbaPlayer
|
|
- [ ] Then dependent tables: Team, Player
|
|
- [ ] Finally stats and transaction tables
|
|
- [ ] Disable foreign key checks during migration if needed
|
|
|
|
### Phase 5: Validation
|
|
- [ ] Run full migration test
|
|
- [ ] Validate all record counts
|
|
- [ ] Check referential integrity
|
|
- [ ] Performance testing
|
|
|
|
---
|
|
|
|
## 📈 Progress Tracking
|
|
|
|
### Test Run History
|
|
| Run # | Date | Issues Found | Issues Fixed | Status | Notes |
|
|
|-------|------|--------------|--------------|--------|-------|
|
|
| 1 | 2025-08-18 16:52 | 24 | 0 | Discovery Complete | Initial discovery run |
|
|
| 2 | 2025-08-18 17:53 | 3 new | 4 | Phase 1 Complete | Schema fixes successful |
|
|
| 3 | | | | Planned | Phase 2: NULL constraints |
|
|
|
|
### Test Run #2 Details (Phase 1)
|
|
**Duration**: ~3 minutes
|
|
**Focus**: Critical schema issues
|
|
**Approach**: Fixed 4 blocking schema problems
|
|
|
|
**Issues Resolved**:
|
|
1. ✅ CONSTRAINT-CURRENT-INJURY_COUNT-001 → Made nullable
|
|
2. ✅ DATA_QUALITY-PLAYER-NAME-001 → Increased VARCHAR limits
|
|
3. ✅ MIGRATION_LOGIC-TEAM-INTEGER-001 → Discord IDs to strings
|
|
4. ✅ MIGRATION_LOGIC-DRAFTDATA-INTEGER-001 → Channel IDs to strings
|
|
|
|
**New Issues Found**:
|
|
1. 🆕 CONSTRAINT-CURRENT-BSTATCOUNT-001 → NULL stats count
|
|
2. 🆕 CONSTRAINT-CURRENT-PSTATCOUNT-001 → NULL stats count (predicted)
|
|
3. 🆕 CONSTRAINT-TEAM-AUTO_DRAFT-001 → NULL auto draft flag
|
|
|
|
**Migration Results**:
|
|
- ✅ **7 tables migrated successfully** (vs 0 in Run #1)
|
|
- ✅ **5,432 records migrated** (manager, division, sbaplayer, careers, draftdata)
|
|
- ✅ **No integer overflow errors**
|
|
- ✅ **No string length errors**
|
|
- ⚠️ **3 new NULL constraint issues discovered**
|
|
|
|
### Next Actions (Phase 2)
|
|
1. **Immediate**: Fix 3 new NULL constraint issues discovered in Phase 1
|
|
- [ ] CONSTRAINT-CURRENT-BSTATCOUNT-001: Make bstatcount nullable
|
|
- [ ] CONSTRAINT-CURRENT-PSTATCOUNT-001: Make pstatcount nullable
|
|
- [ ] CONSTRAINT-TEAM-AUTO_DRAFT-001: Make auto_draft nullable or set default False
|
|
2. **Then**: Handle missing tables gracefully (custom command tables)
|
|
3. **Next**: Fix dependency order and foreign key issues
|
|
4. **Finally**: Data cleaning and validation
|
|
|
|
### Success Metrics (Current Status)
|
|
- **Tables Successfully Migrating**: 7/30 (23%) ⬆️ from 0/30
|
|
- **Records Successfully Migrated**: ~5,432 ⬆️ from 0
|
|
- **Critical Issues Resolved**: 4/8 (50%) ⬆️ from 0/8
|
|
- **Schema Issues**: ✅ Resolved (integers, string lengths)
|
|
- **NULL Constraints**: ⚠️ 3 new issues discovered
|
|
|
|
---
|
|
|
|
*This tracker will be updated after each test run to monitor progress toward successful migration.* |