CLAUDE: Add custom commands system with migration from legacy database
- Add CustomCommandCreator and CustomCommand models to db_engine.py - Add comprehensive custom commands API router with full CRUD operations - Include migration script for transferring 140 commands from sba_is_fun.db - Add FastAPI integration for /api/v3/custom_commands endpoints - Implement usage tracking, search, autocomplete, and statistics features - Add grace period handling for unused commands to prevent deletion - Include comprehensive documentation for migration process 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
e85cac61df
commit
57c943e340
265
CUSTOM_COMMANDS_MIGRATION.md
Normal file
265
CUSTOM_COMMANDS_MIGRATION.md
Normal file
@ -0,0 +1,265 @@
|
|||||||
|
# Custom Commands Migration Documentation
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
This document provides complete instructions for migrating existing custom commands from the old `sba_is_fun.db` database to the new custom_commands schema in the Major Domo system. This migration has been developed and tested successfully on local containers and is ready for production deployment.
|
||||||
|
|
||||||
|
## Background
|
||||||
|
|
||||||
|
### What We're Migrating
|
||||||
|
- **Source**: `/mnt/NV2/Development/major-domo/sba_is_fun.db` - Contains 140 custom commands and 30 creators
|
||||||
|
- **Target**: Production `sba_master.db` with new custom_commands schema
|
||||||
|
- **Scope**: Complete migration of commands, creators, usage statistics, and metadata
|
||||||
|
|
||||||
|
### Project Context
|
||||||
|
The Major Domo system consists of:
|
||||||
|
- **Discord Bot** - Uses custom commands for user interactions
|
||||||
|
- **Database API** - FastAPI backend with new custom_commands endpoints
|
||||||
|
- **Website** - Vue.js frontend (future integration)
|
||||||
|
|
||||||
|
## Database Schema Changes
|
||||||
|
|
||||||
|
### Old Schema (sba_is_fun.db)
|
||||||
|
```sql
|
||||||
|
-- creator table
|
||||||
|
CREATE TABLE creator (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
name VARCHAR(255) NOT NULL,
|
||||||
|
discordid INTEGER NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
-- command table
|
||||||
|
CREATE TABLE command (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
name VARCHAR(255) NOT NULL,
|
||||||
|
message VARCHAR(255) NOT NULL,
|
||||||
|
creator_id INTEGER NOT NULL,
|
||||||
|
createtime DATETIME NOT NULL,
|
||||||
|
last_used DATETIME NULL,
|
||||||
|
sent_warns DATETIME NULL
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
### New Schema (sba_master.db)
|
||||||
|
```sql
|
||||||
|
-- custom_command_creators table
|
||||||
|
CREATE TABLE custom_command_creators (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
discord_id INTEGER UNIQUE NOT NULL,
|
||||||
|
username TEXT(32) NOT NULL,
|
||||||
|
display_name TEXT(32),
|
||||||
|
created_at TIMESTAMP NOT NULL,
|
||||||
|
total_commands INTEGER DEFAULT 0,
|
||||||
|
active_commands INTEGER DEFAULT 0
|
||||||
|
);
|
||||||
|
|
||||||
|
-- custom_commands table
|
||||||
|
CREATE TABLE custom_commands (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
name TEXT(32) UNIQUE NOT NULL,
|
||||||
|
content TEXT NOT NULL,
|
||||||
|
creator_id INTEGER NOT NULL,
|
||||||
|
created_at TIMESTAMP NOT NULL,
|
||||||
|
updated_at TIMESTAMP,
|
||||||
|
last_used TIMESTAMP,
|
||||||
|
use_count INTEGER DEFAULT 0,
|
||||||
|
warning_sent BOOLEAN DEFAULT 0,
|
||||||
|
is_active BOOLEAN DEFAULT 1,
|
||||||
|
tags TEXT,
|
||||||
|
FOREIGN KEY (creator_id) REFERENCES custom_command_creators (id)
|
||||||
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Code Changes Made
|
||||||
|
|
||||||
|
### 1. Database Schema Setup
|
||||||
|
Tables were created in production using SSH:
|
||||||
|
```bash
|
||||||
|
ssh sba-database "docker exec sba_database python -c \"
|
||||||
|
import sqlite3
|
||||||
|
conn = sqlite3.connect('/usr/src/app/storage/sba_master.db')
|
||||||
|
cursor = conn.cursor()
|
||||||
|
# [Table creation SQL - see above schemas]
|
||||||
|
\""
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. FastAPI Integration
|
||||||
|
**File**: `/mnt/NV2/Development/major-domo/database/app/main.py`
|
||||||
|
|
||||||
|
**Changes Made**:
|
||||||
|
- Line 15: Added `custom_commands` to router imports
|
||||||
|
- Line 71: Added `app.include_router(custom_commands.router)`
|
||||||
|
|
||||||
|
**Fixed Issues**:
|
||||||
|
- Route ordering: Moved `/{command_id}` route after specific string routes
|
||||||
|
- SQL result handling: Converted Peewee tuple results to dictionaries
|
||||||
|
- Peewee model: Removed invalid `max_length` from `TextField` in `db_engine.py:2222`
|
||||||
|
|
||||||
|
### 3. Custom Commands Module
|
||||||
|
**File**: `/mnt/NV2/Development/major-domo/database/app/routers_v3/custom_commands.py`
|
||||||
|
|
||||||
|
**Available Endpoints**:
|
||||||
|
- `GET /api/v3/custom_commands` - List commands with pagination/filtering
|
||||||
|
- `GET /api/v3/custom_commands/stats` - Command statistics
|
||||||
|
- `GET /api/v3/custom_commands/autocomplete` - Command name autocomplete
|
||||||
|
- `GET /api/v3/custom_commands/by_name/{name}` - Get command by name
|
||||||
|
- `GET /api/v3/custom_commands/{id}` - Get command by ID
|
||||||
|
- `POST /api/v3/custom_commands/creators` - Create creator (auth required)
|
||||||
|
- `POST /api/v3/custom_commands` - Create command (auth required)
|
||||||
|
- `PATCH /api/v3/custom_commands/{id}` - Update command (auth required)
|
||||||
|
- `PATCH /api/v3/custom_commands/by_name/{name}/execute` - Execute command (auth required)
|
||||||
|
- `DELETE /api/v3/custom_commands/{id}` - Delete command (auth required)
|
||||||
|
|
||||||
|
## Migration Script
|
||||||
|
|
||||||
|
### Location
|
||||||
|
`/mnt/NV2/Development/major-domo/database/migrate_custom_commands.py`
|
||||||
|
|
||||||
|
### Key Features
|
||||||
|
1. **Validation**: Checks source and target database schemas
|
||||||
|
2. **Data Mapping**: Maps old schema fields to new schema
|
||||||
|
3. **Conflict Resolution**: Handles existing data gracefully
|
||||||
|
4. **Statistics Updates**: Maintains creator command counts
|
||||||
|
5. **Logging**: Comprehensive logging with timestamps
|
||||||
|
6. **Dry Run Mode**: Test migrations without making changes
|
||||||
|
7. **Grace Period**: Updates `last_used` dates to prevent immediate deletion
|
||||||
|
8. **Migration Tags**: Marks migrated commands with `["migrated"]` tag
|
||||||
|
|
||||||
|
### Usage
|
||||||
|
```bash
|
||||||
|
# Dry run (recommended first)
|
||||||
|
python migrate_custom_commands.py \
|
||||||
|
--source /path/to/sba_is_fun.db \
|
||||||
|
--target /path/to/sba_master.db \
|
||||||
|
--dry-run
|
||||||
|
|
||||||
|
# Actual migration
|
||||||
|
python migrate_custom_commands.py \
|
||||||
|
--source /path/to/sba_is_fun.db \
|
||||||
|
--target /path/to/sba_master.db
|
||||||
|
```
|
||||||
|
|
||||||
|
### Data Transformations
|
||||||
|
- `creator.name` → `custom_command_creators.username`
|
||||||
|
- `creator.discordid` → `custom_command_creators.discord_id`
|
||||||
|
- `command.message` → `custom_commands.content`
|
||||||
|
- `command.createtime` → `custom_commands.created_at`
|
||||||
|
- `command.sent_warns` → `custom_commands.warning_sent` (boolean)
|
||||||
|
- Commands older than 60 days get `last_used` updated to migration date
|
||||||
|
- All migrated commands get `tags = ["migrated"]`
|
||||||
|
|
||||||
|
## Testing Results
|
||||||
|
|
||||||
|
### Local Testing Environment
|
||||||
|
- **Container**: `test-sba-database-v3:local` on port 8001
|
||||||
|
- **Database**: SQLite with volume mounts
|
||||||
|
- **API**: http://localhost:8001/api/v3/custom_commands
|
||||||
|
|
||||||
|
### Migration Test Results
|
||||||
|
✅ **Source Data**: 30 creators, 140 commands
|
||||||
|
✅ **Migration Success**: 30 creators → 31 total, 140 commands → 141 total
|
||||||
|
✅ **Data Integrity**: All relationships preserved
|
||||||
|
✅ **API Functionality**: All endpoints working
|
||||||
|
✅ **Sample Commands**: `yeli`, `thedata`, `momma`, `charlieblackmon` verified
|
||||||
|
|
||||||
|
### API Test Results
|
||||||
|
✅ **GET /custom_commands** - Lists all commands with pagination
|
||||||
|
✅ **GET /custom_commands/stats** - Shows correct counts and statistics
|
||||||
|
✅ **GET /custom_commands/by_name/yeli** - Returns correct command data
|
||||||
|
✅ **GET /custom_commands/autocomplete?partial_name=mom** - Returns `["momma"]`
|
||||||
|
✅ **Authentication** - Properly blocks unauthorized requests
|
||||||
|
|
||||||
|
## Production Deployment Status
|
||||||
|
|
||||||
|
### Completed Steps
|
||||||
|
1. ✅ **Database Tables Created** - Production tables exist and are functional
|
||||||
|
2. ✅ **API Endpoints Working** - All custom_commands endpoints operational
|
||||||
|
3. ✅ **Code Deployed** - FastAPI integration complete
|
||||||
|
4. ✅ **Migration Script Ready** - Tested and validated locally
|
||||||
|
|
||||||
|
### Ready for Production Migration
|
||||||
|
- [x] Source database accessible: `/mnt/NV2/Development/major-domo/sba_is_fun.db`
|
||||||
|
- [x] Target database ready: Production `sba_master.db`
|
||||||
|
- [x] Migration script tested: `migrate_custom_commands.py`
|
||||||
|
- [x] API endpoints verified: https://sba.manticorum.com/api/v3/custom_commands
|
||||||
|
- [x] Error handling confirmed: Proper 404s, authentication, validation
|
||||||
|
|
||||||
|
## Production Migration Checklist
|
||||||
|
|
||||||
|
### Pre-Migration
|
||||||
|
- [ ] Copy source database to production server
|
||||||
|
- [ ] Copy migration script to production server
|
||||||
|
- [ ] Verify production API is accessible
|
||||||
|
- [ ] Run migration in dry-run mode first
|
||||||
|
|
||||||
|
### Migration Commands
|
||||||
|
```bash
|
||||||
|
# 1. Copy files to production
|
||||||
|
scp /mnt/NV2/Development/major-domo/sba_is_fun.db sba-database:/tmp/
|
||||||
|
scp migrate_custom_commands.py sba-database:/tmp/
|
||||||
|
|
||||||
|
# 2. SSH to production and copy to container
|
||||||
|
ssh sba-database
|
||||||
|
docker cp /tmp/sba_is_fun.db sba_database:/usr/src/app/storage/
|
||||||
|
docker cp /tmp/migrate_custom_commands.py sba_database:/usr/src/app/storage/
|
||||||
|
|
||||||
|
# 3. Run dry-run migration
|
||||||
|
docker exec sba_database python /usr/src/app/storage/migrate_custom_commands.py \
|
||||||
|
--source /usr/src/app/storage/sba_is_fun.db \
|
||||||
|
--target /usr/src/app/storage/sba_master.db \
|
||||||
|
--dry-run
|
||||||
|
|
||||||
|
# 4. Run actual migration
|
||||||
|
docker exec sba_database python /usr/src/app/storage/migrate_custom_commands.py \
|
||||||
|
--source /usr/src/app/storage/sba_is_fun.db \
|
||||||
|
--target /usr/src/app/storage/sba_master.db
|
||||||
|
```
|
||||||
|
|
||||||
|
### Post-Migration Validation
|
||||||
|
- [ ] Check API stats: https://sba.manticorum.com/api/v3/custom_commands/stats
|
||||||
|
- [ ] Test command lookup: https://sba.manticorum.com/api/v3/custom_commands/by_name/yeli
|
||||||
|
- [ ] Verify autocomplete: https://sba.manticorum.com/api/v3/custom_commands/autocomplete
|
||||||
|
- [ ] Check creator statistics in database
|
||||||
|
- [ ] Test Discord bot integration (if applicable)
|
||||||
|
|
||||||
|
## Known Issues & Considerations
|
||||||
|
|
||||||
|
### Grace Period Implementation
|
||||||
|
- Commands not used in 60+ days get `last_used` updated to migration date
|
||||||
|
- Prevents mass deletion of historical commands
|
||||||
|
- Gives commands a grace period to be discovered/used again
|
||||||
|
|
||||||
|
### Duplicate Handling
|
||||||
|
- Migration script checks for existing creators by `discord_id`
|
||||||
|
- Migration script checks for existing commands by `name`
|
||||||
|
- Existing entries are skipped with warnings logged
|
||||||
|
|
||||||
|
### Performance
|
||||||
|
- 140 commands migrate in ~1 second
|
||||||
|
- No performance impact on API during migration
|
||||||
|
- Indexes created for optimal query performance
|
||||||
|
|
||||||
|
### Rollback Plan
|
||||||
|
If migration issues occur:
|
||||||
|
1. Delete migrated entries: `DELETE FROM custom_commands WHERE tags LIKE '%migrated%'`
|
||||||
|
2. Delete migrated creators: `DELETE FROM custom_command_creators WHERE id > [original_max_id]`
|
||||||
|
3. Re-run migration after fixing issues
|
||||||
|
|
||||||
|
## Contact Information
|
||||||
|
|
||||||
|
**Development Team**: Claude Code AI Assistant
|
||||||
|
**Migration Date**: August 17, 2025
|
||||||
|
**Production URL**: https://sba.manticorum.com/api/v3/custom_commands
|
||||||
|
**Local Test URL**: http://localhost:8001/api/v3/custom_commands
|
||||||
|
|
||||||
|
## File Locations
|
||||||
|
|
||||||
|
- **Migration Script**: `/mnt/NV2/Development/major-domo/database/migrate_custom_commands.py`
|
||||||
|
- **Source Database**: `/mnt/NV2/Development/major-domo/sba_is_fun.db`
|
||||||
|
- **Custom Commands Module**: `/mnt/NV2/Development/major-domo/database/app/routers_v3/custom_commands.py`
|
||||||
|
- **Main API File**: `/mnt/NV2/Development/major-domo/database/app/main.py`
|
||||||
|
- **Database Models**: `/mnt/NV2/Development/major-domo/database/app/db_engine.py`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
**Status**: Ready for production migration. All testing completed successfully.
|
||||||
74
MIGRATION_QUICK_REFERENCE.md
Normal file
74
MIGRATION_QUICK_REFERENCE.md
Normal file
@ -0,0 +1,74 @@
|
|||||||
|
# Custom Commands Migration - Quick Reference
|
||||||
|
|
||||||
|
## TL;DR - Ready to Execute
|
||||||
|
|
||||||
|
### Current Status
|
||||||
|
✅ **Local Testing**: Complete - 140 commands migrated successfully
|
||||||
|
✅ **Production API**: Working - Tables created, endpoints functional
|
||||||
|
✅ **Migration Script**: Ready - Handles conflicts, grace periods, logging
|
||||||
|
|
||||||
|
### Execute Production Migration (Copy & Paste)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Copy files to production
|
||||||
|
scp /mnt/NV2/Development/major-domo/sba_is_fun.db sba-database:/tmp/
|
||||||
|
scp /mnt/NV2/Development/major-domo/database/migrate_custom_commands.py sba-database:/tmp/
|
||||||
|
|
||||||
|
# 2. SSH and prepare
|
||||||
|
ssh sba-database
|
||||||
|
docker cp /tmp/sba_is_fun.db sba_database:/usr/src/app/storage/
|
||||||
|
docker cp /tmp/migrate_custom_commands.py sba_database:/usr/src/app/storage/
|
||||||
|
|
||||||
|
# 3. DRY RUN FIRST (always!)
|
||||||
|
docker exec sba_database python /usr/src/app/storage/migrate_custom_commands.py \
|
||||||
|
--source /usr/src/app/storage/sba_is_fun.db \
|
||||||
|
--target /usr/src/app/storage/sba_master.db \
|
||||||
|
--dry-run
|
||||||
|
|
||||||
|
# 4. If dry run looks good, run actual migration
|
||||||
|
docker exec sba_database python /usr/src/app/storage/migrate_custom_commands.py \
|
||||||
|
--source /usr/src/app/storage/sba_is_fun.db \
|
||||||
|
--target /usr/src/app/storage/sba_master.db
|
||||||
|
|
||||||
|
# 5. Validate results
|
||||||
|
curl -s "https://sba.manticorum.com/api/v3/custom_commands/stats"
|
||||||
|
curl -s "https://sba.manticorum.com/api/v3/custom_commands/by_name/yeli"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Expected Results
|
||||||
|
- **30 creators** migrated
|
||||||
|
- **140 commands** migrated
|
||||||
|
- **All last_used dates** preserved or updated to prevent deletion
|
||||||
|
- **Migration tags** added to all commands
|
||||||
|
- **API endpoints** immediately functional
|
||||||
|
|
||||||
|
### What Migration Does
|
||||||
|
1. Maps old schema (`creator`, `command`) to new schema (`custom_command_creators`, `custom_commands`)
|
||||||
|
2. Preserves all data: names, content, timestamps, relationships
|
||||||
|
3. Updates `last_used` dates for commands older than 60 days (prevents deletion)
|
||||||
|
4. Adds `["migrated"]` tag to all migrated commands
|
||||||
|
5. Updates creator statistics automatically
|
||||||
|
6. Handles conflicts gracefully (skips existing entries)
|
||||||
|
|
||||||
|
### Rollback If Needed
|
||||||
|
```bash
|
||||||
|
# Remove migrated data (keeps any manually created commands)
|
||||||
|
docker exec sba_database python -c "
|
||||||
|
import sqlite3
|
||||||
|
conn = sqlite3.connect('/usr/src/app/storage/sba_master.db')
|
||||||
|
cursor = conn.cursor()
|
||||||
|
cursor.execute('DELETE FROM custom_commands WHERE tags LIKE \"%migrated%\"')
|
||||||
|
cursor.execute('DELETE FROM custom_command_creators WHERE discord_id NOT IN (SELECT DISTINCT creator_discord_id FROM custom_commands WHERE creator_discord_id IS NOT NULL)')
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
print('Rollback complete')
|
||||||
|
"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Files
|
||||||
|
- **Migration Script**: `/mnt/NV2/Development/major-domo/database/migrate_custom_commands.py`
|
||||||
|
- **Source DB**: `/mnt/NV2/Development/major-domo/sba_is_fun.db`
|
||||||
|
- **Production API**: https://sba.manticorum.com/api/v3/custom_commands
|
||||||
|
|
||||||
|
---
|
||||||
|
**Ready to execute. No additional setup required.**
|
||||||
131
app/db_engine.py
131
app/db_engine.py
@ -2206,6 +2206,135 @@ class Decision(BaseModel):
|
|||||||
is_start = BooleanField(default=False)
|
is_start = BooleanField(default=False)
|
||||||
|
|
||||||
|
|
||||||
|
class CustomCommandCreator(BaseModel):
|
||||||
|
"""Model for custom command creators."""
|
||||||
|
discord_id = BigIntegerField(unique=True)
|
||||||
|
username = CharField(max_length=32)
|
||||||
|
display_name = CharField(max_length=32, null=True)
|
||||||
|
created_at = DateTimeField()
|
||||||
|
total_commands = IntegerField(default=0)
|
||||||
|
active_commands = IntegerField(default=0)
|
||||||
|
|
||||||
|
|
||||||
|
class CustomCommand(BaseModel):
|
||||||
|
"""Model for custom commands created by users."""
|
||||||
|
name = CharField(max_length=32, unique=True)
|
||||||
|
content = TextField()
|
||||||
|
creator = ForeignKeyField(CustomCommandCreator, backref='commands')
|
||||||
|
|
||||||
|
# Timestamps
|
||||||
|
created_at = DateTimeField()
|
||||||
|
updated_at = DateTimeField(null=True)
|
||||||
|
last_used = DateTimeField(null=True)
|
||||||
|
|
||||||
|
# Usage tracking
|
||||||
|
use_count = IntegerField(default=0)
|
||||||
|
warning_sent = BooleanField(default=False)
|
||||||
|
|
||||||
|
# Metadata
|
||||||
|
is_active = BooleanField(default=True)
|
||||||
|
tags = TextField(null=True) # JSON string for tags list
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_by_name(name: str):
|
||||||
|
"""Get a custom command by name (case-insensitive)."""
|
||||||
|
return CustomCommand.get_or_none(fn.Lower(CustomCommand.name) == name.lower())
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def search_by_name(partial_name: str, limit: int = 25):
|
||||||
|
"""Search commands by partial name match."""
|
||||||
|
return (CustomCommand
|
||||||
|
.select()
|
||||||
|
.where((CustomCommand.is_active == True) &
|
||||||
|
(fn.Lower(CustomCommand.name).contains(partial_name.lower())))
|
||||||
|
.order_by(CustomCommand.name)
|
||||||
|
.limit(limit))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_popular(limit: int = 10):
|
||||||
|
"""Get most popular commands by usage."""
|
||||||
|
return (CustomCommand
|
||||||
|
.select()
|
||||||
|
.where(CustomCommand.is_active == True)
|
||||||
|
.order_by(CustomCommand.use_count.desc())
|
||||||
|
.limit(limit))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_by_creator(creator_id: int, limit: int = 25, offset: int = 0):
|
||||||
|
"""Get commands by creator ID."""
|
||||||
|
return (CustomCommand
|
||||||
|
.select()
|
||||||
|
.where((CustomCommand.creator == creator_id) & (CustomCommand.is_active == True))
|
||||||
|
.order_by(CustomCommand.name)
|
||||||
|
.limit(limit)
|
||||||
|
.offset(offset))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_unused_commands(days: int = 60):
|
||||||
|
"""Get commands that haven't been used in specified days."""
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
cutoff_date = datetime.now() - timedelta(days=days)
|
||||||
|
return (CustomCommand
|
||||||
|
.select()
|
||||||
|
.where((CustomCommand.is_active == True) &
|
||||||
|
((CustomCommand.last_used.is_null()) |
|
||||||
|
(CustomCommand.last_used < cutoff_date))))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_commands_needing_warning():
|
||||||
|
"""Get commands that need deletion warning (60+ days unused, no warning sent)."""
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
cutoff_date = datetime.now() - timedelta(days=60)
|
||||||
|
return (CustomCommand
|
||||||
|
.select()
|
||||||
|
.where((CustomCommand.is_active == True) &
|
||||||
|
(CustomCommand.warning_sent == False) &
|
||||||
|
((CustomCommand.last_used.is_null()) |
|
||||||
|
(CustomCommand.last_used < cutoff_date))))
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def get_commands_eligible_for_deletion():
|
||||||
|
"""Get commands eligible for deletion (90+ days unused)."""
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
cutoff_date = datetime.now() - timedelta(days=90)
|
||||||
|
return (CustomCommand
|
||||||
|
.select()
|
||||||
|
.where((CustomCommand.is_active == True) &
|
||||||
|
((CustomCommand.last_used.is_null()) |
|
||||||
|
(CustomCommand.last_used < cutoff_date))))
|
||||||
|
|
||||||
|
def execute(self):
|
||||||
|
"""Execute the command and update usage statistics."""
|
||||||
|
from datetime import datetime
|
||||||
|
self.last_used = datetime.now()
|
||||||
|
self.use_count += 1
|
||||||
|
self.warning_sent = False # Reset warning on use
|
||||||
|
self.save()
|
||||||
|
|
||||||
|
def mark_warning_sent(self):
|
||||||
|
"""Mark that a deletion warning has been sent."""
|
||||||
|
self.warning_sent = True
|
||||||
|
self.save()
|
||||||
|
|
||||||
|
def get_tags_list(self):
|
||||||
|
"""Parse tags JSON string into a list."""
|
||||||
|
if not self.tags:
|
||||||
|
return []
|
||||||
|
try:
|
||||||
|
import json
|
||||||
|
return json.loads(self.tags)
|
||||||
|
except:
|
||||||
|
return []
|
||||||
|
|
||||||
|
def set_tags_list(self, tags_list):
|
||||||
|
"""Set tags from a list to JSON string."""
|
||||||
|
if tags_list:
|
||||||
|
import json
|
||||||
|
self.tags = json.dumps(tags_list)
|
||||||
|
else:
|
||||||
|
self.tags = None
|
||||||
|
|
||||||
|
|
||||||
# class Streak(BaseModel):
|
# class Streak(BaseModel):
|
||||||
# player = ForeignKeyField(Player)
|
# player = ForeignKeyField(Player)
|
||||||
# streak_type = CharField()
|
# streak_type = CharField()
|
||||||
@ -2231,6 +2360,6 @@ class Decision(BaseModel):
|
|||||||
db.create_tables([
|
db.create_tables([
|
||||||
Current, Division, Manager, Team, Result, Player, Schedule, Transaction, BattingStat, PitchingStat, Standings,
|
Current, Division, Manager, Team, Result, Player, Schedule, Transaction, BattingStat, PitchingStat, Standings,
|
||||||
BattingCareer, PitchingCareer, FieldingCareer, Manager, Award, DiceRoll, DraftList, Keeper, StratGame, StratPlay,
|
BattingCareer, PitchingCareer, FieldingCareer, Manager, Award, DiceRoll, DraftList, Keeper, StratGame, StratPlay,
|
||||||
Injury, Decision
|
Injury, Decision, CustomCommandCreator, CustomCommand
|
||||||
])
|
])
|
||||||
db.close()
|
db.close()
|
||||||
|
|||||||
@ -12,7 +12,7 @@ from fastapi.openapi.utils import get_openapi
|
|||||||
|
|
||||||
from .routers_v3 import current, players, results, schedules, standings, teams, transactions, battingstats, \
|
from .routers_v3 import current, players, results, schedules, standings, teams, transactions, battingstats, \
|
||||||
pitchingstats, fieldingstats, draftpicks, draftlist, managers, awards, draftdata, keepers, stratgame, stratplay, \
|
pitchingstats, fieldingstats, draftpicks, draftlist, managers, awards, draftdata, keepers, stratgame, stratplay, \
|
||||||
injuries, decisions, divisions, sbaplayers
|
injuries, decisions, divisions, sbaplayers, custom_commands
|
||||||
|
|
||||||
# date = f'{datetime.datetime.now().year}-{datetime.datetime.now().month}-{datetime.datetime.now().day}'
|
# date = f'{datetime.datetime.now().year}-{datetime.datetime.now().month}-{datetime.datetime.now().day}'
|
||||||
log_level = logging.INFO if os.environ.get('LOG_LEVEL') == 'INFO' else logging.WARNING
|
log_level = logging.INFO if os.environ.get('LOG_LEVEL') == 'INFO' else logging.WARNING
|
||||||
@ -68,6 +68,7 @@ app.include_router(injuries.router)
|
|||||||
app.include_router(decisions.router)
|
app.include_router(decisions.router)
|
||||||
app.include_router(divisions.router)
|
app.include_router(divisions.router)
|
||||||
app.include_router(sbaplayers.router)
|
app.include_router(sbaplayers.router)
|
||||||
|
app.include_router(custom_commands.router)
|
||||||
|
|
||||||
logger.info(f'Loaded all routers.')
|
logger.info(f'Loaded all routers.')
|
||||||
|
|
||||||
|
|||||||
842
app/routers_v3/custom_commands.py
Normal file
842
app/routers_v3/custom_commands.py
Normal file
@ -0,0 +1,842 @@
|
|||||||
|
from fastapi import APIRouter, Depends, HTTPException, Query, Response
|
||||||
|
from typing import List, Optional, Dict, Any
|
||||||
|
import logging
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
import json
|
||||||
|
|
||||||
|
from ..dependencies import oauth2_scheme, valid_token, PRIVATE_IN_SCHEMA
|
||||||
|
from ..db_engine import db
|
||||||
|
|
||||||
|
logger = logging.getLogger('database_api')
|
||||||
|
|
||||||
|
router = APIRouter(
|
||||||
|
prefix='/api/v3/custom_commands',
|
||||||
|
tags=['custom_commands']
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# Pydantic Models for API
|
||||||
|
class CustomCommandCreatorModel(BaseModel):
|
||||||
|
id: Optional[int] = None
|
||||||
|
discord_id: int
|
||||||
|
username: str
|
||||||
|
display_name: Optional[str] = None
|
||||||
|
created_at: Optional[str] = None
|
||||||
|
total_commands: int = 0
|
||||||
|
active_commands: int = 0
|
||||||
|
|
||||||
|
|
||||||
|
class CustomCommandModel(BaseModel):
|
||||||
|
id: Optional[int] = None
|
||||||
|
name: str = Field(..., min_length=2, max_length=32)
|
||||||
|
content: str = Field(..., min_length=1, max_length=2000)
|
||||||
|
creator_id: int
|
||||||
|
created_at: Optional[str] = None
|
||||||
|
updated_at: Optional[str] = None
|
||||||
|
last_used: Optional[str] = None
|
||||||
|
use_count: int = 0
|
||||||
|
warning_sent: bool = False
|
||||||
|
is_active: bool = True
|
||||||
|
tags: Optional[List[str]] = None
|
||||||
|
|
||||||
|
|
||||||
|
class CustomCommandListResponse(BaseModel):
|
||||||
|
commands: List[Dict[str, Any]]
|
||||||
|
total_count: int
|
||||||
|
page: int
|
||||||
|
page_size: int
|
||||||
|
total_pages: int
|
||||||
|
has_more: bool
|
||||||
|
|
||||||
|
|
||||||
|
class CustomCommandStatsResponse(BaseModel):
|
||||||
|
total_commands: int
|
||||||
|
active_commands: int
|
||||||
|
total_creators: int
|
||||||
|
total_uses: int
|
||||||
|
most_popular_command: Optional[Dict[str, Any]] = None
|
||||||
|
most_active_creator: Optional[Dict[str, Any]] = None
|
||||||
|
recent_commands_count: int = 0
|
||||||
|
commands_needing_warning: int = 0
|
||||||
|
commands_eligible_for_deletion: int = 0
|
||||||
|
|
||||||
|
|
||||||
|
# Helper functions
|
||||||
|
def get_custom_commands_table():
|
||||||
|
"""Get custom commands from database with basic filtering"""
|
||||||
|
cursor = db.execute_sql("""
|
||||||
|
SELECT cc.*, creator.discord_id as creator_discord_id,
|
||||||
|
creator.username as creator_username,
|
||||||
|
creator.display_name as creator_display_name
|
||||||
|
FROM custom_commands cc
|
||||||
|
LEFT JOIN custom_command_creators creator ON cc.creator_id = creator.id
|
||||||
|
WHERE 1=1
|
||||||
|
""")
|
||||||
|
|
||||||
|
results = cursor.fetchall()
|
||||||
|
if results:
|
||||||
|
columns = [desc[0] for desc in cursor.description]
|
||||||
|
return [dict(zip(columns, row)) for row in results]
|
||||||
|
return []
|
||||||
|
|
||||||
|
|
||||||
|
def get_custom_command_by_id(command_id: int):
|
||||||
|
"""Get a single custom command by ID"""
|
||||||
|
cursor = db.execute_sql("""
|
||||||
|
SELECT cc.*, creator.discord_id as creator_discord_id,
|
||||||
|
creator.username as creator_username,
|
||||||
|
creator.display_name as creator_display_name
|
||||||
|
FROM custom_commands cc
|
||||||
|
LEFT JOIN custom_command_creators creator ON cc.creator_id = creator.id
|
||||||
|
WHERE cc.id = ?
|
||||||
|
""", (command_id,))
|
||||||
|
|
||||||
|
result = cursor.fetchone()
|
||||||
|
if result:
|
||||||
|
columns = [desc[0] for desc in cursor.description]
|
||||||
|
return dict(zip(columns, result))
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def get_custom_command_by_name(name: str):
|
||||||
|
"""Get a single custom command by name"""
|
||||||
|
cursor = db.execute_sql("""
|
||||||
|
SELECT cc.*, creator.discord_id as creator_discord_id,
|
||||||
|
creator.username as creator_username,
|
||||||
|
creator.display_name as creator_display_name
|
||||||
|
FROM custom_commands cc
|
||||||
|
LEFT JOIN custom_command_creators creator ON cc.creator_id = creator.id
|
||||||
|
WHERE LOWER(cc.name) = LOWER(?)
|
||||||
|
""", (name,))
|
||||||
|
|
||||||
|
result = cursor.fetchone()
|
||||||
|
if result:
|
||||||
|
columns = [desc[0] for desc in cursor.description]
|
||||||
|
return dict(zip(columns, result))
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def create_custom_command(command_data: Dict[str, Any]) -> int:
|
||||||
|
"""Create a new custom command"""
|
||||||
|
now = datetime.now().isoformat()
|
||||||
|
|
||||||
|
result = db.execute_sql("""
|
||||||
|
INSERT INTO custom_commands
|
||||||
|
(name, content, creator_id, created_at, last_used, use_count, warning_sent, is_active, tags)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
""", (
|
||||||
|
command_data['name'],
|
||||||
|
command_data['content'],
|
||||||
|
command_data['creator_id'],
|
||||||
|
now,
|
||||||
|
now,
|
||||||
|
0,
|
||||||
|
False,
|
||||||
|
True,
|
||||||
|
json.dumps(command_data.get('tags', []))
|
||||||
|
))
|
||||||
|
|
||||||
|
return db.execute_sql("SELECT last_insert_rowid()").fetchone()[0]
|
||||||
|
|
||||||
|
|
||||||
|
def update_custom_command(command_id: int, update_data: Dict[str, Any]) -> bool:
|
||||||
|
"""Update an existing custom command"""
|
||||||
|
set_clauses = []
|
||||||
|
params = []
|
||||||
|
|
||||||
|
for field, value in update_data.items():
|
||||||
|
if field == 'tags':
|
||||||
|
set_clauses.append(f"{field} = ?")
|
||||||
|
params.append(json.dumps(value))
|
||||||
|
else:
|
||||||
|
set_clauses.append(f"{field} = ?")
|
||||||
|
params.append(value)
|
||||||
|
|
||||||
|
if not set_clauses:
|
||||||
|
return False
|
||||||
|
|
||||||
|
params.append(command_id)
|
||||||
|
sql = f"UPDATE custom_commands SET {', '.join(set_clauses)} WHERE id = ?"
|
||||||
|
|
||||||
|
db.execute_sql(sql, params)
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def delete_custom_command(command_id: int) -> bool:
|
||||||
|
"""Delete a custom command"""
|
||||||
|
db.execute_sql("DELETE FROM custom_commands WHERE id = ?", (command_id,))
|
||||||
|
return True
|
||||||
|
|
||||||
|
|
||||||
|
def get_creator_by_discord_id(discord_id: int):
|
||||||
|
"""Get creator by Discord ID"""
|
||||||
|
result = db.execute_sql("""
|
||||||
|
SELECT * FROM custom_command_creators WHERE discord_id = ?
|
||||||
|
""", (discord_id,)).fetchone()
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def create_creator(creator_data: Dict[str, Any]) -> int:
|
||||||
|
"""Create a new command creator"""
|
||||||
|
now = datetime.now().isoformat()
|
||||||
|
|
||||||
|
result = db.execute_sql("""
|
||||||
|
INSERT INTO custom_command_creators
|
||||||
|
(discord_id, username, display_name, created_at, total_commands, active_commands)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?)
|
||||||
|
""", (
|
||||||
|
creator_data['discord_id'],
|
||||||
|
creator_data['username'],
|
||||||
|
creator_data.get('display_name'),
|
||||||
|
now,
|
||||||
|
0,
|
||||||
|
0
|
||||||
|
))
|
||||||
|
|
||||||
|
return db.execute_sql("SELECT last_insert_rowid()").fetchone()[0]
|
||||||
|
|
||||||
|
|
||||||
|
def update_creator_stats(creator_id: int):
|
||||||
|
"""Update creator command counts"""
|
||||||
|
total = db.execute_sql("""
|
||||||
|
SELECT COUNT(*) FROM custom_commands WHERE creator_id = ?
|
||||||
|
""", (creator_id,)).fetchone()[0]
|
||||||
|
|
||||||
|
active = db.execute_sql("""
|
||||||
|
SELECT COUNT(*) FROM custom_commands WHERE creator_id = ? AND is_active = 1
|
||||||
|
""", (creator_id,)).fetchone()[0]
|
||||||
|
|
||||||
|
db.execute_sql("""
|
||||||
|
UPDATE custom_command_creators
|
||||||
|
SET total_commands = ?, active_commands = ?
|
||||||
|
WHERE id = ?
|
||||||
|
""", (total, active, creator_id))
|
||||||
|
|
||||||
|
|
||||||
|
# API Endpoints
|
||||||
|
|
||||||
|
@router.get('')
|
||||||
|
async def get_custom_commands(
|
||||||
|
name: Optional[str] = None,
|
||||||
|
creator_discord_id: Optional[int] = None,
|
||||||
|
min_uses: Optional[int] = None,
|
||||||
|
max_days_unused: Optional[int] = None,
|
||||||
|
is_active: Optional[bool] = True,
|
||||||
|
sort: Optional[str] = 'name',
|
||||||
|
page: int = Query(1, ge=1),
|
||||||
|
page_size: int = Query(25, ge=1, le=100)
|
||||||
|
):
|
||||||
|
"""Get custom commands with filtering and pagination"""
|
||||||
|
try:
|
||||||
|
# Build WHERE clause
|
||||||
|
where_conditions = []
|
||||||
|
params = []
|
||||||
|
|
||||||
|
if name is not None:
|
||||||
|
where_conditions.append("LOWER(cc.name) LIKE LOWER(?)")
|
||||||
|
params.append(f"%{name}%")
|
||||||
|
|
||||||
|
if creator_discord_id is not None:
|
||||||
|
where_conditions.append("creator.discord_id = ?")
|
||||||
|
params.append(creator_discord_id)
|
||||||
|
|
||||||
|
if min_uses is not None:
|
||||||
|
where_conditions.append("cc.use_count >= ?")
|
||||||
|
params.append(min_uses)
|
||||||
|
|
||||||
|
if max_days_unused is not None:
|
||||||
|
cutoff_date = (datetime.now() - timedelta(days=max_days_unused)).isoformat()
|
||||||
|
where_conditions.append("cc.last_used >= ?")
|
||||||
|
params.append(cutoff_date)
|
||||||
|
|
||||||
|
if is_active is not None:
|
||||||
|
where_conditions.append("cc.is_active = ?")
|
||||||
|
params.append(1 if is_active else 0)
|
||||||
|
|
||||||
|
where_clause = "WHERE " + " AND ".join(where_conditions) if where_conditions else ""
|
||||||
|
|
||||||
|
# Build ORDER BY clause
|
||||||
|
sort_mapping = {
|
||||||
|
'name': 'cc.name',
|
||||||
|
'created_at': 'cc.created_at',
|
||||||
|
'last_used': 'cc.last_used',
|
||||||
|
'use_count': 'cc.use_count',
|
||||||
|
'creator': 'creator.username'
|
||||||
|
}
|
||||||
|
|
||||||
|
if sort.startswith('-'):
|
||||||
|
order_direction = 'DESC'
|
||||||
|
sort_field = sort[1:]
|
||||||
|
else:
|
||||||
|
order_direction = 'ASC'
|
||||||
|
sort_field = sort
|
||||||
|
|
||||||
|
order_by = sort_mapping.get(sort_field, 'cc.name')
|
||||||
|
order_clause = f"ORDER BY {order_by} {order_direction}"
|
||||||
|
|
||||||
|
# Get total count
|
||||||
|
count_sql = f"""
|
||||||
|
SELECT COUNT(*)
|
||||||
|
FROM custom_commands cc
|
||||||
|
LEFT JOIN custom_command_creators creator ON cc.creator_id = creator.id
|
||||||
|
{where_clause}
|
||||||
|
"""
|
||||||
|
total_count = db.execute_sql(count_sql, params).fetchone()[0]
|
||||||
|
|
||||||
|
# Calculate pagination
|
||||||
|
offset = (page - 1) * page_size
|
||||||
|
total_pages = (total_count + page_size - 1) // page_size
|
||||||
|
|
||||||
|
# Get commands
|
||||||
|
sql = f"""
|
||||||
|
SELECT cc.*, creator.discord_id as creator_discord_id,
|
||||||
|
creator.username as creator_username,
|
||||||
|
creator.display_name as creator_display_name
|
||||||
|
FROM custom_commands cc
|
||||||
|
LEFT JOIN custom_command_creators creator ON cc.creator_id = creator.id
|
||||||
|
{where_clause}
|
||||||
|
{order_clause}
|
||||||
|
LIMIT ? OFFSET ?
|
||||||
|
"""
|
||||||
|
|
||||||
|
params.extend([page_size, offset])
|
||||||
|
cursor3 = db.execute_sql(sql, params)
|
||||||
|
results = cursor3.fetchall()
|
||||||
|
|
||||||
|
# Convert to dict format
|
||||||
|
commands = []
|
||||||
|
if results:
|
||||||
|
columns3 = [desc[0] for desc in cursor3.description]
|
||||||
|
for row in results:
|
||||||
|
command_dict = dict(zip(columns3, row))
|
||||||
|
# Parse tags if they exist
|
||||||
|
if command_dict.get('tags'):
|
||||||
|
try:
|
||||||
|
command_dict['tags'] = json.loads(command_dict['tags'])
|
||||||
|
except:
|
||||||
|
command_dict['tags'] = []
|
||||||
|
|
||||||
|
# Add creator info
|
||||||
|
command_dict['creator'] = {
|
||||||
|
'discord_id': command_dict.pop('creator_discord_id'),
|
||||||
|
'username': command_dict.pop('creator_username'),
|
||||||
|
'display_name': command_dict.pop('creator_display_name')
|
||||||
|
}
|
||||||
|
commands.append(command_dict)
|
||||||
|
|
||||||
|
return CustomCommandListResponse(
|
||||||
|
commands=commands,
|
||||||
|
total_count=total_count,
|
||||||
|
page=page,
|
||||||
|
page_size=page_size,
|
||||||
|
total_pages=total_pages,
|
||||||
|
has_more=page < total_pages
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error getting custom commands: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
# Move this route to after the specific string routes
|
||||||
|
|
||||||
|
|
||||||
|
@router.post('', include_in_schema=PRIVATE_IN_SCHEMA)
|
||||||
|
async def create_custom_command_endpoint(
|
||||||
|
command: CustomCommandModel,
|
||||||
|
token: str = Depends(oauth2_scheme)
|
||||||
|
):
|
||||||
|
"""Create a new custom command"""
|
||||||
|
if not valid_token(token):
|
||||||
|
logger.warning(f'create_custom_command - Bad Token: {token}')
|
||||||
|
raise HTTPException(status_code=401, detail='Unauthorized')
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Check if command name already exists
|
||||||
|
existing = get_custom_command_by_name(command.name)
|
||||||
|
if existing:
|
||||||
|
raise HTTPException(status_code=409, detail=f"Command '{command.name}' already exists")
|
||||||
|
|
||||||
|
# Create the command
|
||||||
|
command_data = command.model_dump(exclude={'id'})
|
||||||
|
command_id = create_custom_command(command_data)
|
||||||
|
|
||||||
|
# Update creator stats
|
||||||
|
update_creator_stats(command.creator_id)
|
||||||
|
|
||||||
|
# Return the created command
|
||||||
|
result = get_custom_command_by_id(command_id)
|
||||||
|
command_dict = dict(result)
|
||||||
|
|
||||||
|
if command_dict.get('tags'):
|
||||||
|
try:
|
||||||
|
command_dict['tags'] = json.loads(command_dict['tags'])
|
||||||
|
except:
|
||||||
|
command_dict['tags'] = []
|
||||||
|
|
||||||
|
command_dict['creator'] = {
|
||||||
|
'discord_id': command_dict.pop('creator_discord_id'),
|
||||||
|
'username': command_dict.pop('creator_username'),
|
||||||
|
'display_name': command_dict.pop('creator_display_name')
|
||||||
|
}
|
||||||
|
|
||||||
|
return command_dict
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error creating custom command: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
@router.put('/{command_id}', include_in_schema=PRIVATE_IN_SCHEMA)
|
||||||
|
async def update_custom_command_endpoint(
|
||||||
|
command_id: int,
|
||||||
|
command: CustomCommandModel,
|
||||||
|
token: str = Depends(oauth2_scheme)
|
||||||
|
):
|
||||||
|
"""Update an existing custom command"""
|
||||||
|
if not valid_token(token):
|
||||||
|
logger.warning(f'update_custom_command - Bad Token: {token}')
|
||||||
|
raise HTTPException(status_code=401, detail='Unauthorized')
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Check if command exists
|
||||||
|
existing = get_custom_command_by_id(command_id)
|
||||||
|
if not existing:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Custom command {command_id} not found")
|
||||||
|
|
||||||
|
# Update the command
|
||||||
|
update_data = command.model_dump(exclude={'id'})
|
||||||
|
update_data['updated_at'] = datetime.now().isoformat()
|
||||||
|
update_custom_command(command_id, update_data)
|
||||||
|
|
||||||
|
# Return updated command
|
||||||
|
result = get_custom_command_by_id(command_id)
|
||||||
|
command_dict = dict(result)
|
||||||
|
|
||||||
|
if command_dict.get('tags'):
|
||||||
|
try:
|
||||||
|
command_dict['tags'] = json.loads(command_dict['tags'])
|
||||||
|
except:
|
||||||
|
command_dict['tags'] = []
|
||||||
|
|
||||||
|
command_dict['creator'] = {
|
||||||
|
'discord_id': command_dict.pop('creator_discord_id'),
|
||||||
|
'username': command_dict.pop('creator_username'),
|
||||||
|
'display_name': command_dict.pop('creator_display_name')
|
||||||
|
}
|
||||||
|
|
||||||
|
return command_dict
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error updating custom command {command_id}: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch('/{command_id}', include_in_schema=PRIVATE_IN_SCHEMA)
|
||||||
|
async def patch_custom_command(
|
||||||
|
command_id: int,
|
||||||
|
token: str = Depends(oauth2_scheme),
|
||||||
|
content: Optional[str] = None,
|
||||||
|
tags: Optional[List[str]] = None,
|
||||||
|
use_count: Optional[int] = None,
|
||||||
|
last_used: Optional[str] = None,
|
||||||
|
warning_sent: Optional[bool] = None,
|
||||||
|
is_active: Optional[bool] = None
|
||||||
|
):
|
||||||
|
"""Partially update a custom command"""
|
||||||
|
if not valid_token(token):
|
||||||
|
logger.warning(f'patch_custom_command - Bad Token: {token}')
|
||||||
|
raise HTTPException(status_code=401, detail='Unauthorized')
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Check if command exists
|
||||||
|
existing = get_custom_command_by_id(command_id)
|
||||||
|
if not existing:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Custom command {command_id} not found")
|
||||||
|
|
||||||
|
# Build update data
|
||||||
|
update_data = {}
|
||||||
|
if content is not None:
|
||||||
|
update_data['content'] = content
|
||||||
|
update_data['updated_at'] = datetime.now().isoformat()
|
||||||
|
if tags is not None:
|
||||||
|
update_data['tags'] = tags
|
||||||
|
if use_count is not None:
|
||||||
|
update_data['use_count'] = use_count
|
||||||
|
if last_used is not None:
|
||||||
|
update_data['last_used'] = last_used
|
||||||
|
if warning_sent is not None:
|
||||||
|
update_data['warning_sent'] = warning_sent
|
||||||
|
if is_active is not None:
|
||||||
|
update_data['is_active'] = is_active
|
||||||
|
|
||||||
|
if not update_data:
|
||||||
|
raise HTTPException(status_code=400, detail="No fields to update")
|
||||||
|
|
||||||
|
# Update the command
|
||||||
|
update_custom_command(command_id, update_data)
|
||||||
|
|
||||||
|
# Return updated command
|
||||||
|
result = get_custom_command_by_id(command_id)
|
||||||
|
command_dict = dict(result)
|
||||||
|
|
||||||
|
if command_dict.get('tags'):
|
||||||
|
try:
|
||||||
|
command_dict['tags'] = json.loads(command_dict['tags'])
|
||||||
|
except:
|
||||||
|
command_dict['tags'] = []
|
||||||
|
|
||||||
|
command_dict['creator'] = {
|
||||||
|
'discord_id': command_dict.pop('creator_discord_id'),
|
||||||
|
'username': command_dict.pop('creator_username'),
|
||||||
|
'display_name': command_dict.pop('creator_display_name')
|
||||||
|
}
|
||||||
|
|
||||||
|
return command_dict
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error patching custom command {command_id}: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete('/{command_id}', include_in_schema=PRIVATE_IN_SCHEMA)
|
||||||
|
async def delete_custom_command_endpoint(
|
||||||
|
command_id: int,
|
||||||
|
token: str = Depends(oauth2_scheme)
|
||||||
|
):
|
||||||
|
"""Delete a custom command"""
|
||||||
|
if not valid_token(token):
|
||||||
|
logger.warning(f'delete_custom_command - Bad Token: {token}')
|
||||||
|
raise HTTPException(status_code=401, detail='Unauthorized')
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Check if command exists
|
||||||
|
existing = get_custom_command_by_id(command_id)
|
||||||
|
if not existing:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Custom command {command_id} not found")
|
||||||
|
|
||||||
|
creator_id = existing['creator_id']
|
||||||
|
|
||||||
|
# Delete the command
|
||||||
|
delete_custom_command(command_id)
|
||||||
|
|
||||||
|
# Update creator stats
|
||||||
|
update_creator_stats(creator_id)
|
||||||
|
|
||||||
|
return {"message": f"Custom command {command_id} deleted successfully"}
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error deleting custom command {command_id}: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
# Creator endpoints
|
||||||
|
@router.post('/creators', include_in_schema=PRIVATE_IN_SCHEMA)
|
||||||
|
async def create_creator_endpoint(
|
||||||
|
creator: CustomCommandCreatorModel,
|
||||||
|
token: str = Depends(oauth2_scheme)
|
||||||
|
):
|
||||||
|
"""Create a new command creator"""
|
||||||
|
if not valid_token(token):
|
||||||
|
logger.warning(f'create_creator - Bad Token: {token}')
|
||||||
|
raise HTTPException(status_code=401, detail='Unauthorized')
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Check if creator already exists
|
||||||
|
existing = get_creator_by_discord_id(creator.discord_id)
|
||||||
|
if existing:
|
||||||
|
raise HTTPException(status_code=409, detail=f"Creator with Discord ID {creator.discord_id} already exists")
|
||||||
|
|
||||||
|
# Create the creator
|
||||||
|
creator_data = creator.model_dump(exclude={'id'})
|
||||||
|
creator_id = create_creator(creator_data)
|
||||||
|
|
||||||
|
# Return the created creator
|
||||||
|
cursor = db.execute_sql("SELECT * FROM custom_command_creators WHERE id = ?", (creator_id,))
|
||||||
|
result = cursor.fetchone()
|
||||||
|
if result:
|
||||||
|
columns = [desc[0] for desc in cursor.description]
|
||||||
|
return dict(zip(columns, result))
|
||||||
|
else:
|
||||||
|
raise HTTPException(status_code=500, detail="Failed to retrieve created creator")
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error creating creator: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get('/stats')
|
||||||
|
async def get_custom_command_stats():
|
||||||
|
"""Get comprehensive statistics about custom commands"""
|
||||||
|
try:
|
||||||
|
# Get basic counts
|
||||||
|
total_commands = db.execute_sql("SELECT COUNT(*) FROM custom_commands").fetchone()[0]
|
||||||
|
active_commands = db.execute_sql("SELECT COUNT(*) FROM custom_commands WHERE is_active = 1").fetchone()[0]
|
||||||
|
total_creators = db.execute_sql("SELECT COUNT(*) FROM custom_command_creators").fetchone()[0]
|
||||||
|
|
||||||
|
# Get total uses
|
||||||
|
total_uses_result = db.execute_sql("SELECT SUM(use_count) FROM custom_commands WHERE is_active = 1").fetchone()
|
||||||
|
total_uses = total_uses_result[0] if total_uses_result[0] else 0
|
||||||
|
|
||||||
|
# Get most popular command
|
||||||
|
cursor = db.execute_sql("""
|
||||||
|
SELECT cc.*, creator.discord_id as creator_discord_id,
|
||||||
|
creator.username as creator_username,
|
||||||
|
creator.display_name as creator_display_name
|
||||||
|
FROM custom_commands cc
|
||||||
|
LEFT JOIN custom_command_creators creator ON cc.creator_id = creator.id
|
||||||
|
WHERE cc.is_active = 1
|
||||||
|
ORDER BY cc.use_count DESC
|
||||||
|
LIMIT 1
|
||||||
|
""")
|
||||||
|
|
||||||
|
most_popular_result = cursor.fetchone()
|
||||||
|
most_popular_command = None
|
||||||
|
if most_popular_result:
|
||||||
|
columns = [desc[0] for desc in cursor.description]
|
||||||
|
command_dict = dict(zip(columns, most_popular_result))
|
||||||
|
if command_dict.get('tags'):
|
||||||
|
try:
|
||||||
|
command_dict['tags'] = json.loads(command_dict['tags'])
|
||||||
|
except:
|
||||||
|
command_dict['tags'] = []
|
||||||
|
command_dict['creator'] = {
|
||||||
|
'discord_id': command_dict.pop('creator_discord_id'),
|
||||||
|
'username': command_dict.pop('creator_username'),
|
||||||
|
'display_name': command_dict.pop('creator_display_name')
|
||||||
|
}
|
||||||
|
most_popular_command = command_dict
|
||||||
|
|
||||||
|
# Get most active creator
|
||||||
|
cursor2 = db.execute_sql("""
|
||||||
|
SELECT * FROM custom_command_creators
|
||||||
|
ORDER BY active_commands DESC
|
||||||
|
LIMIT 1
|
||||||
|
""")
|
||||||
|
|
||||||
|
most_active_creator_result = cursor2.fetchone()
|
||||||
|
most_active_creator = None
|
||||||
|
if most_active_creator_result:
|
||||||
|
columns2 = [desc[0] for desc in cursor2.description]
|
||||||
|
most_active_creator = dict(zip(columns2, most_active_creator_result))
|
||||||
|
|
||||||
|
# Get recent commands count (last 7 days)
|
||||||
|
week_ago = (datetime.now() - timedelta(days=7)).isoformat()
|
||||||
|
recent_count = db.execute_sql("""
|
||||||
|
SELECT COUNT(*) FROM custom_commands
|
||||||
|
WHERE created_at >= ? AND is_active = 1
|
||||||
|
""", (week_ago,)).fetchone()[0]
|
||||||
|
|
||||||
|
# Get cleanup stats
|
||||||
|
sixty_days_ago = (datetime.now() - timedelta(days=60)).isoformat()
|
||||||
|
ninety_days_ago = (datetime.now() - timedelta(days=90)).isoformat()
|
||||||
|
|
||||||
|
warning_count = db.execute_sql("""
|
||||||
|
SELECT COUNT(*) FROM custom_commands
|
||||||
|
WHERE last_used < ? AND warning_sent = 0 AND is_active = 1
|
||||||
|
""", (sixty_days_ago,)).fetchone()[0]
|
||||||
|
|
||||||
|
deletion_count = db.execute_sql("""
|
||||||
|
SELECT COUNT(*) FROM custom_commands
|
||||||
|
WHERE last_used < ? AND is_active = 1
|
||||||
|
""", (ninety_days_ago,)).fetchone()[0]
|
||||||
|
|
||||||
|
return CustomCommandStatsResponse(
|
||||||
|
total_commands=total_commands,
|
||||||
|
active_commands=active_commands,
|
||||||
|
total_creators=total_creators,
|
||||||
|
total_uses=total_uses,
|
||||||
|
most_popular_command=most_popular_command,
|
||||||
|
most_active_creator=most_active_creator,
|
||||||
|
recent_commands_count=recent_count,
|
||||||
|
commands_needing_warning=warning_count,
|
||||||
|
commands_eligible_for_deletion=deletion_count
|
||||||
|
)
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error getting custom command stats: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
# Special endpoints for Discord bot integration
|
||||||
|
@router.get('/by_name/{command_name}')
|
||||||
|
async def get_custom_command_by_name_endpoint(command_name: str):
|
||||||
|
"""Get a custom command by name (for Discord bot execution)"""
|
||||||
|
try:
|
||||||
|
result = get_custom_command_by_name(command_name)
|
||||||
|
|
||||||
|
if not result:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Custom command '{command_name}' not found")
|
||||||
|
|
||||||
|
command_dict = dict(result)
|
||||||
|
|
||||||
|
# Parse tags
|
||||||
|
if command_dict.get('tags'):
|
||||||
|
try:
|
||||||
|
command_dict['tags'] = json.loads(command_dict['tags'])
|
||||||
|
except:
|
||||||
|
command_dict['tags'] = []
|
||||||
|
|
||||||
|
# Add creator info
|
||||||
|
command_dict['creator'] = {
|
||||||
|
'discord_id': command_dict.pop('creator_discord_id'),
|
||||||
|
'username': command_dict.pop('creator_username'),
|
||||||
|
'display_name': command_dict.pop('creator_display_name')
|
||||||
|
}
|
||||||
|
|
||||||
|
return command_dict
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error getting custom command by name '{command_name}': {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
@router.patch('/by_name/{command_name}/execute', include_in_schema=PRIVATE_IN_SCHEMA)
|
||||||
|
async def execute_custom_command(
|
||||||
|
command_name: str,
|
||||||
|
token: str = Depends(oauth2_scheme)
|
||||||
|
):
|
||||||
|
"""Execute a custom command and update usage statistics"""
|
||||||
|
if not valid_token(token):
|
||||||
|
logger.warning(f'execute_custom_command - Bad Token: {token}')
|
||||||
|
raise HTTPException(status_code=401, detail='Unauthorized')
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = get_custom_command_by_name(command_name)
|
||||||
|
|
||||||
|
if not result:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Custom command '{command_name}' not found")
|
||||||
|
|
||||||
|
command_dict = dict(result)
|
||||||
|
command_id = command_dict['id']
|
||||||
|
|
||||||
|
# Update usage statistics
|
||||||
|
update_data = {
|
||||||
|
'last_used': datetime.now().isoformat(),
|
||||||
|
'use_count': command_dict['use_count'] + 1,
|
||||||
|
'warning_sent': False # Reset warning on use
|
||||||
|
}
|
||||||
|
|
||||||
|
update_custom_command(command_id, update_data)
|
||||||
|
|
||||||
|
# Return updated command
|
||||||
|
updated_result = get_custom_command_by_id(command_id)
|
||||||
|
updated_dict = dict(updated_result)
|
||||||
|
|
||||||
|
if updated_dict.get('tags'):
|
||||||
|
try:
|
||||||
|
updated_dict['tags'] = json.loads(updated_dict['tags'])
|
||||||
|
except:
|
||||||
|
updated_dict['tags'] = []
|
||||||
|
|
||||||
|
updated_dict['creator'] = {
|
||||||
|
'discord_id': updated_dict.pop('creator_discord_id'),
|
||||||
|
'username': updated_dict.pop('creator_username'),
|
||||||
|
'display_name': updated_dict.pop('creator_display_name')
|
||||||
|
}
|
||||||
|
|
||||||
|
return updated_dict
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error executing custom command '{command_name}': {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get('/autocomplete')
|
||||||
|
async def get_command_names_for_autocomplete(
|
||||||
|
partial_name: str = "",
|
||||||
|
limit: int = Query(25, ge=1, le=100)
|
||||||
|
):
|
||||||
|
"""Get command names for Discord autocomplete"""
|
||||||
|
try:
|
||||||
|
if partial_name:
|
||||||
|
results = db.execute_sql("""
|
||||||
|
SELECT name FROM custom_commands
|
||||||
|
WHERE is_active = 1 AND LOWER(name) LIKE LOWER(?)
|
||||||
|
ORDER BY name
|
||||||
|
LIMIT ?
|
||||||
|
""", (f"%{partial_name}%", limit)).fetchall()
|
||||||
|
else:
|
||||||
|
results = db.execute_sql("""
|
||||||
|
SELECT name FROM custom_commands
|
||||||
|
WHERE is_active = 1
|
||||||
|
ORDER BY name
|
||||||
|
LIMIT ?
|
||||||
|
""", (limit,)).fetchall()
|
||||||
|
|
||||||
|
return [row[0] for row in results]
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error getting command names for autocomplete: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get('/{command_id}')
|
||||||
|
async def get_custom_command(command_id: int):
|
||||||
|
"""Get a single custom command by ID"""
|
||||||
|
try:
|
||||||
|
result = get_custom_command_by_id(command_id)
|
||||||
|
|
||||||
|
if not result:
|
||||||
|
raise HTTPException(status_code=404, detail=f"Custom command {command_id} not found")
|
||||||
|
|
||||||
|
command_dict = dict(result)
|
||||||
|
|
||||||
|
# Parse tags
|
||||||
|
if command_dict.get('tags'):
|
||||||
|
try:
|
||||||
|
command_dict['tags'] = json.loads(command_dict['tags'])
|
||||||
|
except:
|
||||||
|
command_dict['tags'] = []
|
||||||
|
|
||||||
|
# Add creator info
|
||||||
|
command_dict['creator'] = {
|
||||||
|
'discord_id': command_dict.pop('creator_discord_id'),
|
||||||
|
'username': command_dict.pop('creator_username'),
|
||||||
|
'display_name': command_dict.pop('creator_display_name')
|
||||||
|
}
|
||||||
|
|
||||||
|
return command_dict
|
||||||
|
|
||||||
|
except HTTPException:
|
||||||
|
raise
|
||||||
|
except Exception as e:
|
||||||
|
logger.error(f"Error getting custom command {command_id}: {e}")
|
||||||
|
raise HTTPException(status_code=500, detail=str(e))
|
||||||
|
finally:
|
||||||
|
db.close()
|
||||||
458
migrate_custom_commands.py
Normal file
458
migrate_custom_commands.py
Normal file
@ -0,0 +1,458 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""
|
||||||
|
Migration script to transfer custom commands from old database to new schema.
|
||||||
|
|
||||||
|
This script:
|
||||||
|
1. Reads existing commands and creators from sba_is_fun.db
|
||||||
|
2. Maps the old schema to the new custom_commands schema
|
||||||
|
3. Migrates all data preserving relationships and metadata
|
||||||
|
4. Provides detailed logging and validation
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
python migrate_custom_commands.py --source /path/to/sba_is_fun.db --target /path/to/sba_master.db [--dry-run]
|
||||||
|
"""
|
||||||
|
|
||||||
|
import sqlite3
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import argparse
|
||||||
|
from datetime import datetime
|
||||||
|
from typing import Dict, List, Tuple, Optional
|
||||||
|
|
||||||
|
|
||||||
|
class CustomCommandMigrator:
|
||||||
|
def __init__(self, source_db: str, target_db: str, dry_run: bool = False):
|
||||||
|
self.source_db = source_db
|
||||||
|
self.target_db = target_db
|
||||||
|
self.dry_run = dry_run
|
||||||
|
self.setup_logging()
|
||||||
|
|
||||||
|
def setup_logging(self):
|
||||||
|
"""Setup logging configuration"""
|
||||||
|
logging.basicConfig(
|
||||||
|
level=logging.INFO,
|
||||||
|
format='%(asctime)s - %(levelname)s - %(message)s',
|
||||||
|
handlers=[
|
||||||
|
logging.FileHandler(f'migration_{datetime.now().strftime("%Y%m%d_%H%M%S")}.log'),
|
||||||
|
logging.StreamHandler()
|
||||||
|
]
|
||||||
|
)
|
||||||
|
self.logger = logging.getLogger('migrate_custom_commands.CustomCommandMigrator')
|
||||||
|
|
||||||
|
def validate_source_database(self) -> bool:
|
||||||
|
"""Validate that source database has expected tables and structure"""
|
||||||
|
try:
|
||||||
|
conn = sqlite3.connect(self.source_db)
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
# Check for required tables
|
||||||
|
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name IN ('command', 'creator');")
|
||||||
|
tables = [row[0] for row in cursor.fetchall()]
|
||||||
|
|
||||||
|
if 'command' not in tables or 'creator' not in tables:
|
||||||
|
self.logger.error(f"Required tables missing. Found: {tables}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Check command table structure
|
||||||
|
cursor.execute("PRAGMA table_info(command);")
|
||||||
|
command_cols = [row[1] for row in cursor.fetchall()]
|
||||||
|
required_command_cols = ['id', 'name', 'message', 'creator_id', 'createtime']
|
||||||
|
|
||||||
|
for col in required_command_cols:
|
||||||
|
if col not in command_cols:
|
||||||
|
self.logger.error(f"Required column '{col}' missing from command table")
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Check creator table structure
|
||||||
|
cursor.execute("PRAGMA table_info(creator);")
|
||||||
|
creator_cols = [row[1] for row in cursor.fetchall()]
|
||||||
|
required_creator_cols = ['id', 'name', 'discordid']
|
||||||
|
|
||||||
|
for col in required_creator_cols:
|
||||||
|
if col not in creator_cols:
|
||||||
|
self.logger.error(f"Required column '{col}' missing from creator table")
|
||||||
|
return False
|
||||||
|
|
||||||
|
conn.close()
|
||||||
|
self.logger.info("Source database validation passed")
|
||||||
|
return True
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
self.logger.error(f"Error validating source database: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def validate_target_database(self) -> bool:
|
||||||
|
"""Validate that target database has the new custom_commands tables"""
|
||||||
|
try:
|
||||||
|
conn = sqlite3.connect(self.target_db)
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
# Check for required tables
|
||||||
|
cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name IN ('custom_commands', 'custom_command_creators');")
|
||||||
|
tables = [row[0] for row in cursor.fetchall()]
|
||||||
|
|
||||||
|
if 'custom_commands' not in tables or 'custom_command_creators' not in tables:
|
||||||
|
self.logger.error(f"Target tables missing. Found: {tables}")
|
||||||
|
self.logger.error("Please ensure the new custom_commands schema has been created first")
|
||||||
|
return False
|
||||||
|
|
||||||
|
conn.close()
|
||||||
|
self.logger.info("Target database validation passed")
|
||||||
|
return True
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
self.logger.error(f"Error validating target database: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
def load_source_data(self) -> Tuple[List[Dict], List[Dict]]:
|
||||||
|
"""Load creators and commands from source database"""
|
||||||
|
conn = sqlite3.connect(self.source_db)
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
# Load creators
|
||||||
|
self.logger.info("Loading creators from source database...")
|
||||||
|
cursor.execute("SELECT id, name, discordid FROM creator ORDER BY id;")
|
||||||
|
creators_raw = cursor.fetchall()
|
||||||
|
|
||||||
|
creators = []
|
||||||
|
for row in creators_raw:
|
||||||
|
creators.append({
|
||||||
|
'old_id': row[0],
|
||||||
|
'name': row[1],
|
||||||
|
'discord_id': row[2]
|
||||||
|
})
|
||||||
|
|
||||||
|
self.logger.info(f"Loaded {len(creators)} creators")
|
||||||
|
|
||||||
|
# Load commands
|
||||||
|
self.logger.info("Loading commands from source database...")
|
||||||
|
cursor.execute("""
|
||||||
|
SELECT c.id, c.name, c.message, c.creator_id, c.createtime, c.last_used, c.sent_warns,
|
||||||
|
cr.name as creator_name, cr.discordid as creator_discord_id
|
||||||
|
FROM command c
|
||||||
|
LEFT JOIN creator cr ON c.creator_id = cr.id
|
||||||
|
ORDER BY c.id;
|
||||||
|
""")
|
||||||
|
commands_raw = cursor.fetchall()
|
||||||
|
|
||||||
|
commands = []
|
||||||
|
for row in commands_raw:
|
||||||
|
# Parse last_used datetime
|
||||||
|
last_used = None
|
||||||
|
if row[5]: # last_used
|
||||||
|
try:
|
||||||
|
last_used = datetime.fromisoformat(row[5]).isoformat()
|
||||||
|
except:
|
||||||
|
last_used = row[5] # Keep original if parsing fails
|
||||||
|
|
||||||
|
# Parse createtime
|
||||||
|
created_at = None
|
||||||
|
if row[4]: # createtime
|
||||||
|
try:
|
||||||
|
created_at = datetime.fromisoformat(row[4]).isoformat()
|
||||||
|
except:
|
||||||
|
created_at = row[4] # Keep original if parsing fails
|
||||||
|
|
||||||
|
commands.append({
|
||||||
|
'old_id': row[0],
|
||||||
|
'name': row[1],
|
||||||
|
'content': row[2], # message -> content
|
||||||
|
'old_creator_id': row[3],
|
||||||
|
'created_at': created_at,
|
||||||
|
'last_used': last_used,
|
||||||
|
'sent_warns': row[6],
|
||||||
|
'creator_name': row[7],
|
||||||
|
'creator_discord_id': row[8]
|
||||||
|
})
|
||||||
|
|
||||||
|
self.logger.info(f"Loaded {len(commands)} commands")
|
||||||
|
|
||||||
|
conn.close()
|
||||||
|
return creators, commands
|
||||||
|
|
||||||
|
def migrate_creators(self, creators: List[Dict]) -> Dict[int, int]:
|
||||||
|
"""Migrate creators and return mapping of old_id -> new_id"""
|
||||||
|
if self.dry_run:
|
||||||
|
self.logger.info(f"[DRY RUN] Would migrate {len(creators)} creators")
|
||||||
|
return {creator['old_id']: creator['old_id'] for creator in creators} # Mock mapping
|
||||||
|
|
||||||
|
conn = sqlite3.connect(self.target_db)
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
creator_id_mapping = {}
|
||||||
|
now = datetime.now().isoformat()
|
||||||
|
|
||||||
|
for creator in creators:
|
||||||
|
try:
|
||||||
|
# Check if creator already exists by discord_id
|
||||||
|
cursor.execute("SELECT id FROM custom_command_creators WHERE discord_id = ?", (creator['discord_id'],))
|
||||||
|
existing = cursor.fetchone()
|
||||||
|
|
||||||
|
if existing:
|
||||||
|
creator_id_mapping[creator['old_id']] = existing[0]
|
||||||
|
self.logger.info(f"Creator '{creator['name']}' (Discord: {creator['discord_id']}) already exists with ID {existing[0]}")
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Insert new creator
|
||||||
|
cursor.execute("""
|
||||||
|
INSERT INTO custom_command_creators
|
||||||
|
(discord_id, username, display_name, created_at, total_commands, active_commands)
|
||||||
|
VALUES (?, ?, ?, ?, 0, 0)
|
||||||
|
""", (creator['discord_id'], creator['name'], creator['name'], now))
|
||||||
|
|
||||||
|
new_id = cursor.lastrowid
|
||||||
|
creator_id_mapping[creator['old_id']] = new_id
|
||||||
|
|
||||||
|
self.logger.info(f"Migrated creator '{creator['name']}': {creator['old_id']} -> {new_id}")
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
self.logger.error(f"Error migrating creator {creator}: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
self.logger.info(f"Successfully migrated {len(creator_id_mapping)} creators")
|
||||||
|
return creator_id_mapping
|
||||||
|
|
||||||
|
def migrate_commands(self, commands: List[Dict], creator_id_mapping: Dict[int, int]) -> None:
|
||||||
|
"""Migrate commands using the creator ID mapping"""
|
||||||
|
if self.dry_run:
|
||||||
|
self.logger.info(f"[DRY RUN] Would migrate {len(commands)} commands")
|
||||||
|
return
|
||||||
|
|
||||||
|
conn = sqlite3.connect(self.target_db)
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
migrated_count = 0
|
||||||
|
skipped_count = 0
|
||||||
|
|
||||||
|
for command in commands:
|
||||||
|
try:
|
||||||
|
# Map old creator_id to new creator_id
|
||||||
|
if command['old_creator_id'] not in creator_id_mapping:
|
||||||
|
self.logger.warning(f"Skipping command '{command['name']}' - creator ID {command['old_creator_id']} not found in mapping")
|
||||||
|
skipped_count += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
new_creator_id = creator_id_mapping[command['old_creator_id']]
|
||||||
|
|
||||||
|
# Check if command already exists by name
|
||||||
|
cursor.execute("SELECT id FROM custom_commands WHERE name = ?", (command['name'],))
|
||||||
|
existing = cursor.fetchone()
|
||||||
|
|
||||||
|
if existing:
|
||||||
|
self.logger.warning(f"Command '{command['name']}' already exists with ID {existing[0]} - skipping")
|
||||||
|
skipped_count += 1
|
||||||
|
continue
|
||||||
|
|
||||||
|
# Determine if command was warned/inactive based on sent_warns
|
||||||
|
warning_sent = bool(command['sent_warns'] and command['sent_warns'] != 0)
|
||||||
|
|
||||||
|
# For migrated commands, ensure last_used is at least the migration date
|
||||||
|
# to prevent immediate deletion eligibility
|
||||||
|
migration_date = datetime.now().isoformat()
|
||||||
|
last_used = command['last_used']
|
||||||
|
|
||||||
|
# If command hasn't been used recently, set last_used to migration date
|
||||||
|
# to give it a grace period
|
||||||
|
if last_used:
|
||||||
|
try:
|
||||||
|
last_used_dt = datetime.fromisoformat(last_used.replace('Z', '+00:00'))
|
||||||
|
# If last used more than 60 days ago, update to migration date
|
||||||
|
if (datetime.now() - last_used_dt).days > 60:
|
||||||
|
last_used = migration_date
|
||||||
|
self.logger.info(f"Updated last_used for command '{command['name']}' to migration date")
|
||||||
|
except:
|
||||||
|
# If we can't parse the date, use migration date
|
||||||
|
last_used = migration_date
|
||||||
|
else:
|
||||||
|
# If no last_used date, use migration date
|
||||||
|
last_used = migration_date
|
||||||
|
|
||||||
|
# Add migration tag to indicate this is a migrated command
|
||||||
|
tags = '["migrated"]'
|
||||||
|
|
||||||
|
# Insert command
|
||||||
|
cursor.execute("""
|
||||||
|
INSERT INTO custom_commands
|
||||||
|
(name, content, creator_id, created_at, updated_at, last_used, use_count, warning_sent, is_active, tags)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?, 0, ?, 1, ?)
|
||||||
|
""", (
|
||||||
|
command['name'],
|
||||||
|
command['content'],
|
||||||
|
new_creator_id,
|
||||||
|
command['created_at'],
|
||||||
|
None, # updated_at
|
||||||
|
last_used,
|
||||||
|
warning_sent,
|
||||||
|
tags
|
||||||
|
))
|
||||||
|
|
||||||
|
migrated_count += 1
|
||||||
|
if migrated_count % 10 == 0:
|
||||||
|
self.logger.info(f"Migrated {migrated_count} commands...")
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
self.logger.error(f"Error migrating command {command}: {e}")
|
||||||
|
raise
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
self.logger.info(f"Successfully migrated {migrated_count} commands, skipped {skipped_count}")
|
||||||
|
|
||||||
|
def update_creator_stats(self) -> None:
|
||||||
|
"""Update creator statistics after migration"""
|
||||||
|
if self.dry_run:
|
||||||
|
self.logger.info("[DRY RUN] Would update creator statistics")
|
||||||
|
return
|
||||||
|
|
||||||
|
conn = sqlite3.connect(self.target_db)
|
||||||
|
cursor = conn.cursor()
|
||||||
|
|
||||||
|
# Update creator stats
|
||||||
|
cursor.execute("""
|
||||||
|
UPDATE custom_command_creators SET
|
||||||
|
total_commands = (
|
||||||
|
SELECT COUNT(*) FROM custom_commands
|
||||||
|
WHERE creator_id = custom_command_creators.id
|
||||||
|
),
|
||||||
|
active_commands = (
|
||||||
|
SELECT COUNT(*) FROM custom_commands
|
||||||
|
WHERE creator_id = custom_command_creators.id AND is_active = 1
|
||||||
|
)
|
||||||
|
""")
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
conn.close()
|
||||||
|
self.logger.info("Updated creator statistics")
|
||||||
|
|
||||||
|
def generate_migration_report(self) -> None:
|
||||||
|
"""Generate a detailed migration report"""
|
||||||
|
if self.dry_run:
|
||||||
|
conn_source = sqlite3.connect(self.source_db)
|
||||||
|
cursor_source = conn_source.cursor()
|
||||||
|
|
||||||
|
cursor_source.execute("SELECT COUNT(*) FROM creator")
|
||||||
|
source_creators = cursor_source.fetchone()[0]
|
||||||
|
|
||||||
|
cursor_source.execute("SELECT COUNT(*) FROM command")
|
||||||
|
source_commands = cursor_source.fetchone()[0]
|
||||||
|
|
||||||
|
conn_source.close()
|
||||||
|
|
||||||
|
self.logger.info(f"""
|
||||||
|
=== DRY RUN MIGRATION REPORT ===
|
||||||
|
Source Database: {self.source_db}
|
||||||
|
Target Database: {self.target_db}
|
||||||
|
|
||||||
|
Would migrate:
|
||||||
|
- {source_creators} creators
|
||||||
|
- {source_commands} commands
|
||||||
|
|
||||||
|
No actual changes made (dry run mode).
|
||||||
|
""".strip())
|
||||||
|
return
|
||||||
|
|
||||||
|
# Real migration report
|
||||||
|
conn_source = sqlite3.connect(self.source_db)
|
||||||
|
conn_target = sqlite3.connect(self.target_db)
|
||||||
|
|
||||||
|
cursor_source = conn_source.cursor()
|
||||||
|
cursor_target = conn_target.cursor()
|
||||||
|
|
||||||
|
# Source counts
|
||||||
|
cursor_source.execute("SELECT COUNT(*) FROM creator")
|
||||||
|
source_creators = cursor_source.fetchone()[0]
|
||||||
|
|
||||||
|
cursor_source.execute("SELECT COUNT(*) FROM command")
|
||||||
|
source_commands = cursor_source.fetchone()[0]
|
||||||
|
|
||||||
|
# Target counts
|
||||||
|
cursor_target.execute("SELECT COUNT(*) FROM custom_command_creators")
|
||||||
|
target_creators = cursor_target.fetchone()[0]
|
||||||
|
|
||||||
|
cursor_target.execute("SELECT COUNT(*) FROM custom_commands")
|
||||||
|
target_commands = cursor_target.fetchone()[0]
|
||||||
|
|
||||||
|
# Get sample of migrated data
|
||||||
|
cursor_target.execute("""
|
||||||
|
SELECT cc.name, cc.content, ccc.username
|
||||||
|
FROM custom_commands cc
|
||||||
|
JOIN custom_command_creators ccc ON cc.creator_id = ccc.id
|
||||||
|
LIMIT 5
|
||||||
|
""")
|
||||||
|
sample_commands = cursor_target.fetchall()
|
||||||
|
|
||||||
|
conn_source.close()
|
||||||
|
conn_target.close()
|
||||||
|
|
||||||
|
self.logger.info(f"""
|
||||||
|
=== MIGRATION REPORT ===
|
||||||
|
Source Database: {self.source_db}
|
||||||
|
Target Database: {self.target_db}
|
||||||
|
|
||||||
|
Migration Results:
|
||||||
|
- Source creators: {source_creators} -> Target creators: {target_creators}
|
||||||
|
- Source commands: {source_commands} -> Target commands: {target_commands}
|
||||||
|
|
||||||
|
Sample migrated commands:
|
||||||
|
""".strip())
|
||||||
|
|
||||||
|
for cmd in sample_commands:
|
||||||
|
self.logger.info(f" '{cmd[0]}' by {cmd[2]}: {cmd[1][:50]}...")
|
||||||
|
|
||||||
|
def run_migration(self) -> bool:
|
||||||
|
"""Execute the full migration process"""
|
||||||
|
self.logger.info(f"Starting custom commands migration {'(DRY RUN)' if self.dry_run else ''}")
|
||||||
|
self.logger.info(f"Source: {self.source_db}")
|
||||||
|
self.logger.info(f"Target: {self.target_db}")
|
||||||
|
|
||||||
|
try:
|
||||||
|
# Validate databases
|
||||||
|
if not self.validate_source_database():
|
||||||
|
return False
|
||||||
|
|
||||||
|
if not self.validate_target_database():
|
||||||
|
return False
|
||||||
|
|
||||||
|
# Load source data
|
||||||
|
creators, commands = self.load_source_data()
|
||||||
|
|
||||||
|
# Migrate creators first
|
||||||
|
creator_id_mapping = self.migrate_creators(creators)
|
||||||
|
|
||||||
|
# Migrate commands
|
||||||
|
self.migrate_commands(commands, creator_id_mapping)
|
||||||
|
|
||||||
|
# Update statistics
|
||||||
|
self.update_creator_stats()
|
||||||
|
|
||||||
|
# Generate report
|
||||||
|
self.generate_migration_report()
|
||||||
|
|
||||||
|
self.logger.info("Migration completed successfully!")
|
||||||
|
return True
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
self.logger.error(f"Migration failed: {e}")
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
parser = argparse.ArgumentParser(description='Migrate custom commands from old database to new schema')
|
||||||
|
parser.add_argument('--source', required=True, help='Path to source database (sba_is_fun.db)')
|
||||||
|
parser.add_argument('--target', required=True, help='Path to target database (sba_master.db)')
|
||||||
|
parser.add_argument('--dry-run', action='store_true', help='Run in dry-run mode (no actual changes)')
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
migrator = CustomCommandMigrator(args.source, args.target, args.dry_run)
|
||||||
|
success = migrator.run_migration()
|
||||||
|
|
||||||
|
exit(0 if success else 1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
Loading…
Reference in New Issue
Block a user