347 lines
10 KiB
Markdown
347 lines
10 KiB
Markdown
# Database Migrations
|
|
|
|
This folder contains SQL migration scripts for the membership platform.
|
|
|
|
## Running the Sprint 1-3 Migration
|
|
|
|
The `sprint_1_2_3_migration.sql` file adds all necessary columns and tables for the Members Only features (Sprints 1, 2, and 3).
|
|
|
|
### Prerequisites
|
|
|
|
- PostgreSQL installed and running
|
|
- Database created (e.g., `membership_db`)
|
|
- Database connection credentials from your `.env` file
|
|
|
|
### Option 1: Using psql command line
|
|
|
|
```bash
|
|
# Navigate to the migrations directory
|
|
cd /Users/andika/Documents/Works/Koncept\ Kit/KKN/membership-website/backend/migrations
|
|
|
|
# Run the migration (replace with your database credentials)
|
|
psql -U your_username -d membership_db -f sprint_1_2_3_migration.sql
|
|
|
|
# Or if you have a connection string
|
|
psql "postgresql://user:password@localhost:5432/membership_db" -f sprint_1_2_3_migration.sql
|
|
```
|
|
|
|
### Option 2: Using pgAdmin or another GUI tool
|
|
|
|
1. Open pgAdmin and connect to your database
|
|
2. Open the Query Tool
|
|
3. Load the `sprint_1_2_3_migration.sql` file
|
|
4. Execute the script
|
|
|
|
### Option 3: Using Python script
|
|
|
|
```bash
|
|
cd /Users/andika/Documents/Works/Koncept\ Kit/KKN/membership-website/backend
|
|
|
|
# Run the migration using Python
|
|
python3 -c "
|
|
import psycopg2
|
|
import os
|
|
from dotenv import load_dotenv
|
|
|
|
load_dotenv()
|
|
DATABASE_URL = os.getenv('DATABASE_URL')
|
|
|
|
conn = psycopg2.connect(DATABASE_URL)
|
|
cur = conn.cursor()
|
|
|
|
with open('migrations/sprint_1_2_3_migration.sql', 'r') as f:
|
|
sql = f.read()
|
|
cur.execute(sql)
|
|
|
|
conn.commit()
|
|
cur.close()
|
|
conn.close()
|
|
print('Migration completed successfully!')
|
|
"
|
|
```
|
|
|
|
### What Gets Added
|
|
|
|
**Users Table:**
|
|
- `profile_photo_url` - Stores Cloudflare R2 URL for profile photos
|
|
- `social_media_facebook` - Facebook profile/page URL
|
|
- `social_media_instagram` - Instagram handle or URL
|
|
- `social_media_twitter` - Twitter/X handle or URL
|
|
- `social_media_linkedin` - LinkedIn profile URL
|
|
|
|
**Events Table:**
|
|
- `microsoft_calendar_id` - Microsoft Calendar event ID for syncing
|
|
- `microsoft_calendar_sync_enabled` - Boolean flag for sync status
|
|
|
|
**New Tables:**
|
|
- `event_galleries` - Stores event photos with captions
|
|
- `newsletter_archives` - Stores newsletter documents
|
|
- `financial_reports` - Stores annual financial reports
|
|
- `bylaws_documents` - Stores organization bylaws
|
|
- `storage_usage` - Tracks Cloudflare R2 storage usage
|
|
|
|
### Verification
|
|
|
|
After running the migration, verify it worked:
|
|
|
|
```sql
|
|
-- Check users table columns
|
|
SELECT column_name, data_type
|
|
FROM information_schema.columns
|
|
WHERE table_name = 'users'
|
|
AND column_name IN ('profile_photo_url', 'social_media_facebook');
|
|
|
|
-- Check new tables exist
|
|
SELECT table_name
|
|
FROM information_schema.tables
|
|
WHERE table_name IN ('event_galleries', 'storage_usage');
|
|
|
|
-- Check storage_usage has initial record
|
|
SELECT * FROM storage_usage;
|
|
```
|
|
|
|
### Troubleshooting
|
|
|
|
**Error: "relation does not exist"**
|
|
- Make sure you're connected to the correct database
|
|
- Verify the `users` and `events` tables exist first
|
|
|
|
**Error: "column already exists"**
|
|
- This is safe to ignore - the script uses `IF NOT EXISTS` clauses
|
|
|
|
**Error: "permission denied"**
|
|
- Make sure your database user has ALTER TABLE privileges
|
|
- You may need to run as a superuser or database owner
|
|
|
|
### Rollback (if needed)
|
|
|
|
If you need to undo the migration:
|
|
|
|
```sql
|
|
-- Remove new columns from users
|
|
ALTER TABLE users DROP COLUMN IF EXISTS profile_photo_url;
|
|
ALTER TABLE users DROP COLUMN IF EXISTS social_media_facebook;
|
|
ALTER TABLE users DROP COLUMN IF EXISTS social_media_instagram;
|
|
ALTER TABLE users DROP COLUMN IF EXISTS social_media_twitter;
|
|
ALTER TABLE users DROP COLUMN IF EXISTS social_media_linkedin;
|
|
|
|
-- Remove new columns from events
|
|
ALTER TABLE events DROP COLUMN IF EXISTS microsoft_calendar_id;
|
|
ALTER TABLE events DROP COLUMN IF EXISTS microsoft_calendar_sync_enabled;
|
|
|
|
-- Remove new tables
|
|
DROP TABLE IF EXISTS event_galleries;
|
|
DROP TABLE IF EXISTS newsletter_archives;
|
|
DROP TABLE IF EXISTS financial_reports;
|
|
DROP TABLE IF EXISTS bylaws_documents;
|
|
DROP TABLE IF EXISTS storage_usage;
|
|
```
|
|
|
|
---
|
|
|
|
## Running Phase 1-4.5 Migrations (December 2025)
|
|
|
|
These migrations add features from client feedback phases 1-4.5:
|
|
- Member Since field for imported users
|
|
- Approval → Validation terminology update
|
|
- Terms of Service acceptance tracking
|
|
- Reminder email tracking for admin dashboard
|
|
|
|
### Quick Start
|
|
|
|
Run all migrations at once:
|
|
|
|
```bash
|
|
cd backend/migrations
|
|
psql $DATABASE_URL -f run_all_migrations.sql
|
|
```
|
|
|
|
### Individual Migration Files
|
|
|
|
The migrations are numbered in the order they should be run:
|
|
|
|
1. **001_add_member_since_field.sql** - Adds editable `member_since` field for imported users
|
|
2. **002_rename_approval_to_validation.sql** - Updates terminology from "approval" to "validation"
|
|
3. **003_add_tos_acceptance.sql** - Adds Terms of Service acceptance tracking
|
|
4. **004_add_reminder_tracking_fields.sql** - Adds reminder email tracking for admin dashboard
|
|
|
|
### Run Individual Migrations
|
|
|
|
```bash
|
|
cd backend/migrations
|
|
|
|
# Run migrations one by one
|
|
psql $DATABASE_URL -f 001_add_member_since_field.sql
|
|
psql $DATABASE_URL -f 002_rename_approval_to_validation.sql
|
|
psql $DATABASE_URL -f 003_add_tos_acceptance.sql
|
|
psql $DATABASE_URL -f 004_add_reminder_tracking_fields.sql
|
|
```
|
|
|
|
### Using psql Interactive Mode
|
|
|
|
```bash
|
|
# Connect to your database
|
|
psql $DATABASE_URL
|
|
|
|
# Inside psql, run:
|
|
\i backend/migrations/001_add_member_since_field.sql
|
|
\i backend/migrations/002_rename_approval_to_validation.sql
|
|
\i backend/migrations/003_add_tos_acceptance.sql
|
|
\i backend/migrations/004_add_reminder_tracking_fields.sql
|
|
```
|
|
|
|
### What Each Migration Adds
|
|
|
|
**Migration 001 - Member Since Field:**
|
|
- Adds `member_since` column (nullable timestamp)
|
|
- Backfills active members with their `created_at` date
|
|
- Allows admins to edit dates for imported users
|
|
|
|
**Migration 002 - Approval → Validation Terminology:**
|
|
- Updates `pending_approval` → `pending_validation`
|
|
- Updates `pre_approved` → `pre_validated`
|
|
- Aligns database with client's terminology requirements
|
|
|
|
**Migration 003 - ToS Acceptance:**
|
|
- Adds `accepts_tos` boolean field (default false)
|
|
- Adds `tos_accepted_at` timestamp field
|
|
- Backfills existing users as having accepted ToS
|
|
|
|
**Migration 004 - Reminder Tracking:**
|
|
- Adds 8 fields to track reminder emails:
|
|
- `email_verification_reminders_sent` + `last_email_verification_reminder_at`
|
|
- `event_attendance_reminders_sent` + `last_event_attendance_reminder_at`
|
|
- `payment_reminders_sent` + `last_payment_reminder_at`
|
|
- `renewal_reminders_sent` + `last_renewal_reminder_at`
|
|
- Enables admin dashboard to show users needing personal outreach
|
|
|
|
### Verification
|
|
|
|
After running migrations, verify they completed successfully:
|
|
|
|
```sql
|
|
-- Check if new columns exist
|
|
SELECT column_name, data_type
|
|
FROM information_schema.columns
|
|
WHERE table_name = 'users'
|
|
AND column_name IN (
|
|
'member_since',
|
|
'accepts_tos',
|
|
'tos_accepted_at',
|
|
'email_verification_reminders_sent',
|
|
'last_email_verification_reminder_at',
|
|
'event_attendance_reminders_sent',
|
|
'last_event_attendance_reminder_at',
|
|
'payment_reminders_sent',
|
|
'last_payment_reminder_at',
|
|
'renewal_reminders_sent',
|
|
'last_renewal_reminder_at'
|
|
)
|
|
ORDER BY column_name;
|
|
|
|
-- Check status values were updated
|
|
SELECT status, COUNT(*)
|
|
FROM users
|
|
GROUP BY status;
|
|
```
|
|
|
|
### Rollback Phase 1-4.5 Migrations (If Needed)
|
|
|
|
```sql
|
|
-- Rollback 004: Remove reminder tracking fields
|
|
ALTER TABLE users
|
|
DROP COLUMN IF EXISTS email_verification_reminders_sent,
|
|
DROP COLUMN IF EXISTS last_email_verification_reminder_at,
|
|
DROP COLUMN IF EXISTS event_attendance_reminders_sent,
|
|
DROP COLUMN IF EXISTS last_event_attendance_reminder_at,
|
|
DROP COLUMN IF EXISTS payment_reminders_sent,
|
|
DROP COLUMN IF EXISTS last_payment_reminder_at,
|
|
DROP COLUMN IF EXISTS renewal_reminders_sent,
|
|
DROP COLUMN IF EXISTS last_renewal_reminder_at;
|
|
|
|
-- Rollback 003: Remove ToS fields
|
|
ALTER TABLE users
|
|
DROP COLUMN IF EXISTS accepts_tos,
|
|
DROP COLUMN IF EXISTS tos_accepted_at;
|
|
|
|
-- Rollback 002: Revert validation to approval
|
|
UPDATE users SET status = 'pending_approval' WHERE status = 'pending_validation';
|
|
UPDATE users SET status = 'pre_approved' WHERE status = 'pre_validated';
|
|
|
|
-- Rollback 001: Remove member_since field
|
|
ALTER TABLE users DROP COLUMN IF EXISTS member_since;
|
|
```
|
|
|
|
---
|
|
|
|
## Running Phase RBAC Migration (December 2025)
|
|
|
|
This migration adds RBAC permission management, user invitations, and CSV import tracking capabilities.
|
|
|
|
### Quick Start
|
|
|
|
```bash
|
|
cd backend/migrations
|
|
psql $DATABASE_URL -f 005_add_rbac_and_invitations.sql
|
|
```
|
|
|
|
### What This Migration Adds
|
|
|
|
**UserRole Enum Update:**
|
|
- Adds `superadmin` role to UserRole enum
|
|
|
|
**New Tables:**
|
|
1. **permissions** - Granular permission definitions (60+ permissions)
|
|
2. **role_permissions** - Junction table linking roles to permissions
|
|
3. **user_invitations** - Email-based invitation tracking with tokens
|
|
4. **import_jobs** - CSV import job tracking with error logging
|
|
|
|
**New Enum Types:**
|
|
- `invitationstatus` (pending, accepted, expired, revoked)
|
|
- `importjobstatus` (processing, completed, failed, partial)
|
|
|
|
### Verification
|
|
|
|
After running the migration, verify it completed successfully:
|
|
|
|
```sql
|
|
-- Check if superadmin role exists
|
|
SELECT enumlabel FROM pg_enum
|
|
WHERE enumtypid = 'userrole'::regtype
|
|
ORDER BY enumlabel;
|
|
|
|
-- Check if new tables exist
|
|
SELECT table_name FROM information_schema.tables
|
|
WHERE table_schema = 'public'
|
|
AND table_name IN ('permissions', 'role_permissions', 'user_invitations', 'import_jobs')
|
|
ORDER BY table_name;
|
|
|
|
-- Check table structures
|
|
\d permissions
|
|
\d role_permissions
|
|
\d user_invitations
|
|
\d import_jobs
|
|
```
|
|
|
|
### Next Steps After Migration
|
|
|
|
1. **Seed Permissions**: Run `permissions_seed.py` to populate default permissions
|
|
2. **Upgrade Admin to Superadmin**: Update existing admin users to superadmin role
|
|
3. **Assign Permissions**: Configure permissions for admin, member, and guest roles
|
|
|
|
### Rollback (If Needed)
|
|
|
|
```sql
|
|
-- Remove all RBAC tables and enums
|
|
DROP TABLE IF EXISTS import_jobs CASCADE;
|
|
DROP TABLE IF EXISTS user_invitations CASCADE;
|
|
DROP TABLE IF EXISTS role_permissions CASCADE;
|
|
DROP TABLE IF EXISTS permissions CASCADE;
|
|
DROP TYPE IF EXISTS importjobstatus;
|
|
DROP TYPE IF EXISTS invitationstatus;
|
|
|
|
-- Note: Cannot remove 'superadmin' from UserRole enum without recreating
|
|
-- the entire enum. Only rollback if no users have the superadmin role.
|
|
```
|
|
|