# Database Migrations

This folder contains SQL migration scripts for the membership platform.

## Running the Sprint 1-3 Migration

The `sprint_1_2_3_migration.sql` file adds all necessary columns and tables for the Members Only features (Sprints 1, 2, and 3).

### Prerequisites

- PostgreSQL installed and running
- Database created (e.g., `membership_db`)
- Database connection credentials from your `.env` file

### Option 1: Using psql command line

```bash
# Navigate to the migrations directory
cd /Users/andika/Documents/Works/Koncept\ Kit/KKN/membership-website/backend/migrations

# Run the migration (replace with your database credentials)
psql -U your_username -d membership_db -f sprint_1_2_3_migration.sql

# Or if you have a connection string
psql "postgresql://user:password@localhost:5432/membership_db" -f sprint_1_2_3_migration.sql
```

### Option 2: Using pgAdmin or another GUI tool

1. Open pgAdmin and connect to your database
2. Open the Query Tool
3. Load the `sprint_1_2_3_migration.sql` file
4. Execute the script

### Option 3: Using Python script

```bash
cd /Users/andika/Documents/Works/Koncept\ Kit/KKN/membership-website/backend

# Run the migration using Python
python3 -c "
import psycopg2
import os
from dotenv import load_dotenv

load_dotenv()
DATABASE_URL = os.getenv('DATABASE_URL')

conn = psycopg2.connect(DATABASE_URL)
cur = conn.cursor()

with open('migrations/sprint_1_2_3_migration.sql', 'r') as f:
    sql = f.read()
    cur.execute(sql)

conn.commit()
cur.close()
conn.close()
print('Migration completed successfully!')
"
```

### What Gets Added

**Users Table:**
- `profile_photo_url` - Stores Cloudflare R2 URL for profile photos
- `social_media_facebook` - Facebook profile/page URL
- `social_media_instagram` - Instagram handle or URL
- `social_media_twitter` - Twitter/X handle or URL
- `social_media_linkedin` - LinkedIn profile URL

**Events Table:**
- `microsoft_calendar_id` - Microsoft Calendar event ID for syncing
- `microsoft_calendar_sync_enabled` - Boolean flag for sync status

**New Tables:**
- `event_galleries` - Stores event photos with captions
- `newsletter_archives` - Stores newsletter documents
- `financial_reports` - Stores annual financial reports
- `bylaws_documents` - Stores organization bylaws
- `storage_usage` - Tracks Cloudflare R2 storage usage

### Verification

After running the migration, verify it worked:

```sql
-- Check users table columns
SELECT column_name, data_type
FROM information_schema.columns
WHERE table_name = 'users'
AND column_name IN ('profile_photo_url', 'social_media_facebook');

-- Check new tables exist
SELECT table_name
FROM information_schema.tables
WHERE table_name IN ('event_galleries', 'storage_usage');

-- Check storage_usage has initial record
SELECT * FROM storage_usage;
```

### Troubleshooting

**Error: "relation does not exist"**
- Make sure you're connected to the correct database
- Verify the `users` and `events` tables exist first

**Error: "column already exists"**
- This is safe to ignore - the script uses `IF NOT EXISTS` clauses

**Error: "permission denied"**
- Make sure your database user has ALTER TABLE privileges
- You may need to run as a superuser or database owner

### Rollback (if needed)

If you need to undo the migration:

```sql
-- Remove new columns from users
ALTER TABLE users DROP COLUMN IF EXISTS profile_photo_url;
ALTER TABLE users DROP COLUMN IF EXISTS social_media_facebook;
ALTER TABLE users DROP COLUMN IF EXISTS social_media_instagram;
ALTER TABLE users DROP COLUMN IF EXISTS social_media_twitter;
ALTER TABLE users DROP COLUMN IF EXISTS social_media_linkedin;

-- Remove new columns from events
ALTER TABLE events DROP COLUMN IF EXISTS microsoft_calendar_id;
ALTER TABLE events DROP COLUMN IF EXISTS microsoft_calendar_sync_enabled;

-- Remove new tables
DROP TABLE IF EXISTS event_galleries;
DROP TABLE IF EXISTS newsletter_archives;
DROP TABLE IF EXISTS financial_reports;
DROP TABLE IF EXISTS bylaws_documents;
DROP TABLE IF EXISTS storage_usage;
```

---

## Running Phase 1-4.5 Migrations (December 2025)

These migrations add features from client feedback phases 1-4.5:
- Member Since field for imported users
- Approval → Validation terminology update
- Terms of Service acceptance tracking
- Reminder email tracking for admin dashboard

### Quick Start

Run all migrations at once:

```bash
cd backend/migrations
psql $DATABASE_URL -f run_all_migrations.sql
```

### Individual Migration Files

The migrations are numbered in the order they should be run:

1. **001_add_member_since_field.sql** - Adds editable `member_since` field for imported users
2. **002_rename_approval_to_validation.sql** - Updates terminology from "approval" to "validation"
3. **003_add_tos_acceptance.sql** - Adds Terms of Service acceptance tracking
4. **004_add_reminder_tracking_fields.sql** - Adds reminder email tracking for admin dashboard

### Run Individual Migrations

```bash
cd backend/migrations

# Run migrations one by one
psql $DATABASE_URL -f 001_add_member_since_field.sql
psql $DATABASE_URL -f 002_rename_approval_to_validation.sql
psql $DATABASE_URL -f 003_add_tos_acceptance.sql
psql $DATABASE_URL -f 004_add_reminder_tracking_fields.sql
```

### Using psql Interactive Mode

```bash
# Connect to your database
psql $DATABASE_URL

# Inside psql, run:
\i backend/migrations/001_add_member_since_field.sql
\i backend/migrations/002_rename_approval_to_validation.sql
\i backend/migrations/003_add_tos_acceptance.sql
\i backend/migrations/004_add_reminder_tracking_fields.sql
```

### What Each Migration Adds

**Migration 001 - Member Since Field:**
- Adds `member_since` column (nullable timestamp)
- Backfills active members with their `created_at` date
- Allows admins to edit dates for imported users

**Migration 002 - Approval → Validation Terminology:**
- Updates `pending_approval` → `pending_validation`
- Updates `pre_approved` → `pre_validated`
- Aligns database with client's terminology requirements

**Migration 003 - ToS Acceptance:**
- Adds `accepts_tos` boolean field (default false)
- Adds `tos_accepted_at` timestamp field
- Backfills existing users as having accepted ToS

**Migration 004 - Reminder Tracking:**
- Adds 8 fields to track reminder emails:
  - `email_verification_reminders_sent` + `last_email_verification_reminder_at`
  - `event_attendance_reminders_sent` + `last_event_attendance_reminder_at`
  - `payment_reminders_sent` + `last_payment_reminder_at`
  - `renewal_reminders_sent` + `last_renewal_reminder_at`
- Enables admin dashboard to show users needing personal outreach

### Verification

After running migrations, verify they completed successfully:

```sql
-- Check if new columns exist
SELECT column_name, data_type
FROM information_schema.columns
WHERE table_name = 'users'
  AND column_name IN (
    'member_since',
    'accepts_tos',
    'tos_accepted_at',
    'email_verification_reminders_sent',
    'last_email_verification_reminder_at',
    'event_attendance_reminders_sent',
    'last_event_attendance_reminder_at',
    'payment_reminders_sent',
    'last_payment_reminder_at',
    'renewal_reminders_sent',
    'last_renewal_reminder_at'
  )
ORDER BY column_name;

-- Check status values were updated
SELECT status, COUNT(*)
FROM users
GROUP BY status;
```

### Rollback Phase 1-4.5 Migrations (If Needed)

```sql
-- Rollback 004: Remove reminder tracking fields
ALTER TABLE users
DROP COLUMN IF EXISTS email_verification_reminders_sent,
DROP COLUMN IF EXISTS last_email_verification_reminder_at,
DROP COLUMN IF EXISTS event_attendance_reminders_sent,
DROP COLUMN IF EXISTS last_event_attendance_reminder_at,
DROP COLUMN IF EXISTS payment_reminders_sent,
DROP COLUMN IF EXISTS last_payment_reminder_at,
DROP COLUMN IF EXISTS renewal_reminders_sent,
DROP COLUMN IF EXISTS last_renewal_reminder_at;

-- Rollback 003: Remove ToS fields
ALTER TABLE users
DROP COLUMN IF EXISTS accepts_tos,
DROP COLUMN IF EXISTS tos_accepted_at;

-- Rollback 002: Revert validation to approval
UPDATE users SET status = 'pending_approval' WHERE status = 'pending_validation';
UPDATE users SET status = 'pre_approved' WHERE status = 'pre_validated';

-- Rollback 001: Remove member_since field
ALTER TABLE users DROP COLUMN IF EXISTS member_since;
```

---

## Running Phase RBAC Migration (December 2025)

This migration adds RBAC permission management, user invitations, and CSV import tracking capabilities.

### Quick Start

```bash
cd backend/migrations
psql $DATABASE_URL -f 005_add_rbac_and_invitations.sql
```

### What This Migration Adds

**UserRole Enum Update:**
- Adds `superadmin` role to UserRole enum

**New Tables:**
1. **permissions** - Granular permission definitions (60+ permissions)
2. **role_permissions** - Junction table linking roles to permissions
3. **user_invitations** - Email-based invitation tracking with tokens
4. **import_jobs** - CSV import job tracking with error logging

**New Enum Types:**
- `invitationstatus` (pending, accepted, expired, revoked)
- `importjobstatus` (processing, completed, failed, partial)

### Verification

After running the migration, verify it completed successfully:

```sql
-- Check if superadmin role exists
SELECT enumlabel FROM pg_enum
WHERE enumtypid = 'userrole'::regtype
ORDER BY enumlabel;

-- Check if new tables exist
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name IN ('permissions', 'role_permissions', 'user_invitations', 'import_jobs')
ORDER BY table_name;

-- Check table structures
\d permissions
\d role_permissions
\d user_invitations
\d import_jobs
```

### Next Steps After Migration

1. **Seed Permissions**: Run `permissions_seed.py` to populate default permissions
2. **Upgrade Admin to Superadmin**: Update existing admin users to superadmin role
3. **Assign Permissions**: Configure permissions for admin, member, and guest roles

### Rollback (If Needed)

```sql
-- Remove all RBAC tables and enums
DROP TABLE IF EXISTS import_jobs CASCADE;
DROP TABLE IF EXISTS user_invitations CASCADE;
DROP TABLE IF EXISTS role_permissions CASCADE;
DROP TABLE IF EXISTS permissions CASCADE;
DROP TYPE IF EXISTS importjobstatus;
DROP TYPE IF EXISTS invitationstatus;

-- Note: Cannot remove 'superadmin' from UserRole enum without recreating
-- the entire enum. Only rollback if no users have the superadmin role.
```