membership-be/migrations/README.md

Database Migrations

This folder contains SQL migration scripts for the membership platform.

Running the Sprint 1-3 Migration

The sprint_1_2_3_migration.sql file adds all necessary columns and tables for the Members Only features (Sprints 1, 2, and 3).

Prerequisites

• PostgreSQL installed and running
• Database created (e.g., membership_db)
• Database connection credentials from your .env file

Option 1: Using psql command line

# Navigate to the migrations directory
cd /Users/andika/Documents/Works/Koncept\ Kit/KKN/membership-website/backend/migrations

# Run the migration (replace with your database credentials)
psql -U your_username -d membership_db -f sprint_1_2_3_migration.sql

# Or if you have a connection string
psql "postgresql://user:password@localhost:5432/membership_db" -f sprint_1_2_3_migration.sql

Option 2: Using pgAdmin or another GUI tool

1. Open pgAdmin and connect to your database
2. Open the Query Tool
3. Load the sprint_1_2_3_migration.sql file
4. Execute the script

Option 3: Using Python script

cd /Users/andika/Documents/Works/Koncept\ Kit/KKN/membership-website/backend

# Run the migration using Python
python3 -c "
import psycopg2
import os
from dotenv import load_dotenv

load_dotenv()
DATABASE_URL = os.getenv('DATABASE_URL')

conn = psycopg2.connect(DATABASE_URL)
cur = conn.cursor()

with open('migrations/sprint_1_2_3_migration.sql', 'r') as f:
    sql = f.read()
    cur.execute(sql)

conn.commit()
cur.close()
conn.close()
print('Migration completed successfully!')
"

What Gets Added

Users Table:

• profile_photo_url - Stores Cloudflare R2 URL for profile photos
• social_media_facebook - Facebook profile/page URL
• social_media_instagram - Instagram handle or URL
• social_media_twitter - Twitter/X handle or URL
• social_media_linkedin - LinkedIn profile URL

Events Table:

• microsoft_calendar_id - Microsoft Calendar event ID for syncing
• microsoft_calendar_sync_enabled - Boolean flag for sync status

New Tables:

• event_galleries - Stores event photos with captions
• newsletter_archives - Stores newsletter documents
• financial_reports - Stores annual financial reports
• bylaws_documents - Stores organization bylaws
• storage_usage - Tracks Cloudflare R2 storage usage

Verification

After running the migration, verify it worked:

-- Check users table columns
SELECT column_name, data_type
FROM information_schema.columns
WHERE table_name = 'users'
AND column_name IN ('profile_photo_url', 'social_media_facebook');

-- Check new tables exist
SELECT table_name
FROM information_schema.tables
WHERE table_name IN ('event_galleries', 'storage_usage');

-- Check storage_usage has initial record
SELECT * FROM storage_usage;

Troubleshooting

Error: "relation does not exist"

• Make sure you're connected to the correct database
• Verify the users and events tables exist first

Error: "column already exists"

• This is safe to ignore - the script uses IF NOT EXISTS clauses

Error: "permission denied"

• Make sure your database user has ALTER TABLE privileges
• You may need to run as a superuser or database owner

Rollback (if needed)

If you need to undo the migration:

-- Remove new columns from users
ALTER TABLE users DROP COLUMN IF EXISTS profile_photo_url;
ALTER TABLE users DROP COLUMN IF EXISTS social_media_facebook;
ALTER TABLE users DROP COLUMN IF EXISTS social_media_instagram;
ALTER TABLE users DROP COLUMN IF EXISTS social_media_twitter;
ALTER TABLE users DROP COLUMN IF EXISTS social_media_linkedin;

-- Remove new columns from events
ALTER TABLE events DROP COLUMN IF EXISTS microsoft_calendar_id;
ALTER TABLE events DROP COLUMN IF EXISTS microsoft_calendar_sync_enabled;

-- Remove new tables
DROP TABLE IF EXISTS event_galleries;
DROP TABLE IF EXISTS newsletter_archives;
DROP TABLE IF EXISTS financial_reports;
DROP TABLE IF EXISTS bylaws_documents;
DROP TABLE IF EXISTS storage_usage;

---

Running Phase 1-4.5 Migrations (December 2025)

These migrations add features from client feedback phases 1-4.5:

• Member Since field for imported users
• Approval → Validation terminology update
• Terms of Service acceptance tracking
• Reminder email tracking for admin dashboard

Quick Start

Run all migrations at once:

cd backend/migrations
psql $DATABASE_URL -f run_all_migrations.sql

Individual Migration Files

The migrations are numbered in the order they should be run:

1. 001_add_member_since_field.sql - Adds editable member_since field for imported users
2. 002_rename_approval_to_validation.sql - Updates terminology from "approval" to "validation"
3. 003_add_tos_acceptance.sql - Adds Terms of Service acceptance tracking
4. 004_add_reminder_tracking_fields.sql - Adds reminder email tracking for admin dashboard

Run Individual Migrations

cd backend/migrations

# Run migrations one by one
psql $DATABASE_URL -f 001_add_member_since_field.sql
psql $DATABASE_URL -f 002_rename_approval_to_validation.sql
psql $DATABASE_URL -f 003_add_tos_acceptance.sql
psql $DATABASE_URL -f 004_add_reminder_tracking_fields.sql

Using psql Interactive Mode

# Connect to your database
psql $DATABASE_URL

# Inside psql, run:
\i backend/migrations/001_add_member_since_field.sql
\i backend/migrations/002_rename_approval_to_validation.sql
\i backend/migrations/003_add_tos_acceptance.sql
\i backend/migrations/004_add_reminder_tracking_fields.sql

What Each Migration Adds

Migration 001 - Member Since Field:

• Adds member_since column (nullable timestamp)
• Backfills active members with their created_at date
• Allows admins to edit dates for imported users

Migration 002 - Approval → Validation Terminology:

• Updates pending_approval → pending_validation
• Updates pre_approved → pre_validated
• Aligns database with client's terminology requirements

Migration 003 - ToS Acceptance:

• Adds accepts_tos boolean field (default false)
• Adds tos_accepted_at timestamp field
• Backfills existing users as having accepted ToS

Migration 004 - Reminder Tracking:

• Adds 8 fields to track reminder emails:
  • email_verification_reminders_sent + last_email_verification_reminder_at
  • event_attendance_reminders_sent + last_event_attendance_reminder_at
  • payment_reminders_sent + last_payment_reminder_at
  • renewal_reminders_sent + last_renewal_reminder_at
• Enables admin dashboard to show users needing personal outreach

Verification

After running migrations, verify they completed successfully:

-- Check if new columns exist
SELECT column_name, data_type
FROM information_schema.columns
WHERE table_name = 'users'
  AND column_name IN (
    'member_since',
    'accepts_tos',
    'tos_accepted_at',
    'email_verification_reminders_sent',
    'last_email_verification_reminder_at',
    'event_attendance_reminders_sent',
    'last_event_attendance_reminder_at',
    'payment_reminders_sent',
    'last_payment_reminder_at',
    'renewal_reminders_sent',
    'last_renewal_reminder_at'
  )
ORDER BY column_name;

-- Check status values were updated
SELECT status, COUNT(*)
FROM users
GROUP BY status;

Rollback Phase 1-4.5 Migrations (If Needed)

-- Rollback 004: Remove reminder tracking fields
ALTER TABLE users
DROP COLUMN IF EXISTS email_verification_reminders_sent,
DROP COLUMN IF EXISTS last_email_verification_reminder_at,
DROP COLUMN IF EXISTS event_attendance_reminders_sent,
DROP COLUMN IF EXISTS last_event_attendance_reminder_at,
DROP COLUMN IF EXISTS payment_reminders_sent,
DROP COLUMN IF EXISTS last_payment_reminder_at,
DROP COLUMN IF EXISTS renewal_reminders_sent,
DROP COLUMN IF EXISTS last_renewal_reminder_at;

-- Rollback 003: Remove ToS fields
ALTER TABLE users
DROP COLUMN IF EXISTS accepts_tos,
DROP COLUMN IF EXISTS tos_accepted_at;

-- Rollback 002: Revert validation to approval
UPDATE users SET status = 'pending_approval' WHERE status = 'pending_validation';
UPDATE users SET status = 'pre_approved' WHERE status = 'pre_validated';

-- Rollback 001: Remove member_since field
ALTER TABLE users DROP COLUMN IF EXISTS member_since;

---

Running Phase RBAC Migration (December 2025)

This migration adds RBAC permission management, user invitations, and CSV import tracking capabilities.

Quick Start

cd backend/migrations
psql $DATABASE_URL -f 005_add_rbac_and_invitations.sql

What This Migration Adds

UserRole Enum Update:

• Adds superadmin role to UserRole enum

New Tables:

1. permissions - Granular permission definitions (60+ permissions)
2. role_permissions - Junction table linking roles to permissions
3. user_invitations - Email-based invitation tracking with tokens
4. import_jobs - CSV import job tracking with error logging

New Enum Types:

• invitationstatus (pending, accepted, expired, revoked)
• importjobstatus (processing, completed, failed, partial)

Verification

After running the migration, verify it completed successfully:

-- Check if superadmin role exists
SELECT enumlabel FROM pg_enum
WHERE enumtypid = 'userrole'::regtype
ORDER BY enumlabel;

-- Check if new tables exist
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'
AND table_name IN ('permissions', 'role_permissions', 'user_invitations', 'import_jobs')
ORDER BY table_name;

-- Check table structures
\d permissions
\d role_permissions
\d user_invitations
\d import_jobs

Next Steps After Migration

1. Seed Permissions: Run permissions_seed.py to populate default permissions
2. Upgrade Admin to Superadmin: Update existing admin users to superadmin role
3. Assign Permissions: Configure permissions for admin, member, and guest roles

Rollback (If Needed)

-- Remove all RBAC tables and enums
DROP TABLE IF EXISTS import_jobs CASCADE;
DROP TABLE IF EXISTS user_invitations CASCADE;
DROP TABLE IF EXISTS role_permissions CASCADE;
DROP TABLE IF EXISTS permissions CASCADE;
DROP TYPE IF EXISTS importjobstatus;
DROP TYPE IF EXISTS invitationstatus;

-- Note: Cannot remove 'superadmin' from UserRole enum without recreating
-- the entire enum. Only rollback if no users have the superadmin role.