7 Commits

Author SHA1 Message Date
b3f1f5f789 Merge pull request 'Prod Deployment Preparation' (#4) from dev into loaf-prod
Reviewed-on: #4
2026-01-04 12:10:12 +00:00
Koncept Kit
6ec0745966 Prod Deployment Preparation 2026-01-04 19:08:54 +07:00
1da045f73f Merge pull request 'Update Gitignore' (#3) from dev into loaf-prod
Reviewed-on: #3
2026-01-02 08:45:29 +00:00
Koncept Kit
85199958bc Update Gitignore 2026-01-02 15:44:34 +07:00
Koncept Kit
487481b322 Test Preparation 2025-12-26 20:03:53 +07:00
fad23c6e57 Merge pull request 'Donation base URL fix' (#2) from main into dev
Reviewed-on: #2
2025-12-18 11:04:08 +00:00
3bcc69f3a2 Merge pull request '- Profile Picture\' (#1) from main into dev
Reviewed-on: #1
2025-12-18 10:30:00 +00:00
24 changed files with 3845 additions and 15 deletions

307
.gitignore vendored
View File

@@ -1 +1,306 @@
.env
# ============================================================================
# Python Backend .gitignore
# For FastAPI + PostgreSQL + Cloudflare R2 + Stripe
# ============================================================================
# ===== Environment Variables =====
.env
.env.*
!.env.example
.envrc
# ===== Python =====
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
*.manifest
*.spec
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff (if ever added):
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff (if ever added):
instance/
.webassets-cache
# Scrapy stuff (if ever added):
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
.python-version
# pipenv
Pipfile.lock
# poetry
poetry.lock
# pdm
.pdm.toml
# PEP 582
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/
# ===== Database =====
# SQLite (development)
*.db
*.sqlite
*.sqlite3
# PostgreSQL dumps
*.sql.gz
*.dump
# Database backups
backups/
*.backup
# ===== Alembic Migrations =====
# Keep migration files but ignore bytecode
alembic/__pycache__/
alembic/versions/__pycache__/
# Keep alembic.ini, env.py, and all migration files in alembic/versions/
# ===== IDE / Editors =====
# VSCode
.vscode/
*.code-workspace
# PyCharm
.idea/
*.iml
*.ipr
*.iws
# Sublime Text
*.sublime-project
*.sublime-workspace
# Vim
*.swp
*.swo
*~
.netrwhist
# Emacs
*~
\#*\#
/.emacs.desktop
/.emacs.desktop.lock
*.elc
# Eclipse
.project
.pydevproject
.settings/
# ===== Operating System =====
# macOS
.DS_Store
.AppleDouble
.LSOverride
._*
.DocumentRevisions-V100
.fseventsd
.Spotlight-V100
.TemporaryItems
.Trashes
.VolumeIcon.icns
.com.apple.timemachine.donotpresent
# Windows
Thumbs.db
Thumbs.db:encryptable
ehthumbs.db
ehthumbs_vista.db
*.stackdump
[Dd]esktop.ini
$RECYCLE.BIN/
*.cab
*.msi
*.msix
*.msm
*.msp
*.lnk
# Linux
.directory
.Trash-*
.nfs*
# ===== Logs & Runtime =====
*.log
logs/
*.out
*.err
*.pid
*.seed
*.pid.lock
# ===== Application-Specific =====
# Uploaded files (R2 storage handles this)
uploads/
temp_uploads/
tmp/
temporary/
# CSV imports
imports/*.csv
!imports/.gitkeep
# Generated reports
reports/
exports/
# Cache directories
.cache/
cache/
# ===== Security & Secrets =====
# API keys and secrets
secrets/
*.pem
*.key
*.cert
*.crt
*.p12
*.pfx
# Stripe webhook secrets
stripe_*.txt
# ===== Testing =====
# Test databases
test.db
test_*.db
# Test coverage
htmlcov/
.coverage
# ===== Miscellaneous =====
# Backup files
*.bak
*.backup
*.old
*.orig
# Compressed files
*.zip
*.tar.gz
*.rar
# Temporary files
*.tmp
*.temp
# Lock files
*.lock
!requirements.txt.lock
# ===== Keep These =====
# Keep these example/template files
!.env.example
!migrations/.gitkeep
!uploads/.gitkeep

247
DATABASE_STATUS.md Normal file
View File

@@ -0,0 +1,247 @@
# Database Status - LOAF Membership Platform
**Database:** `loaf_new`
**Host:** 10.9.23.11:54321
**Last Updated:** 2026-01-03
**Status:** ✅ Fully initialized with seed data
---
## Database Summary
### Tables (18 total)
| Table Name | Status | Records | Purpose |
|------------|--------|---------|---------|
| ✅ alembic_version | Active | 1 | Migration tracking (001_initial_baseline) |
| ✅ users | Active | 0 | User accounts and profiles |
| ✅ events | Active | 0 | Event management |
| ✅ event_rsvps | Active | 0 | Event RSVPs and attendance |
| ✅ event_galleries | Active | 0 | Event photo galleries |
| ✅ roles | Active | 5 | RBAC role definitions |
| ✅ permissions | Active | 25 | RBAC permission definitions |
| ✅ role_permissions | Active | 49 | Role-permission mappings |
| ✅ user_invitations | Active | 0 | Admin invitation system |
| ✅ subscriptions | Active | 0 | User subscriptions |
| ✅ subscription_plans | Active | 3 | Available membership plans |
| ✅ donations | Active | 0 | Donation tracking |
| ✅ import_jobs | Active | 0 | CSV import tracking |
| ✅ import_rollback_audit | Active | 0 | Import rollback audit trail |
| ✅ newsletter_archives | Active | 0 | Newsletter document archive |
| ✅ financial_reports | Active | 0 | Financial document archive |
| ✅ bylaws_documents | Active | 0 | Bylaws document archive |
| ✅ storage_usage | Active | 1 | Storage quota tracking (100GB limit) |
### ENUMs (8 total)
| ENUM Name | Values | Used By |
|-----------|--------|---------|
| ✅ userstatus | pending_email, awaiting_event, pre_approved, payment_pending, active, inactive | users.status |
| ✅ userrole | guest, member, admin, finance, superadmin | users.role, user_invitations.role |
| ✅ rsvpstatus | yes, no, maybe | event_rsvps.rsvp_status |
| ✅ subscriptionstatus | active, past_due, canceled, incomplete, trialing | subscriptions.status |
| ✅ donationtype | one_time, recurring, pledge, in_kind, memorial | donations.donation_type |
| ✅ donationstatus | pending, completed, failed, refunded | donations.status |
| ✅ invitationstatus | pending, accepted, expired, revoked | user_invitations.status |
| ✅ importjobstatus | processing, completed, failed | import_jobs.status |
---
## Seed Data Loaded
### Roles (5)
| Code | Name | System Role | Permissions |
|------|------|-------------|-------------|
| admin | Admin | Yes | 16 |
| finance | Finance | Yes | 7 |
| guest | Guest | Yes | 0 |
| member | Member | Yes | 1 |
| superadmin | Super Admin | Yes | 25 |
### Permissions (25 across 5 modules)
**Users Module (6 permissions):**
- users.view - View Users
- users.create - Create Users
- users.edit - Edit Users
- users.delete - Delete Users
- users.approve - Approve Users
- users.import - Import Users
**Events Module (6 permissions):**
- events.view - View Events
- events.create - Create Events
- events.edit - Edit Events
- events.delete - Delete Events
- events.publish - Publish Events
- events.manage_attendance - Manage Attendance
**Finance Module (5 permissions):**
- finance.view - View Financial Data
- finance.manage_plans - Manage Subscription Plans
- finance.manage_subscriptions - Manage Subscriptions
- finance.view_reports - View Financial Reports
- finance.export - Export Financial Data
**Content Module (3 permissions):**
- content.newsletters - Manage Newsletters
- content.documents - Manage Documents
- content.gallery - Manage Gallery
**System Module (5 permissions):**
- system.settings - System Settings
- system.roles - Manage Roles
- system.invitations - Manage Invitations
- system.storage - Manage Storage
- system.audit - View Audit Logs
### Subscription Plans (3)
| Plan Name | Price | Billing | Custom Pricing | Donation Support |
|-----------|-------|---------|----------------|------------------|
| Pay What You Want Membership | $30.00 (min) | Annual | ✅ Yes | ✅ Yes |
| Annual Individual Membership | $60.00 | Annual | ❌ No | ❌ No |
| Annual Group Membership | $100.00 | Annual | ❌ No | ❌ No |
**Note:** Stripe price IDs need to be configured after Stripe setup.
---
## Migration Status
**Current Revision:** `001_initial_baseline (head)`
**Migration System:** Alembic 1.14.0
**Schema Source:** `migrations/000_initial_schema.sql`
**Seed Source:** `migrations/seed_data.sql`
**Migration History:**
- `001_initial_baseline` - Empty baseline marker (2026-01-02)
**Future migrations** will be generated using:
```bash
alembic revision --autogenerate -m "description"
alembic upgrade head
```
---
## Next Steps
### Immediate (Required)
1. **Create Superadmin User**
```bash
cd backend
python3 create_superadmin.py
```
2. **Configure Stripe Price IDs**
```sql
UPDATE subscription_plans
SET stripe_price_id = 'price_xxx'
WHERE name = 'Annual Individual Membership';
UPDATE subscription_plans
SET stripe_price_id = 'price_yyy'
WHERE name = 'Annual Group Membership';
UPDATE subscription_plans
SET stripe_price_id = 'price_zzz'
WHERE name = 'Pay What You Want Membership';
```
3. **Set Environment Variables**
- Copy `backend/.env.example` to `backend/.env`
- Fill in all required values (DATABASE_URL, JWT_SECRET, SMTP, Stripe, R2)
4. **Test Application**
```bash
# Backend
cd backend
uvicorn server:app --reload
# Frontend (separate terminal)
cd frontend
yarn start
```
### Optional (Recommended)
1. **Add Sample Events**
- Login as superadmin
- Navigate to Admin → Events
- Create 2-3 sample events
2. **Test Registration Flow**
- Register a test user
- Verify email verification works
- Test event RSVP
- Test admin approval flow
3. **Configure Email Templates**
- Review templates in `backend/email_service.py`
- Customize colors, branding, copy
4. **Set Up Monitoring**
- Configure error logging
- Set up uptime monitoring
- Configure backup schedule
---
## Database Maintenance
### Backup Command
```bash
PGPASSWORD='your-password' pg_dump -h 10.9.23.11 -p 54321 -U postgres loaf_new > backup_$(date +%Y%m%d_%H%M%S).sql
```
### Restore Command
```bash
PGPASSWORD='your-password' psql -h 10.9.23.11 -p 54321 -U postgres -d loaf_new < backup_file.sql
```
### Health Check Queries
```sql
-- Check user count by status
SELECT status, COUNT(*) FROM users GROUP BY status;
-- Check upcoming events
SELECT title, start_at FROM events WHERE start_at > NOW() ORDER BY start_at LIMIT 5;
-- Check active subscriptions
SELECT COUNT(*) FROM subscriptions WHERE status = 'active';
-- Check storage usage
SELECT
total_bytes_used / 1024 / 1024 / 1024 as used_gb,
max_bytes_allowed / 1024 / 1024 / 1024 as max_gb,
ROUND((total_bytes_used::numeric / max_bytes_allowed * 100)::numeric, 2) as percent_used
FROM storage_usage;
```
---
## Support & Resources
- **Deployment Guide:** See `DEPLOYMENT.md` for complete deployment instructions
- **API Documentation:** http://localhost:8000/docs (when backend running)
- **Alembic Guide:** See `backend/alembic/README.md` for migration documentation
- **Project Documentation:** See `CLAUDE.md` for codebase overview
---
## Changelog
**2026-01-03:**
- ✅ Created all 17 data tables
- ✅ Created all 8 ENUMs
- ✅ Loaded seed data (5 roles, 25 permissions, 3 subscription plans)
- ✅ Initialized Alembic tracking (001_initial_baseline)
- ✅ Created superadmin user helper script
**Status:** Database is fully initialized and ready for use. Next step: Create superadmin user and start application.

379
DEPLOYMENT.md Normal file
View File

@@ -0,0 +1,379 @@
# Deployment Guide - LOAF Membership Platform
## Fresh Database Installation
Follow these steps in order for a **brand new deployment**:
### Step 1: Create PostgreSQL Database
```bash
# Connect to PostgreSQL
psql -U postgres
# Create database
CREATE DATABASE membership_db;
# Create user (if needed)
CREATE USER loaf_admin WITH PASSWORD 'your-secure-password';
GRANT ALL PRIVILEGES ON DATABASE membership_db TO loaf_admin;
# Exit PostgreSQL
\q
```
### Step 2: Run Initial Schema
```bash
cd backend
# Apply the complete schema (creates all 17 tables, 8 enums, indexes)
psql -U loaf_admin -d membership_db -f migrations/000_initial_schema.sql
```
**What this creates:**
- ✅ 17 tables: users, events, subscriptions, roles, permissions, etc.
- ✅ 8 custom enums: UserStatus, UserRole, RSVPStatus, etc.
- ✅ All indexes and foreign keys
- ✅ All constraints and defaults
### Step 3: Mark Database for Alembic Tracking
```bash
# Mark the database as being at the baseline
alembic stamp head
```
### Step 4: Verify Setup
```bash
# Check Alembic status
alembic current
# Expected output: 001_initial_baseline (head)
# Check database tables
psql -U loaf_admin -d membership_db -c "\dt"
# Should show 17 tables
```
### Step 5: Set Environment Variables
Create `backend/.env`:
```env
# Database
DATABASE_URL=postgresql://loaf_admin:your-password@localhost:5432/membership_db
# JWT
JWT_SECRET=your-secret-key-minimum-32-characters-long
JWT_ALGORITHM=HS256
ACCESS_TOKEN_EXPIRE_MINUTES=30
# Email (SMTP)
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USERNAME=your-email@gmail.com
SMTP_PASSWORD=your-app-password
SMTP_FROM_EMAIL=noreply@loafmembers.org
SMTP_FROM_NAME=LOAF Membership
# Frontend URL
FRONTEND_URL=https://members.loafmembers.org
# Cloudflare R2
R2_ENDPOINT_URL=https://your-account-id.r2.cloudflarestorage.com
R2_ACCESS_KEY_ID=your-r2-access-key
R2_SECRET_ACCESS_KEY=your-r2-secret-key
R2_BUCKET_NAME=loaf-membership
R2_PUBLIC_URL=https://cdn.loafmembers.org
# Stripe
STRIPE_SECRET_KEY=sk_live_...
STRIPE_WEBHOOK_SECRET=whsec_...
STRIPE_PRICE_ID_ANNUAL=price_...
STRIPE_PRICE_ID_GROUP=price_...
```
### Step 6: Install Dependencies
```bash
# Backend
cd backend
pip install -r requirements.txt
# Frontend
cd ../frontend
yarn install
```
### Step 7: Start Services
```bash
# Backend (in backend/)
uvicorn server:app --host 0.0.0.0 --port 8000
# Frontend (in frontend/)
yarn start
```
### Step 8: Create First Superadmin User
```bash
# Connect to database
psql -U loaf_admin -d membership_db
# Create superadmin user
INSERT INTO users (
id, email, password_hash, first_name, last_name,
status, role, email_verified, created_at, updated_at
) VALUES (
gen_random_uuid(),
'admin@loafmembers.org',
'$2b$12$your-bcrypt-hashed-password-here', -- Use bcrypt to hash password
'Admin',
'User',
'active',
'superadmin',
true,
NOW(),
NOW()
);
```
**Generate password hash:**
```python
import bcrypt
password = b"your-secure-password"
hashed = bcrypt.hashpw(password, bcrypt.gensalt())
print(hashed.decode())
```
---
## Existing Database Update
For **updating an existing deployment** with new code:
### Step 1: Backup Database
```bash
pg_dump -U loaf_admin membership_db > backup_$(date +%Y%m%d_%H%M%S).sql
```
### Step 2: Pull Latest Code
```bash
git pull origin main
```
### Step 3: Install New Dependencies
```bash
# Backend
cd backend
pip install -r requirements.txt
# Frontend
cd ../frontend
yarn install
```
### Step 4: Apply Database Migrations
```bash
cd backend
# Check current migration status
alembic current
# Apply pending migrations
alembic upgrade head
# Verify
alembic current
```
### Step 5: Restart Services
```bash
# Restart backend
systemctl restart membership-backend
# Rebuild and restart frontend
cd frontend
yarn build
systemctl restart membership-frontend
```
---
## First-Time Alembic Setup (Existing Database)
If you have an **existing database** that was created with `000_initial_schema.sql` but hasn't been marked for Alembic tracking:
```bash
cd backend
# Mark database as being at the baseline (one-time only)
alembic stamp head
# Verify
alembic current
# Expected output: 001_initial_baseline (head)
```
---
## Database Schema Verification
**Check all tables exist:**
```bash
psql -U loaf_admin -d membership_db -c "\dt"
```
**Expected tables (17 total):**
- users
- events
- event_rsvps
- subscriptions
- subscription_plans
- permissions
- roles
- role_permissions
- user_invitations
- import_jobs
- import_rollback_audit
- event_galleries
- newsletter_archives
- financial_reports
- bylaws_documents
- donations
- storage_usage
**Check enums:**
```bash
psql -U loaf_admin -d membership_db -c "\dT"
```
**Expected enums (8 total):**
- userstatus
- userrole
- rsvpstatus
- subscriptionstatus
- donationtype
- donationstatus
- invitationstatus
- importjobstatus
---
## Rollback Procedures
### Rollback Last Migration
```bash
cd backend
alembic downgrade -1
```
### Rollback to Specific Revision
```bash
alembic downgrade <revision_id>
```
### Complete Database Reset
```bash
# WARNING: This deletes ALL data!
# 1. Backup first
pg_dump -U loaf_admin membership_db > emergency_backup.sql
# 2. Drop database
dropdb membership_db
# 3. Recreate database
createdb membership_db
# 4. Run initial schema
psql -U loaf_admin -d membership_db -f backend/migrations/000_initial_schema.sql
# 5. Mark for Alembic
cd backend
alembic stamp head
```
---
## Troubleshooting
### "relation does not exist" error
The database wasn't initialized properly.
**Solution:**
```bash
psql -U loaf_admin -d membership_db -f backend/migrations/000_initial_schema.sql
alembic stamp head
```
### "Target database is not up to date"
Migrations haven't been applied.
**Solution:**
```bash
cd backend
alembic upgrade head
```
### "Can't locate revision"
Alembic tracking is out of sync.
**Solution:**
```bash
# Check what revision the database thinks it's at
alembic current
# If empty or wrong, manually set it
alembic stamp head
```
### Database connection errors
Check `.env` file has correct `DATABASE_URL`.
**Format:**
```
DATABASE_URL=postgresql://username:password@host:port/database
```
---
## Production Checklist
Before going live:
- [ ] Database created and schema applied
- [ ] Alembic marked as up-to-date (`alembic current` shows baseline)
- [ ] All environment variables set in `.env`
- [ ] Dependencies installed (Python + Node)
- [ ] Superadmin user created
- [ ] SSL certificates configured
- [ ] Backup system in place
- [ ] Monitoring configured
- [ ] Domain DNS pointing to server
- [ ] Email sending verified (SMTP working)
- [ ] Stripe webhook endpoint configured
- [ ] R2 bucket accessible and CORS configured
---
## Support
For issues:
1. Check logs: `tail -f backend/logs/app.log`
2. Check Alembic status: `alembic current`
3. Verify environment variables: `cat backend/.env`
4. Test database connection: `psql -U loaf_admin -d membership_db`

Binary file not shown.

Binary file not shown.

Binary file not shown.

118
alembic.ini Normal file
View File

@@ -0,0 +1,118 @@
# A generic, single database configuration.
[alembic]
# path to migration scripts
# Use forward slashes (/) also on windows to provide an os agnostic path
script_location = alembic
# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and time
# see https://alembic.sqlalchemy.org/en/latest/tutorial.html#editing-the-ini-file
# for all available tokens
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
# sys.path path, will be prepended to sys.path if present.
# defaults to the current working directory.
prepend_sys_path = .
# timezone to use when rendering the date within the migration file
# as well as the filename.
# If specified, requires the python>=3.9 or backports.zoneinfo library.
# Any required deps can installed by adding `alembic[tz]` to the pip requirements
# string value is passed to ZoneInfo()
# leave blank for localtime
# timezone =
# max length of characters to apply to the "slug" field
# truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; This defaults
# to alembic/versions. When using multiple version
# directories, initial revisions must be specified with --version-path.
# The path separator used here should be the separator specified by "version_path_separator" below.
# version_locations = %(here)s/bar:%(here)s/bat:alembic/versions
# version path separator; As mentioned above, this is the character used to split
# version_locations. The default within new alembic.ini files is "os", which uses os.pathsep.
# If this key is omitted entirely, it falls back to the legacy behavior of splitting on spaces and/or commas.
# Valid values for version_path_separator are:
#
# version_path_separator = :
# version_path_separator = ;
# version_path_separator = space
# version_path_separator = newline
version_path_separator = os # Use os.pathsep. Default configuration used for new projects.
# set to 'true' to search source files recursively
# in each "version_locations" directory
# new in Alembic version 1.10
# recursive_version_locations = false
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
# sqlalchemy.url = driver://user:pass@localhost/dbname
# Database URL is configured in alembic/env.py from .env file for security
[post_write_hooks]
# post_write_hooks defines scripts or Python functions that are run
# on newly generated revision scripts. See the documentation for further
# detail and examples
# format using "black" - use the console_scripts runner, against the "black" entrypoint
# hooks = black
# black.type = console_scripts
# black.entrypoint = black
# black.options = -l 79 REVISION_SCRIPT_FILENAME
# lint with attempts to fix using "ruff" - use the exec runner, execute a binary
# hooks = ruff
# ruff.type = exec
# ruff.executable = %(here)s/.venv/bin/ruff
# ruff.options = --fix REVISION_SCRIPT_FILENAME
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARNING
handlers = console
qualname =
[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

1
alembic/README Normal file
View File

@@ -0,0 +1 @@
Generic single-database configuration.

259
alembic/README.md Normal file
View File

@@ -0,0 +1,259 @@
# Alembic Database Migrations
This directory contains **Alembic** database migrations for the LOAF membership platform.
## What is Alembic?
Alembic is a lightweight database migration tool for SQLAlchemy. It allows you to:
- Track database schema changes over time
- Apply migrations incrementally
- Roll back changes if needed
- Auto-generate migration scripts from model changes
## Directory Structure
```
alembic/
├── versions/ # Migration scripts (KEEP IN VERSION CONTROL)
│ └── *.py # Individual migration files
├── env.py # Alembic environment configuration
├── script.py.mako # Template for new migration files
└── README.md # This file
```
## Quick Start
### 1. Create a New Migration
After making changes to `models.py`, generate a migration:
```bash
cd backend
alembic revision --autogenerate -m "add_user_bio_field"
```
This will create a new file in `alembic/versions/` like:
```
3e02c74581c9_add_user_bio_field.py
```
### 2. Review the Generated Migration
**IMPORTANT:** Always review auto-generated migrations before applying them!
```bash
# Open the latest migration file
cat alembic/versions/3e02c74581c9_add_user_bio_field.py
```
Check:
- ✅ The `upgrade()` function contains the correct changes
- ✅ The `downgrade()` function properly reverses those changes
- ✅ No unintended table drops or data loss
### 3. Apply the Migration
```bash
# Apply all pending migrations
alembic upgrade head
# Or apply migrations one at a time
alembic upgrade +1
```
### 4. Rollback a Migration
```bash
# Rollback the last migration
alembic downgrade -1
# Rollback to a specific revision
alembic downgrade 3e02c74581c9
```
## Common Commands
| Command | Description |
|---------|-------------|
| `alembic current` | Show current migration revision |
| `alembic history` | Show migration history |
| `alembic heads` | Show head revisions |
| `alembic upgrade head` | Apply all pending migrations |
| `alembic downgrade -1` | Rollback last migration |
| `alembic revision --autogenerate -m "message"` | Create new migration |
| `alembic stamp head` | Mark database as up-to-date without running migrations |
## Migration Workflow
### For Development
1. **Make changes to `models.py`**
```python
# In models.py
class User(Base):
# ...existing fields...
bio = Column(Text, nullable=True) # New field
```
2. **Generate migration**
```bash
alembic revision --autogenerate -m "add_user_bio_field"
```
3. **Review the generated file**
```python
# In alembic/versions/xxxxx_add_user_bio_field.py
def upgrade():
op.add_column('users', sa.Column('bio', sa.Text(), nullable=True))
def downgrade():
op.drop_column('users', 'bio')
```
4. **Apply migration**
```bash
alembic upgrade head
```
5. **Commit migration file to Git**
```bash
git add alembic/versions/xxxxx_add_user_bio_field.py
git commit -m "Add user bio field"
```
### For Production Deployment
**Fresh Database (New Installation):**
```bash
# 1. Create database
createdb membership_db
# 2. Run initial schema SQL (creates all 17 tables)
psql -U username -d membership_db -f ../migrations/000_initial_schema.sql
# 3. Mark database as up-to-date with Alembic
alembic stamp head
# 4. Verify
alembic current # Should show: 001_initial_baseline (head)
```
**Existing Database (Apply New Migrations):**
```bash
# 1. Pull latest code
git pull origin main
# 2. Apply migrations
alembic upgrade head
# 3. Verify
alembic current
# 4. Restart application
systemctl restart membership-backend
```
## Configuration
### Database Connection
Alembic reads the `DATABASE_URL` from your `.env` file:
```env
DATABASE_URL=postgresql://user:password@localhost:5432/membership_db
```
The connection is configured in `alembic/env.py` (lines 29-36).
### Target Metadata
Alembic uses `Base.metadata` from `models.py` to detect changes:
```python
# In alembic/env.py
from models import Base
target_metadata = Base.metadata
```
## Important Notes
### ✅ DO:
- Always review auto-generated migrations before applying
- Test migrations in development before production
- Commit migration files to version control
- Write descriptive migration messages
- Include both `upgrade()` and `downgrade()` functions
### ❌ DON'T:
- Don't edit migration files after they've been applied in production
- Don't delete migration files from `alembic/versions/`
- Don't modify the `revision` or `down_revision` values
- Don't commit `.pyc` files (already in .gitignore)
## Migration History
| Revision | Description | Date | Type |
|----------|-------------|------|------|
| `001_initial_baseline` | Baseline marker (empty migration) | 2026-01-02 | Baseline |
**Note:** The actual initial schema is created by running `backend/migrations/000_initial_schema.sql`. The baseline migration is an empty marker that indicates the starting point for Alembic tracking.
## Troubleshooting
### "Target database is not up to date"
```bash
# Check current revision
alembic current
# Check pending migrations
alembic history
# Apply missing migrations
alembic upgrade head
```
### "FAILED: Can't locate revision identified by 'xxxxx'"
The database thinks it's at a revision that doesn't exist in your `alembic/versions/`.
**Solution:**
```bash
# Mark database at a known good revision
alembic stamp head
```
### Migration conflicts
If you get merge conflicts in migration files:
1. Resolve conflicts in the migration file
2. Ensure `revision` and `down_revision` chain is correct
3. Test the migration locally
### Fresh database setup
For a completely new database:
```bash
# Step 1: Run initial schema SQL
psql -U username -d membership_db -f ../migrations/000_initial_schema.sql
# Step 2: Mark as up-to-date
alembic stamp head
# Step 3: Verify
alembic current # Should show: 001_initial_baseline (head)
```
## Legacy Migrations
Old numbered SQL migrations (`000_initial_schema.sql` through `011_wordpress_import_enhancements.sql`) are preserved in `backend/migrations/` for reference. These have been consolidated into the initial Alembic migration.
**Going forward, all new migrations must use Alembic.**
## Additional Resources
- [Alembic Documentation](https://alembic.sqlalchemy.org/)
- [SQLAlchemy Documentation](https://docs.sqlalchemy.org/)
- [PostgreSQL Documentation](https://www.postgresql.org/docs/)

96
alembic/env.py Normal file
View File

@@ -0,0 +1,96 @@
import os
import sys
from logging.config import fileConfig
from sqlalchemy import engine_from_config, pool
from alembic import context
from dotenv import load_dotenv
# Add the parent directory to the path so we can import our models
sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
# Load environment variables from .env file
load_dotenv()
# Import all models so Alembic can detect them
from models import Base
import models # This ensures all models are imported
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
# Interpret the config file for Python logging.
# This line sets up loggers basically.
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# Set the SQLAlchemy URL from environment variable
database_url = os.getenv("DATABASE_URL")
if database_url:
config.set_main_option("sqlalchemy.url", database_url)
else:
raise ValueError(
"DATABASE_URL environment variable not set. "
"Please create a .env file with DATABASE_URL=postgresql://user:password@host:port/dbname"
)
# Add your model's MetaData object here for 'autogenerate' support
target_metadata = Base.metadata
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
compare_type=True, # Detect type changes
compare_server_default=True, # Detect default value changes
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section, {}),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata,
compare_type=True, # Detect type changes
compare_server_default=True, # Detect default value changes
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

26
alembic/script.py.mako Normal file
View File

@@ -0,0 +1,26 @@
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
# revision identifiers, used by Alembic.
revision: str = ${repr(up_revision)}
down_revision: Union[str, None] = ${repr(down_revision)}
branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
def upgrade() -> None:
${upgrades if upgrades else "pass"}
def downgrade() -> None:
${downgrades if downgrades else "pass"}

View File

@@ -0,0 +1,59 @@
"""initial_baseline - Use 000_initial_schema.sql for fresh deployments
Revision ID: 001_initial_baseline
Revises:
Create Date: 2026-01-02 16:45:00.000000
IMPORTANT: This is a baseline migration for existing databases.
For FRESH deployments:
1. Run: psql -U user -d dbname -f backend/migrations/000_initial_schema.sql
2. Run: alembic stamp head
For EXISTING deployments (already have database):
1. Run: alembic stamp head (marks database as up-to-date)
This migration intentionally does NOTHING because:
- Fresh deployments use 000_initial_schema.sql to create all tables
- Existing deployments already have all tables from 000_initial_schema.sql
- Future migrations will be incremental changes from this baseline
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '001_initial_baseline'
down_revision: Union[str, None] = None
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""
This migration does nothing.
It serves as a baseline marker that indicates:
- All 17 tables exist (users, events, subscriptions, etc.)
- All 8 enums are defined (UserStatus, UserRole, etc.)
- All indexes and constraints are in place
The actual schema is created by running:
backend/migrations/000_initial_schema.sql
"""
pass
def downgrade() -> None:
"""
Cannot downgrade below baseline.
If you need to completely reset the database:
1. dropdb dbname
2. createdb dbname
3. psql -U user -d dbname -f backend/migrations/000_initial_schema.sql
4. alembic stamp head
"""
pass

105
create_superadmin.py Normal file
View File

@@ -0,0 +1,105 @@
#!/usr/bin/env python3
"""
Create Superadmin User Script
Generates a superadmin user with hashed password for LOAF membership platform
"""
import bcrypt
import sys
import os
from getpass import getpass
def generate_password_hash(password: str) -> str:
"""Generate bcrypt hash for password"""
return bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
def generate_sql(email: str, password_hash: str, first_name: str, last_name: str) -> str:
"""Generate SQL INSERT statement"""
return f"""
-- Create Superadmin User
INSERT INTO users (
id, email, password_hash, first_name, last_name,
status, role, email_verified, created_at, updated_at
) VALUES (
gen_random_uuid(),
'{email}',
'{password_hash}',
'{first_name}',
'{last_name}',
'active',
'superadmin',
true,
NOW(),
NOW()
);
"""
def main():
print("=" * 70)
print("LOAF Membership Platform - Superadmin User Creator")
print("=" * 70)
print()
# Get user input
email = input("Email address: ").strip()
if not email or '@' not in email:
print("❌ Invalid email address")
sys.exit(1)
first_name = input("First name: ").strip()
if not first_name:
print("❌ First name is required")
sys.exit(1)
last_name = input("Last name: ").strip()
if not last_name:
print("❌ Last name is required")
sys.exit(1)
# Get password securely
password = getpass("Password: ")
if len(password) < 8:
print("❌ Password must be at least 8 characters")
sys.exit(1)
password_confirm = getpass("Confirm password: ")
if password != password_confirm:
print("❌ Passwords do not match")
sys.exit(1)
print()
print("Generating password hash...")
password_hash = generate_password_hash(password)
print("✅ Password hash generated")
print()
print("=" * 70)
print("SQL STATEMENT")
print("=" * 70)
sql = generate_sql(email, password_hash, first_name, last_name)
print(sql)
# Save to file
output_file = "create_superadmin.sql"
with open(output_file, 'w') as f:
f.write(sql)
print("=" * 70)
print(f"✅ SQL saved to: {output_file}")
print()
print("Run this command to create the user:")
print(f" psql -U postgres -d loaf_new -f {output_file}")
print()
print("Or copy the SQL above and run it directly in psql")
print("=" * 70)
if __name__ == "__main__":
try:
main()
except KeyboardInterrupt:
print("\n\n❌ Cancelled by user")
sys.exit(1)
except Exception as e:
print(f"\n❌ Error: {e}")
sys.exit(1)

View File

@@ -77,7 +77,10 @@ CREATE TYPE importjobstatus AS ENUM (
'processing',
'completed',
'failed',
'partial'
'partial',
'validating',
'preview_ready',
'rolled_back'
);
COMMIT;
@@ -143,14 +146,25 @@ CREATE TABLE IF NOT EXISTS users (
-- Membership
member_since DATE,
tos_accepted BOOLEAN DEFAULT FALSE,
accepts_tos BOOLEAN DEFAULT FALSE,
tos_accepted_at TIMESTAMP WITH TIME ZONE,
newsletter_subscribed BOOLEAN DEFAULT TRUE,
-- Reminder Tracking
reminder_30_days_sent BOOLEAN DEFAULT FALSE,
reminder_60_days_sent BOOLEAN DEFAULT FALSE,
reminder_85_days_sent BOOLEAN DEFAULT FALSE,
-- Reminder Tracking (from migration 004)
email_verification_reminders_sent INTEGER DEFAULT 0 NOT NULL,
last_email_verification_reminder_at TIMESTAMP WITH TIME ZONE,
event_attendance_reminders_sent INTEGER DEFAULT 0 NOT NULL,
last_event_attendance_reminder_at TIMESTAMP WITH TIME ZONE,
payment_reminders_sent INTEGER DEFAULT 0 NOT NULL,
last_payment_reminder_at TIMESTAMP WITH TIME ZONE,
renewal_reminders_sent INTEGER DEFAULT 0 NOT NULL,
last_renewal_reminder_at TIMESTAMP WITH TIME ZONE,
-- WordPress Import Tracking
import_source VARCHAR(50),
import_job_id UUID REFERENCES import_jobs(id),
wordpress_user_id BIGINT,
wordpress_registered_date TIMESTAMP WITH TIME ZONE,
-- Timestamps
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
@@ -464,11 +478,30 @@ CREATE TABLE IF NOT EXISTS import_jobs (
error_count INTEGER DEFAULT 0,
error_log JSONB DEFAULT '[]'::jsonb,
-- WordPress import enhancements
field_mapping JSONB DEFAULT '{}'::jsonb,
wordpress_metadata JSONB DEFAULT '{}'::jsonb,
imported_user_ids JSONB DEFAULT '[]'::jsonb,
rollback_at TIMESTAMP WITH TIME ZONE,
rollback_by UUID REFERENCES users(id),
started_by UUID REFERENCES users(id),
started_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
completed_at TIMESTAMP WITH TIME ZONE
);
-- Import Rollback Audit table (for tracking rollback operations)
CREATE TABLE IF NOT EXISTS import_rollback_audit (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
import_job_id UUID NOT NULL REFERENCES import_jobs(id),
rolled_back_by UUID NOT NULL REFERENCES users(id),
rolled_back_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
deleted_user_count INTEGER NOT NULL,
deleted_user_ids JSONB NOT NULL,
reason TEXT,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
);
COMMIT;
-- Display progress
@@ -488,6 +521,8 @@ CREATE INDEX IF NOT EXISTS idx_users_role_id ON users(role_id);
CREATE INDEX IF NOT EXISTS idx_users_email_verified ON users(email_verified);
CREATE INDEX IF NOT EXISTS idx_users_rejected_at ON users(rejected_at) WHERE rejected_at IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_users_created_at ON users(created_at);
CREATE INDEX IF NOT EXISTS idx_users_import_job ON users(import_job_id) WHERE import_job_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_users_import_source ON users(import_source) WHERE import_source IS NOT NULL;
-- Events table indexes
CREATE INDEX IF NOT EXISTS idx_events_created_by ON events(created_by);
@@ -514,6 +549,14 @@ CREATE INDEX IF NOT EXISTS idx_donation_type ON donations(donation_type);
CREATE INDEX IF NOT EXISTS idx_donation_status ON donations(status);
CREATE INDEX IF NOT EXISTS idx_donation_created ON donations(created_at);
-- Import Jobs indexes
CREATE INDEX IF NOT EXISTS idx_import_jobs_status ON import_jobs(status);
CREATE INDEX IF NOT EXISTS idx_import_jobs_started_by ON import_jobs(started_by);
-- Import Rollback Audit indexes
CREATE INDEX IF NOT EXISTS idx_rollback_audit_import_job ON import_rollback_audit(import_job_id);
CREATE INDEX IF NOT EXISTS idx_rollback_audit_rolled_back_at ON import_rollback_audit(rolled_back_at DESC);
-- Permissions indexes
CREATE INDEX IF NOT EXISTS idx_permissions_code ON permissions(code);
CREATE INDEX IF NOT EXISTS idx_permissions_module ON permissions(module);

View File

@@ -0,0 +1,153 @@
-- Migration: 011_wordpress_import_enhancements
-- Purpose: Enhance ImportJob and User tables for WordPress CSV import feature
-- Date: 2025-12-24
-- Author: Claude Code
-- ============================================================================
-- PART 1: Enhance ImportJob Table
-- ============================================================================
-- Add new columns to import_jobs table for WordPress import tracking
ALTER TABLE import_jobs
ADD COLUMN IF NOT EXISTS field_mapping JSONB DEFAULT '{}'::jsonb,
ADD COLUMN IF NOT EXISTS wordpress_metadata JSONB DEFAULT '{}'::jsonb,
ADD COLUMN IF NOT EXISTS imported_user_ids JSONB DEFAULT '[]'::jsonb,
ADD COLUMN IF NOT EXISTS rollback_at TIMESTAMP WITH TIME ZONE,
ADD COLUMN IF NOT EXISTS rollback_by UUID REFERENCES users(id);
-- Add comments for documentation
COMMENT ON COLUMN import_jobs.field_mapping IS 'Maps CSV columns to database fields: {csv_column: db_field}';
COMMENT ON COLUMN import_jobs.wordpress_metadata IS 'Stores preview data, validation results, and WordPress-specific metadata';
COMMENT ON COLUMN import_jobs.imported_user_ids IS 'Array of user IDs created from this import job (for rollback)';
COMMENT ON COLUMN import_jobs.rollback_at IS 'Timestamp when this import was rolled back';
COMMENT ON COLUMN import_jobs.rollback_by IS 'Admin user who performed the rollback';
-- ============================================================================
-- PART 2: Add New ImportJob Status Values
-- ============================================================================
-- Add new status values for import workflow
-- Note: PostgreSQL enum values cannot be added conditionally, so we use DO block
DO $$
BEGIN
-- Add 'validating' status if it doesn't exist
IF NOT EXISTS (SELECT 1 FROM pg_enum WHERE enumlabel = 'validating' AND enumtypid = (SELECT oid FROM pg_type WHERE typname = 'importjobstatus')) THEN
ALTER TYPE importjobstatus ADD VALUE 'validating';
END IF;
-- Add 'preview_ready' status if it doesn't exist
IF NOT EXISTS (SELECT 1 FROM pg_enum WHERE enumlabel = 'preview_ready' AND enumtypid = (SELECT oid FROM pg_type WHERE typname = 'importjobstatus')) THEN
ALTER TYPE importjobstatus ADD VALUE 'preview_ready';
END IF;
-- Add 'rolled_back' status if it doesn't exist
IF NOT EXISTS (SELECT 1 FROM pg_enum WHERE enumlabel = 'rolled_back' AND enumtypid = (SELECT oid FROM pg_type WHERE typname = 'importjobstatus')) THEN
ALTER TYPE importjobstatus ADD VALUE 'rolled_back';
END IF;
END$$;
-- ============================================================================
-- PART 3: Enhance User Table for Import Tracking
-- ============================================================================
-- Add columns to track import source and WordPress metadata
ALTER TABLE users
ADD COLUMN IF NOT EXISTS import_source VARCHAR(50),
ADD COLUMN IF NOT EXISTS import_job_id UUID REFERENCES import_jobs(id),
ADD COLUMN IF NOT EXISTS wordpress_user_id BIGINT,
ADD COLUMN IF NOT EXISTS wordpress_registered_date TIMESTAMP WITH TIME ZONE;
-- Add comments for documentation
COMMENT ON COLUMN users.import_source IS 'Source of user creation: wordpress, manual, registration, etc.';
COMMENT ON COLUMN users.import_job_id IS 'Reference to import job that created this user (if imported)';
COMMENT ON COLUMN users.wordpress_user_id IS 'Original WordPress user ID for reference';
COMMENT ON COLUMN users.wordpress_registered_date IS 'Original WordPress registration date';
-- ============================================================================
-- PART 4: Create Indexes for Performance
-- ============================================================================
-- Index for querying users by import job (used in rollback)
CREATE INDEX IF NOT EXISTS idx_users_import_job
ON users(import_job_id)
WHERE import_job_id IS NOT NULL;
-- Index for querying users by import source
CREATE INDEX IF NOT EXISTS idx_users_import_source
ON users(import_source)
WHERE import_source IS NOT NULL;
-- Index for querying import jobs by status
CREATE INDEX IF NOT EXISTS idx_import_jobs_status
ON import_jobs(status);
-- ============================================================================
-- PART 5: Create Rollback Audit Table (Optional but Recommended)
-- ============================================================================
-- Create table to track import rollback history for audit purposes
CREATE TABLE IF NOT EXISTS import_rollback_audit (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
import_job_id UUID NOT NULL REFERENCES import_jobs(id),
rolled_back_by UUID NOT NULL REFERENCES users(id),
rolled_back_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
deleted_user_count INTEGER NOT NULL,
deleted_user_ids JSONB NOT NULL,
reason TEXT,
created_at TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW()
);
-- Index for querying rollback history
CREATE INDEX IF NOT EXISTS idx_rollback_audit_import_job
ON import_rollback_audit(import_job_id);
CREATE INDEX IF NOT EXISTS idx_rollback_audit_rolled_back_at
ON import_rollback_audit(rolled_back_at DESC);
COMMENT ON TABLE import_rollback_audit IS 'Audit trail for import rollback operations';
-- ============================================================================
-- VERIFICATION QUERIES (Run after migration to verify)
-- ============================================================================
-- Verify ImportJob columns exist
-- SELECT column_name, data_type
-- FROM information_schema.columns
-- WHERE table_name = 'import_jobs'
-- AND column_name IN ('field_mapping', 'wordpress_metadata', 'imported_user_ids', 'rollback_at', 'rollback_by');
-- Verify User columns exist
-- SELECT column_name, data_type
-- FROM information_schema.columns
-- WHERE table_name = 'users'
-- AND column_name IN ('import_source', 'import_job_id', 'wordpress_user_id', 'wordpress_registered_date');
-- Verify new enum values exist
-- SELECT enumlabel FROM pg_enum WHERE enumtypid = (SELECT oid FROM pg_type WHERE typname = 'importjobstatus') ORDER BY enumlabel;
-- Verify indexes exist
-- SELECT indexname, indexdef FROM pg_indexes WHERE tablename IN ('users', 'import_jobs', 'import_rollback_audit') ORDER BY indexname;
-- ============================================================================
-- ROLLBACK SCRIPT (if needed)
-- ============================================================================
-- WARNING: This will drop all columns and data related to WordPress imports
-- USE WITH EXTREME CAUTION
-- DROP TABLE IF EXISTS import_rollback_audit CASCADE;
-- DROP INDEX IF EXISTS idx_users_import_job;
-- DROP INDEX IF EXISTS idx_users_import_source;
-- DROP INDEX IF EXISTS idx_import_jobs_status;
-- ALTER TABLE users DROP COLUMN IF EXISTS import_source;
-- ALTER TABLE users DROP COLUMN IF EXISTS import_job_id;
-- ALTER TABLE users DROP COLUMN IF EXISTS wordpress_user_id;
-- ALTER TABLE users DROP COLUMN IF EXISTS wordpress_registered_date;
-- ALTER TABLE import_jobs DROP COLUMN IF EXISTS field_mapping;
-- ALTER TABLE import_jobs DROP COLUMN IF EXISTS wordpress_metadata;
-- ALTER TABLE import_jobs DROP COLUMN IF EXISTS imported_user_ids;
-- ALTER TABLE import_jobs DROP COLUMN IF EXISTS rollback_at;
-- ALTER TABLE import_jobs DROP COLUMN IF EXISTS rollback_by;
-- Note: Cannot easily remove enum values from importjobstatus type without recreating it
-- Manual intervention required if rollback of enum values is needed

View File

@@ -0,0 +1,394 @@
-- ============================================================================
-- Create Tables Only (ENUMs already exist)
-- Run this when ENUMs exist but tables don't
-- ============================================================================
BEGIN;
-- ============================================================================
-- STEP 1: Core Tables
-- ============================================================================
-- Users table
CREATE TABLE IF NOT EXISTS users (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
email VARCHAR(255) UNIQUE NOT NULL,
password_hash VARCHAR(255) NOT NULL,
first_name VARCHAR(100) NOT NULL,
last_name VARCHAR(100) NOT NULL,
phone VARCHAR(20),
address TEXT,
city VARCHAR(100),
state VARCHAR(2),
zipcode VARCHAR(10),
date_of_birth DATE,
-- Profile
profile_image_url TEXT,
bio TEXT,
interests TEXT,
-- Partner Information
partner_first_name VARCHAR(100),
partner_last_name VARCHAR(100),
partner_is_member BOOLEAN DEFAULT FALSE,
partner_plan_to_become_member BOOLEAN DEFAULT FALSE,
-- Referral
referred_by_member_name VARCHAR(200),
-- Newsletter Preferences
newsletter_subscribed BOOLEAN DEFAULT TRUE,
newsletter_publish_name BOOLEAN DEFAULT FALSE,
newsletter_publish_photo BOOLEAN DEFAULT FALSE,
newsletter_publish_birthday BOOLEAN DEFAULT FALSE,
newsletter_publish_none BOOLEAN DEFAULT FALSE,
-- Volunteer & Scholarship
volunteer_interests TEXT,
scholarship_requested BOOLEAN DEFAULT FALSE,
-- Directory
show_in_directory BOOLEAN DEFAULT TRUE,
-- Lead Sources (JSON array)
lead_sources JSONB DEFAULT '[]'::jsonb,
-- Status & Role
status userstatus DEFAULT 'pending_email' NOT NULL,
role userrole DEFAULT 'guest' NOT NULL,
role_id UUID,
-- Rejection Tracking
rejection_reason TEXT,
rejected_at TIMESTAMP WITH TIME ZONE,
rejected_by UUID REFERENCES users(id),
-- Membership
member_since DATE,
accepts_tos BOOLEAN DEFAULT FALSE,
tos_accepted_at TIMESTAMP WITH TIME ZONE,
-- Reminder Tracking (from migration 004)
email_verification_reminders_sent INTEGER DEFAULT 0 NOT NULL,
last_email_verification_reminder_at TIMESTAMP WITH TIME ZONE,
event_attendance_reminders_sent INTEGER DEFAULT 0 NOT NULL,
last_event_attendance_reminder_at TIMESTAMP WITH TIME ZONE,
payment_reminders_sent INTEGER DEFAULT 0 NOT NULL,
last_payment_reminder_at TIMESTAMP WITH TIME ZONE,
renewal_reminders_sent INTEGER DEFAULT 0 NOT NULL,
last_renewal_reminder_at TIMESTAMP WITH TIME ZONE,
-- WordPress Import Tracking
import_source VARCHAR(50),
import_job_id UUID,
wordpress_user_id BIGINT,
wordpress_registered_date TIMESTAMP WITH TIME ZONE,
-- Authentication
email_verified BOOLEAN DEFAULT FALSE,
email_verification_token VARCHAR(255),
email_verification_expires TIMESTAMP WITH TIME ZONE,
password_reset_token VARCHAR(255),
password_reset_expires TIMESTAMP WITH TIME ZONE,
force_password_change BOOLEAN DEFAULT FALSE,
-- Timestamps
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Events table
CREATE TABLE IF NOT EXISTS events (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
title VARCHAR(255) NOT NULL,
description TEXT,
location VARCHAR(255),
start_at TIMESTAMP WITH TIME ZONE NOT NULL,
end_at TIMESTAMP WITH TIME ZONE NOT NULL,
capacity INTEGER,
published BOOLEAN DEFAULT FALSE,
calendar_uid VARCHAR(255) UNIQUE,
created_by UUID REFERENCES users(id),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Event RSVPs
CREATE TABLE IF NOT EXISTS event_rsvps (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
event_id UUID NOT NULL REFERENCES events(id) ON DELETE CASCADE,
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
rsvp_status rsvpstatus NOT NULL,
attended BOOLEAN DEFAULT FALSE,
attended_at TIMESTAMP WITH TIME ZONE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
UNIQUE(event_id, user_id)
);
-- Event Gallery
CREATE TABLE IF NOT EXISTS event_galleries (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
event_id UUID NOT NULL REFERENCES events(id) ON DELETE CASCADE,
image_url TEXT NOT NULL,
caption TEXT,
uploaded_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Import Jobs
CREATE TABLE IF NOT EXISTS import_jobs (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
filename VARCHAR(255) NOT NULL,
file_key VARCHAR(255),
total_rows INTEGER NOT NULL,
processed_rows INTEGER DEFAULT 0,
successful_rows INTEGER DEFAULT 0,
failed_rows INTEGER DEFAULT 0,
status importjobstatus DEFAULT 'processing' NOT NULL,
errors JSONB DEFAULT '[]'::jsonb,
-- WordPress import enhancements
field_mapping JSONB DEFAULT '{}'::jsonb,
wordpress_metadata JSONB DEFAULT '{}'::jsonb,
imported_user_ids JSONB DEFAULT '[]'::jsonb,
rollback_at TIMESTAMP WITH TIME ZONE,
rollback_by UUID REFERENCES users(id),
imported_by UUID NOT NULL REFERENCES users(id),
started_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
completed_at TIMESTAMP WITH TIME ZONE
);
COMMIT;
-- ============================================================================
-- STEP 2: Subscription & Payment Tables
-- ============================================================================
BEGIN;
CREATE TABLE IF NOT EXISTS subscription_plans (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
name VARCHAR(100) NOT NULL,
description TEXT,
price_cents INTEGER NOT NULL,
billing_cycle VARCHAR(20) NOT NULL,
stripe_price_id VARCHAR(255),
custom_cycle_enabled BOOLEAN DEFAULT FALSE,
minimum_price_cents INTEGER DEFAULT 0,
allow_donation BOOLEAN DEFAULT FALSE,
is_active BOOLEAN DEFAULT TRUE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS subscriptions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
plan_id UUID NOT NULL REFERENCES subscription_plans(id),
stripe_subscription_id VARCHAR(255),
stripe_customer_id VARCHAR(255),
base_subscription_cents INTEGER NOT NULL,
donation_cents INTEGER DEFAULT 0,
status subscriptionstatus DEFAULT 'active' NOT NULL,
current_period_start TIMESTAMP WITH TIME ZONE,
current_period_end TIMESTAMP WITH TIME ZONE,
cancel_at_period_end BOOLEAN DEFAULT FALSE,
canceled_at TIMESTAMP WITH TIME ZONE,
manual_payment BOOLEAN DEFAULT FALSE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS donations (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
user_id UUID REFERENCES users(id),
amount_cents INTEGER NOT NULL,
donation_type donationtype NOT NULL,
status donationstatus DEFAULT 'pending' NOT NULL,
stripe_payment_intent_id VARCHAR(255),
donor_name VARCHAR(200),
donor_email VARCHAR(255),
message TEXT,
is_anonymous BOOLEAN DEFAULT FALSE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
completed_at TIMESTAMP WITH TIME ZONE
);
COMMIT;
-- ============================================================================
-- STEP 3: RBAC Tables
-- ============================================================================
BEGIN;
CREATE TABLE IF NOT EXISTS permissions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
code VARCHAR(100) UNIQUE NOT NULL,
name VARCHAR(200) NOT NULL,
description TEXT,
module VARCHAR(50),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS roles (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
code VARCHAR(50) UNIQUE NOT NULL,
name VARCHAR(100) NOT NULL,
description TEXT,
is_system_role BOOLEAN DEFAULT FALSE,
created_by UUID REFERENCES users(id),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS role_permissions (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
role VARCHAR(50),
role_id UUID REFERENCES roles(id) ON DELETE CASCADE,
permission_id UUID NOT NULL REFERENCES permissions(id) ON DELETE CASCADE,
created_by UUID REFERENCES users(id),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS user_invitations (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
email VARCHAR(255) NOT NULL,
role userrole NOT NULL,
token VARCHAR(255) UNIQUE NOT NULL,
invited_by UUID NOT NULL REFERENCES users(id),
invited_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
expires_at TIMESTAMP WITH TIME ZONE NOT NULL,
accepted_by UUID REFERENCES users(id),
accepted_at TIMESTAMP WITH TIME ZONE,
status invitationstatus DEFAULT 'pending' NOT NULL
);
COMMIT;
-- ============================================================================
-- STEP 4: Document Management Tables
-- ============================================================================
BEGIN;
CREATE TABLE IF NOT EXISTS newsletter_archives (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
title VARCHAR(255) NOT NULL,
description TEXT,
published_date TIMESTAMP WITH TIME ZONE NOT NULL,
document_url TEXT NOT NULL,
document_type VARCHAR(50) NOT NULL,
created_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS financial_reports (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
year INTEGER NOT NULL,
title VARCHAR(255) NOT NULL,
document_url TEXT NOT NULL,
document_type VARCHAR(50) NOT NULL,
created_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS bylaws_documents (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
title VARCHAR(255) NOT NULL,
version VARCHAR(50) NOT NULL,
effective_date TIMESTAMP WITH TIME ZONE NOT NULL,
document_url TEXT NOT NULL,
document_type VARCHAR(50) NOT NULL,
is_current BOOLEAN DEFAULT FALSE,
created_by UUID NOT NULL REFERENCES users(id),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
COMMIT;
-- ============================================================================
-- STEP 5: System Tables
-- ============================================================================
BEGIN;
CREATE TABLE IF NOT EXISTS storage_usage (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
total_bytes_used BIGINT DEFAULT 0,
max_bytes_allowed BIGINT,
last_calculated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
CREATE TABLE IF NOT EXISTS import_rollback_audit (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
import_job_id UUID NOT NULL REFERENCES import_jobs(id),
rolled_back_by UUID NOT NULL REFERENCES users(id),
rolled_back_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
deleted_user_count INTEGER NOT NULL,
deleted_user_ids JSONB NOT NULL,
reason TEXT,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Initialize storage_usage with default row
INSERT INTO storage_usage (id, total_bytes_used, max_bytes_allowed)
VALUES (gen_random_uuid(), 0, 107374182400) -- 100GB limit
ON CONFLICT DO NOTHING;
COMMIT;
-- ============================================================================
-- STEP 6: Create Indexes
-- ============================================================================
BEGIN;
-- Users indexes
CREATE INDEX IF NOT EXISTS idx_users_email ON users(email);
CREATE INDEX IF NOT EXISTS idx_users_status ON users(status);
CREATE INDEX IF NOT EXISTS idx_users_role ON users(role);
CREATE INDEX IF NOT EXISTS idx_users_created_at ON users(created_at);
-- Events indexes
CREATE INDEX IF NOT EXISTS idx_events_start_at ON events(start_at);
CREATE INDEX IF NOT EXISTS idx_events_published ON events(published);
CREATE INDEX IF NOT EXISTS idx_events_created_by ON events(created_by);
-- Event RSVPs indexes
CREATE INDEX IF NOT EXISTS idx_event_rsvps_event_id ON event_rsvps(event_id);
CREATE INDEX IF NOT EXISTS idx_event_rsvps_user_id ON event_rsvps(user_id);
CREATE INDEX IF NOT EXISTS idx_event_rsvps_attended ON event_rsvps(attended);
-- Subscriptions indexes
CREATE INDEX IF NOT EXISTS idx_subscriptions_user_id ON subscriptions(user_id);
CREATE INDEX IF NOT EXISTS idx_subscriptions_status ON subscriptions(status);
CREATE INDEX IF NOT EXISTS idx_subscriptions_stripe_customer_id ON subscriptions(stripe_customer_id);
-- Permissions indexes
CREATE INDEX IF NOT EXISTS ix_permissions_code ON permissions(code);
CREATE INDEX IF NOT EXISTS ix_permissions_module ON permissions(module);
-- Roles indexes
CREATE INDEX IF NOT EXISTS ix_roles_code ON roles(code);
-- Role permissions indexes
CREATE INDEX IF NOT EXISTS ix_role_permissions_role ON role_permissions(role);
CREATE INDEX IF NOT EXISTS ix_role_permissions_role_id ON role_permissions(role_id);
-- User invitations indexes
CREATE INDEX IF NOT EXISTS ix_user_invitations_email ON user_invitations(email);
CREATE INDEX IF NOT EXISTS ix_user_invitations_token ON user_invitations(token);
COMMIT;
\echo '✅ All tables created successfully!'
\echo 'Run: psql ... -c "\dt" to verify'

View File

@@ -0,0 +1,80 @@
-- ============================================================================
-- Database Diagnostic Script
-- Run this to check what exists in your database
-- ============================================================================
\echo '=== CHECKING ENUMS ==='
SELECT
t.typname as enum_name,
string_agg(e.enumlabel, ', ' ORDER BY e.enumsortorder) as values
FROM pg_type t
JOIN pg_enum e ON t.oid = e.enumtypid
WHERE t.typname IN (
'userstatus', 'userrole', 'rsvpstatus', 'subscriptionstatus',
'donationtype', 'donationstatus', 'invitationstatus', 'importjobstatus'
)
GROUP BY t.typname
ORDER BY t.typname;
\echo ''
\echo '=== CHECKING TABLES ==='
SELECT
schemaname,
tablename
FROM pg_tables
WHERE schemaname = 'public'
ORDER BY tablename;
\echo ''
\echo '=== CHECKING USERS TABLE STRUCTURE ==='
SELECT
column_name,
data_type,
is_nullable,
column_default
FROM information_schema.columns
WHERE table_name = 'users'
ORDER BY ordinal_position;
\echo ''
\echo '=== CHECKING FOR CRITICAL FIELDS ==='
\echo 'Checking if reminder tracking fields exist...'
SELECT EXISTS (
SELECT 1
FROM information_schema.columns
WHERE table_name = 'users'
AND column_name = 'email_verification_reminders_sent'
) as has_reminder_fields;
\echo ''
\echo 'Checking if accepts_tos field exists (should be accepts_tos, not tos_accepted)...'
SELECT column_name
FROM information_schema.columns
WHERE table_name = 'users'
AND column_name IN ('accepts_tos', 'tos_accepted');
\echo ''
\echo 'Checking if WordPress import fields exist...'
SELECT EXISTS (
SELECT 1
FROM information_schema.columns
WHERE table_name = 'users'
AND column_name = 'import_source'
) as has_import_fields;
\echo ''
\echo '=== CHECKING IMPORT_JOBS TABLE ==='
SELECT column_name
FROM information_schema.columns
WHERE table_name = 'import_jobs'
ORDER BY ordinal_position;
\echo ''
\echo '=== SUMMARY ==='
SELECT
(SELECT COUNT(*) FROM pg_type WHERE typname IN (
'userstatus', 'userrole', 'rsvpstatus', 'subscriptionstatus',
'donationtype', 'donationstatus', 'invitationstatus', 'importjobstatus'
)) as enum_count,
(SELECT COUNT(*) FROM pg_tables WHERE schemaname = 'public') as table_count,
(SELECT COUNT(*) FROM information_schema.columns WHERE table_name = 'users') as users_column_count;

View File

@@ -0,0 +1,169 @@
-- ============================================================================
-- Fix Missing Fields Script
-- Safely adds missing fields without recreating existing structures
-- ============================================================================
BEGIN;
\echo '=== FIXING USERS TABLE ==='
-- Fix TOS field name if needed (tos_accepted -> accepts_tos)
DO $$
BEGIN
IF EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'users' AND column_name = 'tos_accepted'
) AND NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'users' AND column_name = 'accepts_tos'
) THEN
ALTER TABLE users RENAME COLUMN tos_accepted TO accepts_tos;
RAISE NOTICE 'Renamed tos_accepted to accepts_tos';
END IF;
END $$;
-- Add reminder tracking fields if missing
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'users' AND column_name = 'email_verification_reminders_sent'
) THEN
ALTER TABLE users ADD COLUMN email_verification_reminders_sent INTEGER DEFAULT 0 NOT NULL;
ALTER TABLE users ADD COLUMN last_email_verification_reminder_at TIMESTAMP WITH TIME ZONE;
ALTER TABLE users ADD COLUMN event_attendance_reminders_sent INTEGER DEFAULT 0 NOT NULL;
ALTER TABLE users ADD COLUMN last_event_attendance_reminder_at TIMESTAMP WITH TIME ZONE;
ALTER TABLE users ADD COLUMN payment_reminders_sent INTEGER DEFAULT 0 NOT NULL;
ALTER TABLE users ADD COLUMN last_payment_reminder_at TIMESTAMP WITH TIME ZONE;
ALTER TABLE users ADD COLUMN renewal_reminders_sent INTEGER DEFAULT 0 NOT NULL;
ALTER TABLE users ADD COLUMN last_renewal_reminder_at TIMESTAMP WITH TIME ZONE;
RAISE NOTICE 'Added reminder tracking fields';
END IF;
END $$;
-- Add WordPress import fields if missing
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'users' AND column_name = 'import_source'
) THEN
ALTER TABLE users ADD COLUMN import_source VARCHAR(50);
ALTER TABLE users ADD COLUMN import_job_id UUID REFERENCES import_jobs(id);
ALTER TABLE users ADD COLUMN wordpress_user_id BIGINT;
ALTER TABLE users ADD COLUMN wordpress_registered_date TIMESTAMP WITH TIME ZONE;
RAISE NOTICE 'Added WordPress import tracking fields';
END IF;
END $$;
\echo '=== FIXING IMPORT_JOBS TABLE ==='
-- Add WordPress import enhancement fields if missing
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM information_schema.columns
WHERE table_name = 'import_jobs' AND column_name = 'field_mapping'
) THEN
ALTER TABLE import_jobs ADD COLUMN field_mapping JSONB DEFAULT '{}'::jsonb;
ALTER TABLE import_jobs ADD COLUMN wordpress_metadata JSONB DEFAULT '{}'::jsonb;
ALTER TABLE import_jobs ADD COLUMN imported_user_ids JSONB DEFAULT '[]'::jsonb;
ALTER TABLE import_jobs ADD COLUMN rollback_at TIMESTAMP WITH TIME ZONE;
ALTER TABLE import_jobs ADD COLUMN rollback_by UUID REFERENCES users(id);
RAISE NOTICE 'Added WordPress import enhancement fields to import_jobs';
END IF;
END $$;
-- Add validating, preview_ready, rolled_back to ImportJobStatus enum if missing
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_enum e
JOIN pg_type t ON e.enumtypid = t.oid
WHERE t.typname = 'importjobstatus' AND e.enumlabel = 'validating'
) THEN
ALTER TYPE importjobstatus ADD VALUE IF NOT EXISTS 'validating';
RAISE NOTICE 'Added validating to importjobstatus enum';
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_enum e
JOIN pg_type t ON e.enumtypid = t.oid
WHERE t.typname = 'importjobstatus' AND e.enumlabel = 'preview_ready'
) THEN
ALTER TYPE importjobstatus ADD VALUE IF NOT EXISTS 'preview_ready';
RAISE NOTICE 'Added preview_ready to importjobstatus enum';
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_enum e
JOIN pg_type t ON e.enumtypid = t.oid
WHERE t.typname = 'importjobstatus' AND e.enumlabel = 'rolled_back'
) THEN
ALTER TYPE importjobstatus ADD VALUE IF NOT EXISTS 'rolled_back';
RAISE NOTICE 'Added rolled_back to importjobstatus enum';
END IF;
END $$;
-- Add pending_validation, pre_validated, canceled, expired, abandoned to UserStatus enum if missing
DO $$
BEGIN
IF NOT EXISTS (
SELECT 1 FROM pg_enum e
JOIN pg_type t ON e.enumtypid = t.oid
WHERE t.typname = 'userstatus' AND e.enumlabel = 'pending_validation'
) THEN
ALTER TYPE userstatus ADD VALUE IF NOT EXISTS 'pending_validation';
RAISE NOTICE 'Added pending_validation to userstatus enum';
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_enum e
JOIN pg_type t ON e.enumtypid = t.oid
WHERE t.typname = 'userstatus' AND e.enumlabel = 'pre_validated'
) THEN
ALTER TYPE userstatus ADD VALUE IF NOT EXISTS 'pre_validated';
RAISE NOTICE 'Added pre_validated to userstatus enum';
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_enum e
JOIN pg_type t ON e.enumtypid = t.oid
WHERE t.typname = 'userstatus' AND e.enumlabel = 'canceled'
) THEN
ALTER TYPE userstatus ADD VALUE IF NOT EXISTS 'canceled';
RAISE NOTICE 'Added canceled to userstatus enum';
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_enum e
JOIN pg_type t ON e.enumtypid = t.oid
WHERE t.typname = 'userstatus' AND e.enumlabel = 'expired'
) THEN
ALTER TYPE userstatus ADD VALUE IF NOT EXISTS 'expired';
RAISE NOTICE 'Added expired to userstatus enum';
END IF;
IF NOT EXISTS (
SELECT 1 FROM pg_enum e
JOIN pg_type t ON e.enumtypid = t.oid
WHERE t.typname = 'userstatus' AND e.enumlabel = 'abandoned'
) THEN
ALTER TYPE userstatus ADD VALUE IF NOT EXISTS 'abandoned';
RAISE NOTICE 'Added abandoned to userstatus enum';
END IF;
END $$;
COMMIT;
\echo ''
\echo '=== VERIFICATION ==='
SELECT
(SELECT COUNT(*) FROM information_schema.columns WHERE table_name = 'users') as users_columns,
(SELECT COUNT(*) FROM information_schema.columns WHERE table_name = 'import_jobs') as import_jobs_columns,
(SELECT COUNT(*) FROM pg_enum e JOIN pg_type t ON e.enumtypid = t.oid WHERE t.typname = 'userstatus') as userstatus_values,
(SELECT COUNT(*) FROM pg_enum e JOIN pg_type t ON e.enumtypid = t.oid WHERE t.typname = 'importjobstatus') as importjobstatus_values;
\echo ''
\echo '✅ Missing fields have been added!'
\echo 'You can now run: alembic stamp head'

238
migrations/seed_data.sql Normal file
View File

@@ -0,0 +1,238 @@
-- ============================================================================
-- Seed Data for LOAF Membership Platform
-- Run this after creating the database schema
-- ============================================================================
BEGIN;
-- ============================================================================
-- STEP 1: Create Default Roles
-- ============================================================================
INSERT INTO roles (id, code, name, description, is_system_role, created_at, updated_at)
VALUES
(gen_random_uuid(), 'guest', 'Guest', 'Default role for new registrations', true, NOW(), NOW()),
(gen_random_uuid(), 'member', 'Member', 'Active paying members with full access', true, NOW(), NOW()),
(gen_random_uuid(), 'admin', 'Admin', 'Board members with management access', true, NOW(), NOW()),
(gen_random_uuid(), 'finance', 'Finance', 'Treasurer role with financial access', true, NOW(), NOW()),
(gen_random_uuid(), 'superadmin', 'Super Admin', 'Full system access', true, NOW(), NOW())
ON CONFLICT (code) DO NOTHING;
-- ============================================================================
-- STEP 2: Create Permissions
-- ============================================================================
INSERT INTO permissions (id, code, name, description, module, created_at)
VALUES
-- User Management Permissions
(gen_random_uuid(), 'users.view', 'View Users', 'View user list and profiles', 'users', NOW()),
(gen_random_uuid(), 'users.create', 'Create Users', 'Create new users', 'users', NOW()),
(gen_random_uuid(), 'users.edit', 'Edit Users', 'Edit user information', 'users', NOW()),
(gen_random_uuid(), 'users.delete', 'Delete Users', 'Delete users', 'users', NOW()),
(gen_random_uuid(), 'users.approve', 'Approve Users', 'Approve pending memberships', 'users', NOW()),
(gen_random_uuid(), 'users.import', 'Import Users', 'Import users from CSV/external sources', 'users', NOW()),
-- Event Management Permissions
(gen_random_uuid(), 'events.view', 'View Events', 'View event list and details', 'events', NOW()),
(gen_random_uuid(), 'events.create', 'Create Events', 'Create new events', 'events', NOW()),
(gen_random_uuid(), 'events.edit', 'Edit Events', 'Edit event information', 'events', NOW()),
(gen_random_uuid(), 'events.delete', 'Delete Events', 'Delete events', 'events', NOW()),
(gen_random_uuid(), 'events.publish', 'Publish Events', 'Publish/unpublish events', 'events', NOW()),
(gen_random_uuid(), 'events.manage_attendance', 'Manage Attendance', 'Mark event attendance', 'events', NOW()),
-- Financial Permissions
(gen_random_uuid(), 'finance.view', 'View Financial Data', 'View subscriptions and payments', 'finance', NOW()),
(gen_random_uuid(), 'finance.manage_plans', 'Manage Subscription Plans', 'Create/edit subscription plans', 'finance', NOW()),
(gen_random_uuid(), 'finance.manage_subscriptions', 'Manage Subscriptions', 'Manage user subscriptions', 'finance', NOW()),
(gen_random_uuid(), 'finance.view_reports', 'View Financial Reports', 'Access financial reports', 'finance', NOW()),
(gen_random_uuid(), 'finance.export', 'Export Financial Data', 'Export financial data', 'finance', NOW()),
-- Content Management Permissions
(gen_random_uuid(), 'content.newsletters', 'Manage Newsletters', 'Manage newsletter archives', 'content', NOW()),
(gen_random_uuid(), 'content.documents', 'Manage Documents', 'Manage bylaws and documents', 'content', NOW()),
(gen_random_uuid(), 'content.gallery', 'Manage Gallery', 'Manage event galleries', 'content', NOW()),
-- System Permissions
(gen_random_uuid(), 'system.settings', 'System Settings', 'Manage system settings', 'system', NOW()),
(gen_random_uuid(), 'system.roles', 'Manage Roles', 'Create/edit roles and permissions', 'system', NOW()),
(gen_random_uuid(), 'system.invitations', 'Manage Invitations', 'Send admin invitations', 'system', NOW()),
(gen_random_uuid(), 'system.storage', 'Manage Storage', 'View storage usage', 'system', NOW()),
(gen_random_uuid(), 'system.audit', 'View Audit Logs', 'View system audit logs', 'system', NOW())
ON CONFLICT (code) DO NOTHING;
-- ============================================================================
-- STEP 3: Assign Permissions to Roles
-- ============================================================================
-- Guest Role: No permissions (view-only through public pages)
-- No entries needed
-- Member Role: Limited permissions
INSERT INTO role_permissions (id, role, role_id, permission_id, created_at)
SELECT
gen_random_uuid(),
'member',
(SELECT id FROM roles WHERE code = 'member'),
p.id,
NOW()
FROM permissions p
WHERE p.code IN (
'events.view'
)
ON CONFLICT DO NOTHING;
-- Admin Role: Most permissions except financial
INSERT INTO role_permissions (id, role, role_id, permission_id, created_at)
SELECT
gen_random_uuid(),
'admin',
(SELECT id FROM roles WHERE code = 'admin'),
p.id,
NOW()
FROM permissions p
WHERE p.code IN (
-- User Management
'users.view', 'users.create', 'users.edit', 'users.approve', 'users.import',
-- Event Management
'events.view', 'events.create', 'events.edit', 'events.delete', 'events.publish', 'events.manage_attendance',
-- Content Management
'content.newsletters', 'content.documents', 'content.gallery',
-- System (limited)
'system.invitations', 'system.storage'
)
ON CONFLICT DO NOTHING;
-- Finance Role: Financial permissions + basic access
INSERT INTO role_permissions (id, role, role_id, permission_id, created_at)
SELECT
gen_random_uuid(),
'finance',
(SELECT id FROM roles WHERE code = 'finance'),
p.id,
NOW()
FROM permissions p
WHERE p.code IN (
-- Financial
'finance.view', 'finance.manage_plans', 'finance.manage_subscriptions', 'finance.view_reports', 'finance.export',
-- Basic Access
'users.view', 'events.view'
)
ON CONFLICT DO NOTHING;
-- Superadmin Role: All permissions
INSERT INTO role_permissions (id, role, role_id, permission_id, created_at)
SELECT
gen_random_uuid(),
'superadmin',
(SELECT id FROM roles WHERE code = 'superadmin'),
p.id,
NOW()
FROM permissions p
ON CONFLICT DO NOTHING;
-- ============================================================================
-- STEP 4: Create Subscription Plans
-- ============================================================================
INSERT INTO subscription_plans (id, name, description, price_cents, billing_cycle, custom_cycle_enabled, minimum_price_cents, allow_donation, is_active, created_at, updated_at)
VALUES
-- Annual Individual Membership
(
gen_random_uuid(),
'Annual Individual Membership',
'Standard annual membership for one person. Includes access to all LOAF events, member directory, and exclusive content.',
6000, -- $60.00
'annual',
false,
6000,
false,
true,
NOW(),
NOW()
),
-- Annual Group Membership
(
gen_random_uuid(),
'Annual Group Membership',
'Annual membership for two people living at the same address. Both members receive full access to all LOAF benefits.',
10000, -- $100.00
'annual',
false,
10000,
false,
true,
NOW(),
NOW()
),
-- Pay What You Want (with minimum)
(
gen_random_uuid(),
'Pay What You Want Membership',
'Choose your own annual membership amount. Minimum $30. Additional contributions help support our scholarship fund.',
3000, -- $30.00 minimum
'annual',
true, -- Allow custom amount
3000, -- Minimum $30
true, -- Additional amount is treated as donation
true,
NOW(),
NOW()
)
ON CONFLICT DO NOTHING;
-- ============================================================================
-- STEP 5: Initialize Storage Usage (if not already done)
-- ============================================================================
INSERT INTO storage_usage (id, total_bytes_used, max_bytes_allowed, last_calculated_at, created_at, updated_at)
VALUES (gen_random_uuid(), 0, 107374182400, NOW(), NOW(), NOW()) -- 100GB limit
ON CONFLICT DO NOTHING;
COMMIT;
-- ============================================================================
-- Success Message
-- ============================================================================
\echo '✅ Seed data created successfully!'
\echo ''
\echo 'Created:'
\echo ' - 5 default roles (guest, member, admin, finance, superadmin)'
\echo ' - 25 permissions across 5 modules'
\echo ' - Role-permission mappings'
\echo ' - 3 subscription plans'
\echo ' - Storage usage initialization'
\echo ''
\echo 'Next steps:'
\echo ' 1. Create superadmin user (see instructions below)'
\echo ' 2. Configure Stripe price IDs in subscription_plans'
\echo ' 3. Start the application'
\echo ''
\echo '━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━'
\echo 'CREATE SUPERADMIN USER:'
\echo '━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━'
\echo ''
\echo 'Generate password hash in Python:'
\echo ' python3 -c "import bcrypt; print(bcrypt.hashpw(b\"your-password\", bcrypt.gensalt()).decode())"'
\echo ''
\echo 'Then run:'
\echo ' psql -U postgres -d loaf_new'
\echo ''
\echo 'INSERT INTO users ('
\echo ' id, email, password_hash, first_name, last_name,'
\echo ' status, role, email_verified, created_at, updated_at'
\echo ') VALUES ('
\echo ' gen_random_uuid(),'
\echo ' '\''admin@loafmembers.org'\'','
\echo ' '\''$2b$12$YOUR_BCRYPT_HASH_HERE'\'','
\echo ' '\''Admin'\'','
\echo ' '\''User'\'','
\echo ' '\''active'\'','
\echo ' '\''superadmin'\'','
\echo ' true,'
\echo ' NOW(),'
\echo ' NOW()'
\echo ');'
\echo ''

View File

@@ -130,6 +130,12 @@ class User(Base):
rejected_at = Column(DateTime(timezone=True), nullable=True, comment="Timestamp when application was rejected")
rejected_by = Column(UUID(as_uuid=True), ForeignKey('users.id'), nullable=True, comment="Admin who rejected the application")
# WordPress Import Tracking
import_source = Column(String(50), nullable=True, comment="Source of user creation: wordpress, manual, registration")
import_job_id = Column(UUID(as_uuid=True), ForeignKey('import_jobs.id'), nullable=True, comment="Import job that created this user")
wordpress_user_id = Column(BigInteger, nullable=True, comment="Original WordPress user ID")
wordpress_registered_date = Column(DateTime(timezone=True), nullable=True, comment="Original WordPress registration date")
created_at = Column(DateTime, default=lambda: datetime.now(timezone.utc))
updated_at = Column(DateTime, default=lambda: datetime.now(timezone.utc), onupdate=lambda: datetime.now(timezone.utc))
@@ -451,6 +457,9 @@ class ImportJobStatus(enum.Enum):
completed = "completed"
failed = "failed"
partial = "partial"
validating = "validating"
preview_ready = "preview_ready"
rolled_back = "rolled_back"
class ImportJob(Base):
"""Track CSV import jobs with error handling"""
@@ -466,6 +475,13 @@ class ImportJob(Base):
status = Column(SQLEnum(ImportJobStatus), default=ImportJobStatus.processing, nullable=False)
errors = Column(JSON, default=list, nullable=False) # [{row: 5, field: "email", error: "Invalid format"}]
# WordPress import enhancements
field_mapping = Column(JSON, default=dict, nullable=False) # Maps CSV columns to DB fields
wordpress_metadata = Column(JSON, default=dict, nullable=False) # Preview data, validation results
imported_user_ids = Column(JSON, default=list, nullable=False) # User IDs for rollback
rollback_at = Column(DateTime, nullable=True)
rollback_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=True)
# Tracking
imported_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
started_at = Column(DateTime, default=lambda: datetime.now(timezone.utc), nullable=False)
@@ -473,3 +489,22 @@ class ImportJob(Base):
# Relationships
importer = relationship("User", foreign_keys=[imported_by])
rollback_user = relationship("User", foreign_keys=[rollback_by])
class ImportRollbackAudit(Base):
"""Audit trail for import rollback operations"""
__tablename__ = "import_rollback_audit"
id = Column(UUID(as_uuid=True), primary_key=True, default=uuid.uuid4)
import_job_id = Column(UUID(as_uuid=True), ForeignKey("import_jobs.id"), nullable=False)
rolled_back_by = Column(UUID(as_uuid=True), ForeignKey("users.id"), nullable=False)
rolled_back_at = Column(DateTime, default=lambda: datetime.now(timezone.utc), nullable=False)
deleted_user_count = Column(Integer, nullable=False)
deleted_user_ids = Column(JSON, nullable=False) # List of deleted user UUIDs
reason = Column(Text, nullable=True)
created_at = Column(DateTime, default=lambda: datetime.now(timezone.utc), nullable=False)
# Relationships
import_job = relationship("ImportJob")
admin_user = relationship("User", foreign_keys=[rolled_back_by])

View File

@@ -1,4 +1,5 @@
aiosmtplib==5.0.0
alembic==1.14.0
annotated-types==0.7.0
anyio==4.11.0
bcrypt==4.1.3
@@ -37,6 +38,7 @@ pandas==2.3.3
passlib==1.7.4
pathspec==0.12.1
pillow==10.2.0
phpserialize==1.3
platformdirs==4.5.0
pluggy==1.6.0
psycopg2-binary==2.9.11

606
server.py
View File

@@ -17,7 +17,7 @@ import csv
import io
from database import engine, get_db, Base
from models import User, Event, EventRSVP, UserStatus, UserRole, RSVPStatus, SubscriptionPlan, Subscription, SubscriptionStatus, StorageUsage, EventGallery, NewsletterArchive, FinancialReport, BylawsDocument, Permission, RolePermission, Role, UserInvitation, InvitationStatus, ImportJob, ImportJobStatus, Donation, DonationType, DonationStatus
from models import User, Event, EventRSVP, UserStatus, UserRole, RSVPStatus, SubscriptionPlan, Subscription, SubscriptionStatus, StorageUsage, EventGallery, NewsletterArchive, FinancialReport, BylawsDocument, Permission, RolePermission, Role, UserInvitation, InvitationStatus, ImportJob, ImportJobStatus, ImportRollbackAudit, Donation, DonationType, DonationStatus
from auth import (
get_password_hash,
verify_password,
@@ -42,6 +42,7 @@ from email_service import (
from payment_service import create_checkout_session, verify_webhook_signature, get_subscription_end_date
from r2_storage import get_r2_storage
from calendar_service import CalendarService
from wordpress_parser import analyze_csv, format_preview_for_display
# Load environment variables
ROOT_DIR = Path(__file__).parent
@@ -655,9 +656,15 @@ async def login(request: LoginRequest, db: Session = Depends(get_db)):
access_token = create_access_token(data={"sub": str(user.id)})
# Clear verification token on first successful login after verification
# Don't let this fail the login if database commit fails
if user.email_verified and user.email_verification_token:
user.email_verification_token = None
db.commit()
try:
user.email_verification_token = None
db.commit()
except Exception as e:
logger.warning(f"Failed to clear verification token for user {user.id}: {str(e)}")
db.rollback()
# Continue with login - this is not critical
return {
"access_token": access_token,
@@ -887,7 +894,8 @@ async def get_member_directory(
"social_media_facebook": member.social_media_facebook,
"social_media_instagram": member.social_media_instagram,
"social_media_twitter": member.social_media_twitter,
"social_media_linkedin": member.social_media_linkedin
"social_media_linkedin": member.social_media_linkedin,
"created_at": member.created_at.isoformat() if member.created_at else None
} for member in directory_members]
@api_router.get("/members/directory/{user_id}")
@@ -922,7 +930,8 @@ async def get_directory_member_profile(
"social_media_facebook": member.social_media_facebook,
"social_media_instagram": member.social_media_instagram,
"social_media_twitter": member.social_media_twitter,
"social_media_linkedin": member.social_media_linkedin
"social_media_linkedin": member.social_media_linkedin,
"created_at": member.created_at.isoformat() if member.created_at else None
}
# Enhanced Profile Routes (Active Members Only)
@@ -1573,6 +1582,54 @@ async def rsvp_to_event(
return {"message": "RSVP updated successfully"}
@api_router.get("/members/event-activity")
async def get_my_event_activity(
current_user: User = Depends(get_active_member),
db: Session = Depends(get_db)
):
"""
Get current user's event activity including upcoming RSVPs and attendance history
"""
# Get all user's RSVPs
rsvps = db.query(EventRSVP).filter(
EventRSVP.user_id == current_user.id
).order_by(EventRSVP.created_at.desc()).all()
# Categorize events
upcoming_events = []
past_events = []
now = datetime.now(timezone.utc)
for rsvp in rsvps:
event = db.query(Event).filter(Event.id == rsvp.event_id).first()
if not event:
continue
event_data = {
"id": str(event.id),
"title": event.title,
"description": event.description,
"location": event.location,
"start_at": event.start_at.isoformat(),
"end_at": event.end_at.isoformat(),
"rsvp_status": rsvp.rsvp_status.value,
"attended": rsvp.attended,
"attended_at": rsvp.attended_at.isoformat() if rsvp.attended_at else None
}
# Separate upcoming vs past events
if event.end_at > now:
upcoming_events.append(event_data)
else:
past_events.append(event_data)
return {
"upcoming_events": sorted(upcoming_events, key=lambda x: x["start_at"]),
"past_events": sorted(past_events, key=lambda x: x["start_at"], reverse=True),
"total_attended": sum(1 for rsvp in rsvps if rsvp.attended),
"total_rsvps": len(rsvps)
}
# ============================================================================
# Calendar Export Endpoints (Universal iCalendar .ics format)
# ============================================================================
@@ -3144,6 +3201,529 @@ async def get_import_job_details(
}
# ============================================================================
# WordPress CSV Import Endpoints
# ============================================================================
@api_router.post("/admin/import/upload-csv")
async def upload_wordpress_csv(
file: UploadFile = File(...),
current_user: User = Depends(require_permission("users.import")),
db: Session = Depends(get_db)
):
"""
Upload WordPress CSV, parse, and generate status suggestions.
This endpoint:
1. Validates the CSV file
2. Uploads to R2 storage
3. Parses WordPress data (PHP serialized roles, etc.)
4. Generates smart status suggestions
5. Creates ImportJob record with status='preview_ready'
6. Stores preview data in wordpress_metadata field
Returns:
Import job summary with data quality metrics
Requires permission: users.import
"""
# Validate file type
if not file.filename.endswith('.csv'):
raise HTTPException(status_code=400, detail="Only CSV files are supported")
# Validate file size (10MB max)
MAX_FILE_SIZE = 10 * 1024 * 1024 # 10MB
contents = await file.read()
if len(contents) > MAX_FILE_SIZE:
raise HTTPException(status_code=400, detail="File size exceeds 10MB limit")
# Save to temporary file for parsing
import tempfile
with tempfile.NamedTemporaryFile(mode='wb', delete=False, suffix='.csv') as tmp:
tmp.write(contents)
tmp_path = tmp.name
try:
# Fetch existing emails from database to check for duplicates
existing_emails = set(
email.lower() for (email,) in db.query(User.email).all()
)
logger.info(f"Checking against {len(existing_emails)} existing emails in database")
# Parse CSV with WordPress parser
analysis_result = analyze_csv(tmp_path, existing_emails=existing_emails)
# Note: File contents stored in wordpress_metadata, R2 upload optional
# Could implement R2 upload later if needed for archival purposes
# Create ImportJob record
import_job = ImportJob(
filename=file.filename,
file_key=None, # Optional: could add R2 upload later
total_rows=analysis_result['total_rows'],
processed_rows=0,
successful_rows=0,
failed_rows=0,
status=ImportJobStatus.preview_ready,
wordpress_metadata={
'preview_data': analysis_result['preview_data'],
'data_quality': analysis_result['data_quality'],
'valid_rows': analysis_result['valid_rows'],
'warnings': analysis_result['warnings'],
'errors': analysis_result['errors']
},
imported_by=current_user.id
)
db.add(import_job)
db.commit()
db.refresh(import_job)
logger.info(f"WordPress CSV uploaded: {import_job.id} by {current_user.email}")
return {
'import_job_id': str(import_job.id),
'total_rows': analysis_result['total_rows'],
'valid_rows': analysis_result['valid_rows'],
'warnings': analysis_result['warnings'],
'errors': analysis_result['errors'],
'data_quality': analysis_result['data_quality']
}
except Exception as e:
logger.error(f"Failed to upload WordPress CSV: {str(e)}")
raise HTTPException(status_code=500, detail=f"Failed to process CSV: {str(e)}")
finally:
# Clean up temp file
if os.path.exists(tmp_path):
os.unlink(tmp_path)
@api_router.get("/admin/import/{job_id}/preview")
async def get_import_preview(
job_id: str,
page: int = 1,
page_size: int = 50,
current_user: User = Depends(require_permission("users.view")),
db: Session = Depends(get_db)
):
"""
Get paginated preview data for WordPress import status review.
Returns preview data with suggested status mappings that admins
can review and override before executing the import.
Args:
job_id: Import job UUID
page: Page number (1-indexed)
page_size: Number of rows per page (default 50)
Returns:
Paginated preview data with status suggestions and warnings
Requires permission: users.view
"""
# Get import job
job = db.query(ImportJob).filter(ImportJob.id == job_id).first()
if not job:
raise HTTPException(status_code=404, detail="Import job not found")
# Verify job is in preview_ready status
if job.status != ImportJobStatus.preview_ready:
raise HTTPException(
status_code=400,
detail=f"Import job is not in preview_ready status (current: {job.status.value})"
)
# Get preview data from wordpress_metadata
preview_data = job.wordpress_metadata.get('preview_data', [])
# Format for paginated display
paginated = format_preview_for_display(preview_data, page, page_size)
return paginated
@api_router.post("/admin/import/{job_id}/execute")
async def execute_wordpress_import(
job_id: str,
overrides: dict = {},
options: dict = {},
current_user: User = Depends(require_permission("users.import")),
db: Session = Depends(get_db)
):
"""
Execute WordPress import with admin status overrides.
Process:
1. Merge status overrides with suggested mappings
2. Create users in batches (commit every 20 rows)
3. Track imported_user_ids for rollback capability
4. Queue password reset emails (async)
5. Update import job status
Args:
job_id: Import job UUID
overrides: Dict mapping row_number to status override
e.g., {'1': {'status': 'active'}, '5': {'status': 'inactive'}}
options: Import options
- send_password_emails: bool (default True)
- skip_errors: bool (default True)
Returns:
Import results with success/failure counts
Requires permission: users.import
"""
# Get import job
job = db.query(ImportJob).filter(ImportJob.id == job_id).first()
if not job:
raise HTTPException(status_code=404, detail="Import job not found")
# Verify job is in preview_ready status
if job.status != ImportJobStatus.preview_ready:
raise HTTPException(
status_code=400,
detail=f"Import job is not in preview_ready status (current: {job.status.value})"
)
# Update job status to processing
job.status = ImportJobStatus.processing
db.commit()
# Get preview data
preview_data = job.wordpress_metadata.get('preview_data', [])
# Import configuration
send_password_emails = options.get('send_password_emails', True)
skip_errors = options.get('skip_errors', True)
# Track results
imported_user_ids = []
successful_rows = 0
failed_rows = 0
errors = []
# Generate default password for all imported users
default_password_hash = get_password_hash(secrets.token_urlsafe(32))
try:
# Process each row
for idx, row_data in enumerate(preview_data):
row_num = row_data['row_number']
try:
# Skip rows with critical errors
if row_data.get('errors') and skip_errors:
failed_rows += 1
errors.append({
'row': row_num,
'email': row_data.get('email'),
'error': ', '.join(row_data['errors'])
})
continue
# Apply status override if provided
final_status = row_data['suggested_status']
if str(row_num) in overrides:
final_status = overrides[str(row_num)].get('status', final_status)
# Check if user already exists
existing_user = db.query(User).filter(User.email == row_data['email']).first()
if existing_user:
failed_rows += 1
errors.append({
'row': row_num,
'email': row_data['email'],
'error': 'User with this email already exists'
})
continue
# Create user
new_user = User(
email=row_data['email'],
password_hash=default_password_hash,
first_name=row_data.get('first_name', ''),
last_name=row_data.get('last_name', ''),
phone=row_data.get('phone'),
address='', # WordPress CSV doesn't have address data
city='',
state='',
zipcode='',
date_of_birth=row_data.get('date_of_birth'),
status=UserStatus[final_status],
role=UserRole[row_data['suggested_role']],
newsletter_subscribed=row_data.get('newsletter_consent', False),
email_verified=True, # WordPress users are pre-verified
import_source='wordpress',
import_job_id=job.id,
wordpress_user_id=row_data.get('wordpress_user_id'),
wordpress_registered_date=row_data.get('wordpress_registered')
)
db.add(new_user)
db.flush() # Flush to get the ID without committing
imported_user_ids.append(str(new_user.id))
successful_rows += 1
# Commit in batches of 20
if (idx + 1) % 20 == 0:
db.commit()
job.processed_rows = idx + 1
db.commit()
except Exception as e:
logger.error(f"Failed to import row {row_num}: {str(e)}")
failed_rows += 1
errors.append({
'row': row_num,
'email': row_data.get('email', ''),
'error': str(e)
})
if not skip_errors:
db.rollback()
raise HTTPException(status_code=500, detail=f"Import failed at row {row_num}: {str(e)}")
# Final commit
db.commit()
# Update import job
job.processed_rows = len(preview_data)
job.successful_rows = successful_rows
job.failed_rows = failed_rows
job.status = ImportJobStatus.completed if failed_rows == 0 else ImportJobStatus.partial
job.imported_user_ids = imported_user_ids
job.error_log = errors
job.completed_at = datetime.now(timezone.utc)
db.commit()
# Queue password reset emails (async, non-blocking)
password_emails_queued = 0
if send_password_emails and imported_user_ids:
try:
for user_id_str in imported_user_ids:
try:
# Convert to UUID and fetch user
user_uuid = uuid.UUID(user_id_str)
user = db.query(User).filter(User.id == user_uuid).first()
if user:
# Generate password reset token
reset_token = create_password_reset_token(user.email)
reset_url = f"{os.getenv('FRONTEND_URL')}/reset-password?token={reset_token}"
# Send email (async)
await send_password_reset_email(user.email, user.first_name, reset_url)
password_emails_queued += 1
except (ValueError, AttributeError) as e:
logger.warning(f"Skipping invalid user ID: {user_id_str}")
continue
except Exception as e:
logger.error(f"Failed to send password reset emails: {str(e)}")
# Don't fail import if emails fail
logger.info(f"Import executed: {job.id} - {successful_rows}/{len(preview_data)} by {current_user.email}")
return {
'successful_rows': successful_rows,
'failed_rows': failed_rows,
'imported_user_ids': imported_user_ids,
'password_emails_queued': password_emails_queued,
'errors': errors
}
except Exception as e:
db.rollback()
job.status = ImportJobStatus.failed
job.error_log = [{'error': str(e)}]
db.commit()
logger.error(f"Import execution failed: {str(e)}")
raise HTTPException(status_code=500, detail=f"Import execution failed: {str(e)}")
@api_router.post("/admin/import/{job_id}/rollback")
async def rollback_import_job(
job_id: str,
confirm: bool = False,
current_user: User = Depends(require_permission("users.import")),
db: Session = Depends(get_db)
):
"""
Delete all users from a specific import job (full rollback).
Safety checks:
- Requires confirm=True parameter
- Verifies job status is completed or partial
- Cannot rollback twice (checks rollback_at is None)
- Logs action to import_rollback_audit table
Args:
job_id: Import job UUID
confirm: Must be True to execute rollback
Returns:
Number of deleted users and confirmation message
Requires permission: users.import
"""
# Safety check: require explicit confirmation
if not confirm:
raise HTTPException(
status_code=400,
detail="Rollback requires confirm=true parameter"
)
# Get import job
job = db.query(ImportJob).filter(ImportJob.id == job_id).first()
if not job:
raise HTTPException(status_code=404, detail="Import job not found")
# Verify job can be rolled back
if job.status not in [ImportJobStatus.completed, ImportJobStatus.partial]:
raise HTTPException(
status_code=400,
detail=f"Cannot rollback import with status: {job.status.value}"
)
if job.rollback_at:
raise HTTPException(
status_code=400,
detail="Import has already been rolled back"
)
# Get imported user IDs
imported_user_ids = job.imported_user_ids or []
if not imported_user_ids:
raise HTTPException(
status_code=400,
detail="No users to rollback (imported_user_ids is empty)"
)
try:
# Delete all imported users
deleted_count = db.query(User).filter(
User.id.in_([uuid.UUID(uid) for uid in imported_user_ids])
).delete(synchronize_session=False)
# Update import job
job.status = ImportJobStatus.rolled_back
job.rollback_at = datetime.now(timezone.utc)
job.rollback_by = current_user.id
# Create audit record
from models import ImportRollbackAudit
audit = ImportRollbackAudit(
import_job_id=job.id,
rolled_back_by=current_user.id,
deleted_user_count=deleted_count,
deleted_user_ids=imported_user_ids,
reason="Manual rollback by admin"
)
db.add(audit)
db.commit()
logger.warning(f"Import rolled back: {job.id} - {deleted_count} users deleted by {current_user.email}")
return {
'deleted_users': deleted_count,
'message': f'Import rolled back successfully. {deleted_count} users deleted.'
}
except Exception as e:
db.rollback()
logger.error(f"Rollback failed for job {job.id}: {str(e)}")
raise HTTPException(status_code=500, detail=f"Rollback failed: {str(e)}")
@api_router.get("/admin/import/{job_id}/status")
async def get_import_status(
job_id: str,
current_user: User = Depends(require_permission("users.view")),
db: Session = Depends(get_db)
):
"""
Get real-time import progress status for polling.
Use this endpoint to poll for import progress updates
while the import is executing.
Args:
job_id: Import job UUID
Returns:
Current import status with progress percentage
Requires permission: users.view
"""
job = db.query(ImportJob).filter(ImportJob.id == job_id).first()
if not job:
raise HTTPException(status_code=404, detail="Import job not found")
progress_percent = 0.0
if job.total_rows > 0:
progress_percent = (job.processed_rows / job.total_rows) * 100
return {
'status': job.status.value,
'processed_rows': job.processed_rows,
'total_rows': job.total_rows,
'progress_percent': round(progress_percent, 1),
'successful_rows': job.successful_rows,
'failed_rows': job.failed_rows
}
@api_router.get("/admin/import/{job_id}/errors/download")
async def download_error_report(
job_id: str,
current_user: User = Depends(require_permission("users.view")),
db: Session = Depends(get_db)
):
"""
Download CSV report with all import errors.
CSV columns: Row Number, Email, Error Type, Error Message, Original Data
Args:
job_id: Import job UUID
Returns:
StreamingResponse with CSV file
Requires permission: users.view
"""
job = db.query(ImportJob).filter(ImportJob.id == job_id).first()
if not job:
raise HTTPException(status_code=404, detail="Import job not found")
errors = job.error_log or []
if not errors:
raise HTTPException(status_code=404, detail="No errors found for this import job")
# Generate CSV
output = io.StringIO()
writer = csv.DictWriter(output, fieldnames=['Row Number', 'Email', 'Error Type', 'Error Message'])
writer.writeheader()
for error in errors:
writer.writerow({
'Row Number': error.get('row', ''),
'Email': error.get('email', ''),
'Error Type': 'Import Error',
'Error Message': error.get('error', '')
})
# Return as streaming response
output.seek(0)
return StreamingResponse(
iter([output.getvalue()]),
media_type="text/csv",
headers={"Content-Disposition": f"attachment; filename=import_errors_{job_id}.csv"}
)
@api_router.post("/admin/events", response_model=EventResponse)
async def create_event(
request: EventCreate,
@@ -3256,10 +3836,20 @@ async def mark_attendance(
EventRSVP.event_id == event_id,
EventRSVP.user_id == request.user_id
).first()
# Auto-create RSVP if it doesn't exist (for retroactive attendance marking)
if not rsvp:
raise HTTPException(status_code=404, detail="RSVP not found")
rsvp = EventRSVP(
event_id=event_id,
user_id=request.user_id,
rsvp_status=RSVPStatus.yes, # Default to 'yes' for attended events
attended=False,
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc)
)
db.add(rsvp)
db.flush() # Get the ID without committing
rsvp.attended = request.attended
rsvp.attended_at = datetime.now(timezone.utc) if request.attended else None
rsvp.updated_at = datetime.now(timezone.utc)

531
wordpress_parser.py Normal file
View File

@@ -0,0 +1,531 @@
"""
WordPress CSV Parser Module
This module provides utilities for parsing WordPress user export CSV files
and transforming them into LOAF platform-compatible data structures.
Key Features:
- Parse PHP serialized data (WordPress capabilities)
- Map WordPress roles to LOAF roles and statuses
- Validate and standardize user data (DOB, phone numbers)
- Generate smart status suggestions based on approval and subscription data
- Comprehensive data quality analysis and error reporting
Author: Claude Code
Date: 2025-12-24
"""
import csv
import re
import logging
from datetime import datetime
from typing import Dict, List, Optional, Tuple
import phpserialize
logger = logging.getLogger(__name__)
# ============================================================================
# WordPress Role Mapping Configuration
# ============================================================================
ROLE_MAPPING = {
# WordPress admin roles → LOAF admin roles (auto-active)
'administrator': ('superadmin', 'active'),
'loaf_admin': ('admin', 'active'),
'loaf_treasure': ('finance', 'active'),
'loaf_communication': ('admin', 'active'),
# WordPress member roles → LOAF member role (status from approval)
'pms_subscription_plan_63': ('member', None), # Status determined by approval
'registered': ('guest', None), # Default WordPress role
# Fallback for unknown roles
'__default__': ('guest', None)
}
# Role priority order (higher index = higher priority)
ROLE_PRIORITY = [
'registered',
'pms_subscription_plan_63',
'loaf_communication',
'loaf_treasure',
'loaf_admin',
'administrator'
]
# ============================================================================
# PHP Serialization Parsing
# ============================================================================
def parse_php_serialized(data: str) -> List[str]:
"""
Parse WordPress PHP serialized capabilities string.
WordPress stores user capabilities as serialized PHP arrays like:
a:1:{s:10:"registered";b:1;}
a:2:{s:10:"registered";b:1;s:24:"pms_subscription_plan_63";b:1;}
Args:
data: PHP serialized string
Returns:
List of role names (e.g., ['registered', 'pms_subscription_plan_63'])
Examples:
>>> parse_php_serialized('a:1:{s:10:"registered";b:1;}')
['registered']
>>> parse_php_serialized('a:2:{s:10:"registered";b:1;s:24:"pms_subscription_plan_63";b:1;}')
['registered', 'pms_subscription_plan_63']
"""
if not data or pd.isna(data):
return []
try:
# Use phpserialize library to parse
parsed = phpserialize.loads(data.encode('utf-8'))
# Extract role names (keys where value is True)
if isinstance(parsed, dict):
roles = [key.decode('utf-8') if isinstance(key, bytes) else key
for key, value in parsed.items() if value]
return roles
return []
except Exception as e:
logger.warning(f"Failed to parse PHP serialized data: {data[:50]}... Error: {str(e)}")
return []
# ============================================================================
# Role and Status Mapping
# ============================================================================
def map_wordpress_role(wp_roles: List[str]) -> Tuple[str, Optional[str]]:
"""
Map WordPress roles to LOAF role and suggested status.
Priority logic:
1. If user has any admin role → corresponding LOAF admin role with 'active' status
2. If user has subscription → 'member' role (status from approval)
3. Otherwise → 'guest' role (status from approval)
Args:
wp_roles: List of WordPress role names
Returns:
Tuple of (loaf_role, suggested_status)
- loaf_role: One of: superadmin, admin, finance, member, guest
- suggested_status: One of: active, pre_validated, payment_pending, None (determined by approval)
Examples:
>>> map_wordpress_role(['loaf_admin'])
('admin', 'active')
>>> map_wordpress_role(['loaf_treasure'])
('finance', 'active')
>>> map_wordpress_role(['pms_subscription_plan_63', 'registered'])
('member', None)
>>> map_wordpress_role(['registered'])
('guest', None)
"""
if not wp_roles:
return ROLE_MAPPING['__default__']
# Sort roles by priority (highest priority last)
prioritized_roles = sorted(
wp_roles,
key=lambda r: ROLE_PRIORITY.index(r) if r in ROLE_PRIORITY else -1
)
# Map highest priority role
highest_role = prioritized_roles[-1] if prioritized_roles else 'registered'
return ROLE_MAPPING.get(highest_role, ROLE_MAPPING['__default__'])
def suggest_status(approval_status: str, has_subscription: bool, wordpress_role: str = 'guest') -> str:
"""
Suggest LOAF user status based on WordPress approval and subscription data.
Logic:
1. Admin roles (loaf_admin, loaf_treasure, administrator) → always 'active'
2. approved + subscription → 'active'
3. approved without subscription → 'pre_validated'
4. pending → 'payment_pending'
5. Other/empty → 'pre_validated'
Args:
approval_status: WordPress approval status (approved, pending, unapproved, etc.)
has_subscription: Whether user has pms_subscription_plan_63 role
wordpress_role: LOAF role mapped from WordPress (for admin check)
Returns:
Suggested LOAF status: active, pre_validated, payment_pending, or inactive
Examples:
>>> suggest_status('approved', True, 'member')
'active'
>>> suggest_status('approved', False, 'member')
'pre_validated'
>>> suggest_status('pending', True, 'member')
'payment_pending'
>>> suggest_status('', False, 'admin')
'active'
"""
# Admin roles are always active
if wordpress_role in ('superadmin', 'admin', 'finance'):
return 'active'
# Normalize approval status
approval = (approval_status or '').lower().strip()
if approval == 'approved':
return 'active' if has_subscription else 'pre_validated'
elif approval == 'pending':
return 'payment_pending'
elif approval == 'unapproved':
return 'inactive'
else:
# Empty or unknown approval status
return 'pre_validated'
# ============================================================================
# Data Validation and Standardization
# ============================================================================
def standardize_phone(phone: str) -> str:
"""
Standardize phone number by extracting digits only.
Removes all non-digit characters:
- (713) 560-7850 → 7135607850
- 713-725-8902 → 7137258902
- Empty/None → 0000000000 (fallback)
Args:
phone: Phone number in any format
Returns:
10-digit phone number string (or 0000000000 if invalid)
Examples:
>>> standardize_phone('(713) 560-7850')
'7135607850'
>>> standardize_phone('713-725-8902')
'7137258902'
>>> standardize_phone('')
'0000000000'
"""
if not phone or pd.isna(phone):
return '0000000000'
# Extract all digits
digits = re.sub(r'\D', '', str(phone))
# Return 10 digits or fallback
if len(digits) == 10:
return digits
elif len(digits) == 11 and digits[0] == '1':
# Remove leading 1 (US country code)
return digits[1:]
else:
logger.warning(f"Invalid phone format: {phone} (extracted: {digits})")
return '0000000000'
def validate_dob(dob_str: str) -> Tuple[Optional[datetime], Optional[str]]:
"""
Validate and parse date of birth.
Validation rules:
- Must be in MM/DD/YYYY format
- Year must be between 1900 and current year
- Cannot be in the future
- Reject year 0000 or 2025+ (data quality issues in WordPress export)
Args:
dob_str: Date of birth string in MM/DD/YYYY format
Returns:
Tuple of (parsed_datetime, warning_message)
- parsed_datetime: datetime object if valid, None if invalid
- warning_message: Descriptive error message if invalid, None if valid
Examples:
>>> validate_dob('08/02/1962')
(datetime(1962, 8, 2), None)
>>> validate_dob('08/02/0000')
(None, 'Invalid year: 0000')
>>> validate_dob('08/02/2025')
(None, 'Date is in the future')
"""
if not dob_str or pd.isna(dob_str):
return None, 'Missing date of birth'
try:
# Parse MM/DD/YYYY format
parsed = datetime.strptime(str(dob_str).strip(), '%m/%d/%Y')
# Validate year range
if parsed.year == 0:
return None, 'Invalid year: 0000 (data quality issue)'
elif parsed.year < 1900:
return None, f'Year too old: {parsed.year} (likely invalid)'
elif parsed.year > datetime.now().year:
return None, f'Date is in the future: {parsed.year}'
elif parsed > datetime.now():
return None, 'Date is in the future'
return parsed, None
except ValueError as e:
return None, f'Invalid date format: {dob_str} (expected MM/DD/YYYY)'
# ============================================================================
# CSV Analysis and Preview Generation
# ============================================================================
def analyze_csv(file_path: str, existing_emails: Optional[set] = None) -> Dict:
"""
Analyze WordPress CSV file and generate preview data with status suggestions.
This is the main entry point for CSV processing. It:
1. Reads and parses the CSV file
2. Validates each row and generates warnings
3. Maps WordPress roles to LOAF roles
4. Suggests status for each user
5. Tracks data quality metrics
6. Checks for duplicate emails (both within CSV and against existing database)
7. Returns comprehensive analysis and preview data
Args:
file_path: Path to WordPress CSV export file
existing_emails: Set of emails already in the database (optional)
Returns:
Dictionary containing:
- total_rows: Total number of user rows
- valid_rows: Number of rows without critical errors
- warnings: Total warning count
- errors: Total critical error count
- preview_data: List of row dictionaries with suggestions
- data_quality: Dictionary of data quality metrics
Example output:
{
'total_rows': 183,
'valid_rows': 176,
'warnings': 66,
'errors': 7,
'preview_data': [
{
'row_number': 1,
'email': 'user@example.com',
'first_name': 'John',
'last_name': 'Doe',
'phone': '7135607850',
'date_of_birth': '1962-08-02',
'wordpress_roles': ['registered', 'pms_subscription_plan_63'],
'suggested_role': 'member',
'suggested_status': 'active',
'warnings': [],
'errors': []
},
...
],
'data_quality': {
'invalid_dob': 66,
'missing_phone': 7,
'duplicate_email_csv': 0,
'duplicate_email_db': 3,
'unparseable_roles': 2
}
}
"""
import pandas as pd
# Read CSV with pandas
df = pd.read_csv(file_path)
total_rows = len(df)
preview_data = []
data_quality = {
'invalid_dob': 0,
'missing_phone': 0,
'duplicate_email_csv': 0,
'duplicate_email_db': 0,
'unparseable_roles': 0,
'missing_email': 0
}
# Track seen emails for CSV duplicate detection
seen_emails = {}
# Convert existing_emails to set if provided
if existing_emails is None:
existing_emails = set()
for idx, row in df.iterrows():
row_num = idx + 1
warnings = []
errors = []
# Extract and validate email
email = str(row.get('user_email', '')).strip().lower()
if not email or email == 'nan':
errors.append('Missing email address')
data_quality['missing_email'] += 1
else:
# Check for duplicates within CSV
if email in seen_emails:
errors.append(f'Duplicate email in CSV (also in row {seen_emails[email]})')
data_quality['duplicate_email_csv'] += 1
# Check for duplicates in existing database
elif email in existing_emails:
errors.append(f'Email already exists in database')
data_quality['duplicate_email_db'] += 1
else:
seen_emails[email] = row_num
# Extract basic fields
first_name = str(row.get('first_name', '')).strip()
last_name = str(row.get('last_name', '')).strip()
# Parse and validate DOB
dob_parsed, dob_warning = validate_dob(row.get('date_of_birth'))
if dob_warning:
warnings.append(dob_warning)
data_quality['invalid_dob'] += 1
# Standardize phone
phone = standardize_phone(row.get('cell_phone'))
if phone == '0000000000':
warnings.append('Missing or invalid phone number')
data_quality['missing_phone'] += 1
# Parse WordPress roles
wp_capabilities = row.get('wp_capabilities', '')
wp_roles = parse_php_serialized(wp_capabilities)
if not wp_roles and wp_capabilities:
warnings.append('Could not parse WordPress roles')
data_quality['unparseable_roles'] += 1
# Map to LOAF role and status
loaf_role, role_suggested_status = map_wordpress_role(wp_roles)
# Determine if user has subscription
has_subscription = 'pms_subscription_plan_63' in wp_roles
# Get approval status
approval_status = str(row.get('wppb_approval_status', '')).strip()
# Suggest final status
if role_suggested_status:
# Admin roles have fixed status from role mapping
suggested_status = role_suggested_status
else:
# Regular users get status from approval logic
suggested_status = suggest_status(approval_status, has_subscription, loaf_role)
# Build preview row
preview_row = {
'row_number': row_num,
'email': email,
'first_name': first_name,
'last_name': last_name,
'phone': phone,
'date_of_birth': dob_parsed.isoformat() if dob_parsed else None,
'wordpress_user_id': int(row.get('ID', 0)) if pd.notna(row.get('ID')) else None,
'wordpress_registered': str(row.get('user_registered', '')),
'wordpress_roles': wp_roles,
'wordpress_approval_status': approval_status,
'has_subscription': has_subscription,
'suggested_role': loaf_role,
'suggested_status': suggested_status,
'warnings': warnings,
'errors': errors,
'newsletter_consent': str(row.get('newsletter_consent', '')).lower() == 'yes',
'newsletter_checklist': str(row.get('newsletter_checklist', '')).lower() == 'yes'
}
preview_data.append(preview_row)
# Calculate summary statistics
valid_rows = sum(1 for row in preview_data if not row['errors'])
total_warnings = sum(len(row['warnings']) for row in preview_data)
total_errors = sum(len(row['errors']) for row in preview_data)
return {
'total_rows': total_rows,
'valid_rows': valid_rows,
'warnings': total_warnings,
'errors': total_errors,
'preview_data': preview_data,
'data_quality': data_quality
}
# ============================================================================
# Utility Functions
# ============================================================================
def get_status_badge_color(status: str) -> str:
"""
Get appropriate badge color for status display in UI.
Args:
status: User status string
Returns:
Tailwind CSS color class
"""
colors = {
'active': 'bg-green-100 text-green-800',
'pre_validated': 'bg-blue-100 text-blue-800',
'payment_pending': 'bg-yellow-100 text-yellow-800',
'inactive': 'bg-gray-100 text-gray-800',
'pending_email': 'bg-purple-100 text-purple-800',
'awaiting_event': 'bg-indigo-100 text-indigo-800'
}
return colors.get(status, 'bg-gray-100 text-gray-800')
def format_preview_for_display(preview_data: List[Dict], page: int = 1, page_size: int = 50) -> Dict:
"""
Format preview data for paginated display in frontend.
Args:
preview_data: Full preview data list
page: Page number (1-indexed)
page_size: Number of rows per page
Returns:
Dictionary with paginated data and metadata
"""
total_pages = (len(preview_data) + page_size - 1) // page_size
start_idx = (page - 1) * page_size
end_idx = start_idx + page_size
return {
'page': page,
'page_size': page_size,
'total_pages': total_pages,
'total_rows': len(preview_data),
'rows': preview_data[start_idx:end_idx]
}
# ============================================================================
# Module Initialization
# ============================================================================
# Import pandas for CSV processing
try:
import pandas as pd
except ImportError:
logger.error("pandas library not found. Please install: pip install pandas")
raise
logger.info("WordPress parser module loaded successfully")