Solution: Updated backend/r2_storage.py:
- Added ALLOWED_CSV_TYPES for CSV file validation
- Added upload_bytes() method for uploading raw bytes to R2
- Added download_file() method for retrieving files from R2
- Added delete_multiple() method for bulk file deletion
Comprehensive upload endpoint now stores CSVs in R2:
r2_storage = get_r2_storage()
for file_type, (content, filename) in file_contents.items():
_, r2_key, _ = await r2_storage.upload_bytes(
content=content,
folder=f"imports/{job_id}",
filename=f"{file_type}_{filename}",
content_type='text/csv'
)
r2_keys[file_type] = r2_key
---
2. Stripe Transaction ID Tracking
Solution: Updated subscription and donation imports to capture Stripe metadata:
Subscription fields:
- stripe_subscription_id
- stripe_customer_id
- stripe_payment_intent_id
- stripe_invoice_id
- stripe_charge_id
- stripe_receipt_url
- card_last4, card_brand, payment_method
Donation fields:
- stripe_payment_intent_id
- stripe_charge_id
- stripe_receipt_url
- card_last4, card_brand
---
3. Fixed JSON Serialization Error
Problem: Object of type datetime is not JSON serializable when saving import metadata.
Solution: Added serialize_for_json() helper in backend/server.py:
def serialize_for_json(obj):
"""Recursively convert datetime objects to ISO strings for JSON serialization."""
if isinstance(obj, (datetime, date)):
return obj.isoformat()
elif isinstance(obj, dict):
return {k: serialize_for_json(v) for k, v in obj.items()}
elif isinstance(obj, list):
return [serialize_for_json(item) for item in obj]
# ... handles other types
---
4. Fixed Route Ordering (401 Unauthorized)
Problem: /admin/import/comprehensive/upload returned 401 because FastAPI matched "comprehensive" as a {job_id} parameter.
Solution: Moved comprehensive import routes BEFORE generic {job_id} routes in backend/server.py:
# Correct order:
@app.post("/api/admin/import/comprehensive/upload") # Specific route FIRST
# ... other comprehensive routes ...
@app.get("/api/admin/import/{job_id}/preview") # Generic route AFTER
---
5. Improved Date Parsing
Solution: Added additional date formats to backend/wordpress_parser.py:
formats = [
'%m/%d/%Y', '%Y-%m-%d', '%d/%m/%Y', '%B %d, %Y', '%b %d, %Y',
'%Y-%m-%d %H:%M:%S',
'%m/%Y', # Month/Year: 01/2020
'%m-%Y', # Month-Year: 01-2020
'%b-%Y', # Short month-Year: Jan-2020
'%B-%Y', # Full month-Year: January-2020
]
LOAF Membership Platform - Backend API
FastAPI-based backend service for the LOAF (LGBT Organization and Friends) membership management platform.
Table of Contents
- Setup & Installation
- Architecture & Code Structure
- API Documentation
- Deployment Guide
- Troubleshooting
Setup & Installation
Prerequisites
- Python: 3.9 or higher
- PostgreSQL: 13 or higher
- Cloudflare R2: Account and credentials (for file storage)
- Stripe: Account and API keys (for payments)
- SMTP Server: For sending emails (Gmail, SendGrid, etc.)
1. Clone Repository
cd backend
2. Create Virtual Environment
# Create virtual environment
python -m venv venv
# Activate virtual environment
# On macOS/Linux:
source venv/bin/activate
# On Windows:
venv\Scripts\activate
3. Install Dependencies
pip install -r requirements.txt
Key Dependencies:
fastapi==0.110.1- Web frameworkuvicorn==0.25.0- ASGI serversqlalchemy==2.0.44- ORM for databasepsycopg2-binary==2.9.10- PostgreSQL adapterpython-jose[cryptography]==3.5.0- JWT tokenspasslib[bcrypt]==1.7.4- Password hashingaiosmtplib==5.0.0- Async email sendingboto3==1.35.95- AWS SDK (for Cloudflare R2)stripe==11.4.0- Payment processingpython-multipart==0.0.18- File uploads
4. Database Setup
Create PostgreSQL Database
# Connect to PostgreSQL
psql -U postgres
# Create database
CREATE DATABASE membership_db;
# Create user (optional)
CREATE USER membership_user WITH PASSWORD 'your_password';
GRANT ALL PRIVILEGES ON DATABASE membership_db TO membership_user;
# Exit psql
\q
Run Migrations
⚠️ IMPORTANT: For complete migration documentation, see MIGRATIONS.md
For Fresh Database (New Deployment):
# Run comprehensive initial schema (includes all base tables)
psql -U postgres -d membership_db -f migrations/000_initial_schema.sql
This single migration creates:
- All 10 ENUM types (userstatus, userrole, rsvpstatus, etc.)
- All 17 base tables (users, events, subscriptions, donations, RBAC, etc.)
- 30+ performance indexes
- Default storage_usage record
For Existing Database (Incremental Updates):
If you already have a database and need to apply specific updates, see the migration sequence in MIGRATIONS.md.
Next Steps After Migration:
# 1. Seed permissions (59 permissions across 10 modules)
python seed_permissions_rbac.py
# 2. Create superadmin user (interactive)
python create_admin.py
5. Environment Configuration
Create .env file in the backend directory:
# Database
DATABASE_URL=postgresql://postgres:password@localhost:5432/membership_db
# JWT Configuration
JWT_SECRET=your-super-secret-key-change-in-production
JWT_ALGORITHM=HS256
ACCESS_TOKEN_EXPIRE_MINUTES=30
# SMTP Email Configuration
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USERNAME=your-email@gmail.com
SMTP_PASSWORD=your-app-password
SMTP_FROM_EMAIL=noreply@loaf.org
SMTP_FROM_NAME=LOAF Membership Platform
# Frontend URL (for email links and CORS)
FRONTEND_URL=http://localhost:3000
# Cloudflare R2 Storage
R2_ACCOUNT_ID=your-cloudflare-account-id
R2_ACCESS_KEY_ID=your-r2-access-key
R2_SECRET_ACCESS_KEY=your-r2-secret-key
R2_BUCKET_NAME=loaf-membership
R2_PUBLIC_URL=https://your-r2-bucket.r2.dev
# Stripe Payment Processing
STRIPE_SECRET_KEY=sk_test_your_stripe_secret_key
STRIPE_PUBLISHABLE_KEY=pk_test_your_stripe_publishable_key
STRIPE_WEBHOOK_SECRET=whsec_your_webhook_secret
# Server Configuration
PORT=8000
HOST=0.0.0.0
CORS_ORIGINS=http://localhost:3000,http://localhost:3001
# Optional: Sentry (error tracking)
SENTRY_DSN=your-sentry-dsn
Important Notes:
- Gmail SMTP: Use App Password, not your regular password. Enable 2FA and generate app-specific password at https://myaccount.google.com/apppasswords
- JWT_SECRET: Generate a strong random key:
openssl rand -hex 32 - Stripe Webhook Secret: Get from Stripe Dashboard <20> Developers <20> Webhooks
6. Seed Initial Data
# Seed permissions (must be run after migrations)
python seed_permissions_rbac.py
# Create superadmin user (optional)
python create_superadmin.py
7. Start Development Server
# Start with auto-reload
uvicorn server:app --reload --host 0.0.0.0 --port 8000
# Or using the PORT from .env
uvicorn server:app --reload --host 0.0.0.0 --port $PORT
Server will be available at:
- API: http://localhost:8000
- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc
Architecture & Code Structure
Project Structure
backend/
server.py # Main FastAPI app + all API routes (~4000 lines)
models.py # SQLAlchemy ORM models
auth.py # JWT utilities + password hashing
email_service.py # Async email sending with HTML templates
database.py # Database connection & session management
storage_service.py # Cloudflare R2 file upload/download
requirements.txt # Python dependencies
seed_permissions_rbac.py # RBAC permissions seeding script
migrations/ # Database migration SQL scripts
001_initial_schema.sql
009_create_donations.sql
010_add_rejection_fields.sql
.env # Environment variables (not in git)
Core Technologies
| Technology | Purpose | Version |
|---|---|---|
| FastAPI | Web framework | 0.110.1 |
| Uvicorn | ASGI server | 0.25.0 |
| SQLAlchemy | ORM for database | 2.0.44 |
| PostgreSQL | Relational database | 13+ |
| Python-JOSE | JWT token handling | 3.5.0 |
| Bcrypt | Password hashing | via passlib |
| Aiosmtplib | Async email sending | 5.0.0 |
| Boto3 | Cloudflare R2 client | 1.35.95 |
| Stripe | Payment processing | 11.4.0 |
Key Data Models
User Model
Status Flow:
pending_email <20> pending_validation <20> pre_validated <20> payment_pending <20> active
<20>
rejected / inactive
Key Fields:
- Authentication: email, password_hash
- Profile: first_name, last_name, phone, bio, profile_photo_url
- Status tracking: status, role, email_verified
- Rejection: rejection_reason, rejected_at, rejected_by
- Partner info: partner_first_name, partner_last_name, partner_is_member
Event Model
- Event management with RSVP tracking
- Fields: title, description, location, start_at, end_at, capacity, published
Subscription Model
- Links users to subscription plans
- Tracks payment status, billing cycle, donations
- Fields: stripe_subscription_id, base_subscription_cents, donation_cents
Donation Model
- Separate tracking for standalone donations
- Types: member donations vs public donations
- Fields: amount_cents, donation_type, status, donor_email, donor_name
RBAC Models
- Role: Dynamic roles (admin, finance, superadmin)
- Permission: Granular permissions (users.view, events.create, etc.)
- RolePermission: Many-to-many relationship
- UserRoleAssignment: User role assignments
Authentication System
JWT Token Structure:
{
"sub": "user@example.com",
"user_id": "uuid",
"role": "member",
"exp": 1234567890
}
Password Security:
- Bcrypt hashing with automatic salt
- Never store plaintext passwords
- Token expiration: 30 minutes (configurable)
Permission System (RBAC)
59 Granular Permissions across 10 modules:
users.*- User managementevents.*- Event managementsubscriptions.*- Subscription managementdonations.*- Donation trackingpermissions.*- RBAC administration
Example Usage:
@api_router.get("/admin/users")
async def get_users(
current_user: User = Depends(require_permission("users.view")),
db: Session = Depends(get_db)
):
# Only users with 'users.view' permission can access
...
Email Service
5 Email Templates:
- Email verification
- Approval notification
- Rejection with reason
- Donation thank you
- Payment prompt
All emails use branded HTML templates with LOAF color scheme.
Storage Service (Cloudflare R2)
- Profile photo uploads (max 50MB)
- Event cover images
- Gallery images
- Automatic URL generation with CDN
API Documentation
Base URL
Development: http://localhost:8000
Production: https://api.loaf.org
Authentication
Most endpoints require JWT token in Authorization header:
Authorization: Bearer <jwt_token>
Interactive Docs
- Swagger UI: http://localhost:8000/docs
- ReDoc: http://localhost:8000/redoc
Key Endpoint Groups
Authentication (/api/auth)
POST /auth/register- User registrationPOST /auth/login- Login with JWT tokenGET /auth/verify-email- Email verificationGET /auth/me- Current user profile
Member Profile (/api/members)
GET /members/profile- Get profilePUT /members/profile- Update profilePOST /members/profile/upload-photo- Upload profile pictureDELETE /members/profile/delete-photo- Delete profile picture
Events (/api/events)
GET /events- List published eventsGET /events/{id}- Event detailsPOST /events/{id}/rsvp- Submit RSVP
Admin Users (/api/admin/users)
GET /admin/users- List users (with filters)PUT /admin/users/{id}/approve- Approve userPOST /admin/users/{id}/reject- Reject with reasonGET /admin/users/export- Export to CSV
Admin Events (/api/admin/events)
GET /admin/events- List all eventsPOST /admin/events- Create eventPUT /admin/events/{id}- Update eventPUT /admin/events/{id}/attendance- Mark attendance
Subscriptions (/api/admin/subscriptions)
GET /admin/subscriptions- List subscriptionsGET /admin/subscriptions/stats- StatisticsGET /admin/subscriptions/export- Export to CSVPOST /admin/subscriptions/{id}/cancel- Cancel subscription
Donations (/api/admin/donations)
GET /admin/donations- List donationsGET /admin/donations/stats- StatisticsGET /admin/donations/export- Export to CSV
Payments (/api)
POST /checkout/subscription- Create Stripe checkoutPOST /donations/checkout- Create donation checkoutPOST /webhooks/stripe- Handle Stripe webhooks
Response Formats
Success:
{
"id": "uuid",
"field": "value"
}
Error:
{
"detail": "Error message"
}
Status Codes:
- 200 - Success
- 201 - Created
- 400 - Bad Request
- 401 - Unauthorized
- 403 - Forbidden
- 404 - Not Found
- 500 - Server Error
Deployment Guide
Production Setup
1. Server Requirements
- Ubuntu 20.04 LTS+
- Python 3.9+
- PostgreSQL 13+
- 2GB+ RAM
- 20GB+ SSD
2. Install Dependencies
sudo apt update && sudo apt upgrade -y
sudo apt install python3.9 python3.9-venv python3-pip postgresql nginx -y
3. Application Setup
# Create directory
sudo mkdir -p /var/www/loaf-backend
cd /var/www/loaf-backend
# Setup virtual environment
python3.9 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
4. Configure Environment
Create production .env with:
- Strong JWT_SECRET (64+ chars)
- Production database credentials
- Production SMTP (SendGrid recommended)
- Cloudflare R2 production credentials
- Stripe LIVE API keys
- HTTPS frontend URL
5. Database Setup
# Create production database
sudo -u postgres psql
CREATE DATABASE membership_prod;
CREATE USER membership_prod WITH ENCRYPTED PASSWORD 'strong_password';
GRANT ALL PRIVILEGES ON DATABASE membership_prod TO membership_prod;
# Run migrations
for file in migrations/*.sql; do
psql postgresql://membership_prod:password@localhost/membership_prod -f "$file"
done
# Seed permissions
python seed_permissions_rbac.py
6. Systemd Service
Create /etc/systemd/system/loaf-backend.service:
[Unit]
Description=LOAF Backend API
After=network.target postgresql.service
[Service]
Type=notify
User=www-data
Group=www-data
WorkingDirectory=/var/www/loaf-backend
Environment="PATH=/var/www/loaf-backend/venv/bin"
ExecStart=/var/www/loaf-backend/venv/bin/uvicorn server:app --host 0.0.0.0 --port 8000 --workers 4
Restart=always
[Install]
WantedBy=multi-user.target
Start service:
sudo systemctl enable loaf-backend
sudo systemctl start loaf-backend
7. Nginx Reverse Proxy
server {
listen 80;
server_name api.loaf.org;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
8. SSL Certificate
sudo apt install certbot python3-certbot-nginx -y
sudo certbot --nginx -d api.loaf.org
9. Configure Stripe Webhook
- Stripe Dashboard <20> Webhooks
- Add endpoint:
https://api.loaf.org/api/webhooks/stripe - Events: checkout.session.completed, invoice.payment_succeeded
- Copy webhook secret to .env
Monitoring
# View logs
sudo journalctl -u loaf-backend -f
# Database backup (daily cron)
pg_dump membership_prod | gzip > backup_$(date +%Y%m%d).sql.gz
Security Checklist
- Strong passwords and JWT secret
- HTTPS with valid SSL
- Firewall configured
- SSH key authentication only
- Environment variables secured
- CORS restricted to production domain
- Stripe webhook verification enabled
- Regular backups configured
- Error tracking (Sentry) enabled
Troubleshooting
Database Connection Error
# Check PostgreSQL status
sudo systemctl status postgresql
# Verify DATABASE_URL format
postgresql://user:password@host:port/database
Email Not Sending
- Gmail: Use App Password (not regular password)
- Enable 2FA first
- Generate App Password at https://myaccount.google.com/apppasswords
JWT Token Invalid
- Check JWT_SECRET is consistent
- Token expires after 30 minutes
- User needs to log in again
Stripe Webhook Failing
- Verify endpoint is publicly accessible
- Check STRIPE_WEBHOOK_SECRET matches dashboard
- Review webhook logs in Stripe dashboard
File Upload Fails
- Check R2 credentials in .env
- Verify bucket exists and has write permissions
- Test connection with AWS CLI
Additional Resources
- FastAPI Docs: https://fastapi.tiangolo.com/
- Stripe API: https://stripe.com/docs/api
- Cloudflare R2: https://developers.cloudflare.com/r2/
- Project Context: See
CLAUDE.mdandPRD.md
Last Updated: December 18, 2024 Version: 1.0.0 Maintainer: LOAF Development Team