BreachSpider Incident Response Plan

CITED Relevance LLC - Confidential
Last Updated: June 7, 2026
Owner: Joshua Hayes, [email protected]

1. Scope

This plan covers security incidents affecting breachspider.com, api.breachspider.com, docs.breachspider.com, and the underlying OVHcloud VPS infrastructure at 15.204.242.67.

2. Incident Classification

S1 notification requirement: Affected customers notified within 72 hours of confirmed Personal Data breach (GDPR Article 33 / contractual DPA obligation).

3. Detection Sources

• API error rate spikes in /var/log/breachspider/api.log
• Service health endpoint degradation: GET /api/v1/health
• Uptime monitor alerts (Uptime Robot - configured for 1-minute checks)
• Customer reports to [email protected]
• Cloudflare security alerts and WAF triggered events
• OVHcloud infrastructure alerts
• Audit log anomalies in admin_audit_logs table (unusual actions, off-hours logins)
• Alert engine log at /var/log/breachspider/alerts.log

4. Response Procedures

4.1 Initial Assessment (0-30 minutes)

1. Confirm the incident is real (not a false positive from monitoring)
2. Classify severity (S1-S4)
3. Document start time in UTC
4. Create incident log at /root/incidents/YYYY-MM-DD-incident.md
5. Capture initial state: systemctl status breachspider-api.service, tail -100 /var/log/breachspider/api.log

4.2 Containment

S1 - Critical:

# Stop the API service immediately
systemctl stop breachspider-api.service

# Enable Cloudflare Under Attack mode (via Cloudflare dashboard)
# Security > Settings > Security Level: I'm Under Attack

# Revoke all active API keys as precaution
PGPASSWORD=... psql -h 127.0.0.1 -p 5433 -U postgres breachspider_db \
  -c "UPDATE api_keys SET revoked_at=NOW() WHERE revoked_at IS NULL;"

# Rotate DB password immediately
# Update /etc/breachspider/env with new DB_API_PASS
# Run: ALTER USER bs_api WITH PASSWORD 'newpassword';

S2-S3: - Monitor without interrupting service if possible - Enable enhanced logging: set log level to DEBUG in service config - Review recent audit log entries in admin_audit_logs - Consider temporarily enabling Cloudflare "High" security level

4.3 Investigation

# Review recent API errors
grep "ERROR\|CRITICAL" /var/log/breachspider/api.log | tail -200

# Review nginx access log for suspicious patterns
grep -E "(POST|PUT|DELETE|PATCH)" /var/log/nginx/access.log | \
  awk '{print $1, $7, $9}' | sort | uniq -c | sort -rn | head -50

# Review audit log for anomalous actions
PGPASSWORD=... psql -h 127.0.0.1 -p 5433 -U postgres breachspider_db -c "
SELECT action, actor_email, ip_address, resource_type, resource_id, created_at
FROM admin_audit_logs
WHERE created_at > NOW() - INTERVAL '24 hours'
ORDER BY created_at DESC LIMIT 100;"

# Check for active sessions that look suspicious
PGPASSWORD=... psql -h 127.0.0.1 -p 5433 -U postgres breachspider_db -c "
SELECT s.token_hash, m.email, s.created_at, s.last_used_at, s.ip_address
FROM sessions s JOIN members m ON m.id = s.member_id
WHERE s.expires_at > NOW()
ORDER BY s.last_used_at DESC LIMIT 50;"

4.4 Customer Notification

S1 - Confirmed Personal Data breach: - Notify affected customers within 72 hours of confirmation - Email from [email protected] - Notification must include: - What happened (factual, no speculation) - What data was affected (specific categories and volume) - Timeline of incident - What we have done to contain it - What you should do (change any passwords, monitor accounts) - Contact for questions: [email protected]

S2-S3: - Post status update to platform status page within 24 hours - Email affected customers if their data was involved

4.5 Recovery

1. Restore from last known good backup if data integrity is in question
2. Verify database integrity after restore
3. Rotate all credentials before restoring public access:
4. PostgreSQL passwords (bs_api, bs_alerts, postgres)
5. RESEND_API_KEY
6. Stripe API keys
7. Any active API keys used by customers
8. Restore service with enhanced monitoring enabled
9. Monitor for 24 hours post-restoration before declaring incident closed

4.6 Post-Incident (within 7 days)

• Complete incident log with full timeline
• Root cause analysis document
• Update this plan if gaps were identified
• Add to CHANGELOG.md with sanitized incident summary
• Review and update security measures as needed

5. Contacts

6. Backup and Recovery

Backup location: Hostinger FTP offsite + /root/breachspider_backups/ local
Backup schedule: Nightly at 1am UTC
Backup retention: 90 days

Recovery procedure (clean VPS): 1. Provision clean Ubuntu 24.04 VPS 2. Restore PostgreSQL from latest backup: pg_restore from FTP backup 3. Deploy application code from Git repository 4. Restore /etc/breachspider/env from secure storage (1Password/offline vault) 5. Restore nginx config from Git or backup 6. Verify health endpoint: curl https://breachspider.com/api/v1/health 7. Run smoke tests before restoring DNS or removing Cloudflare maintenance page

RTO: 4 hours
RPO: 24 hours (last nightly backup)

7. Evidence Preservation

For S1-S2 incidents, preserve evidence before any remediation:

# Capture full log state
cp /var/log/breachspider/api.log /root/incidents/evidence/api.log.$(date +%s)
cp /var/log/nginx/access.log /root/incidents/evidence/nginx.$(date +%s)

# Dump relevant DB tables for forensic analysis
PGPASSWORD=... pg_dump -h 127.0.0.1 -p 5433 -U postgres breachspider_db \
  -t admin_audit_logs -t sessions -t api_keys \
  > /root/incidents/evidence/db_dump.$(date +%s).sql