Security & Privacy

📖 17 min read 📄 Part 9 of 10

CDN Network - Security and Privacy

DDoS Mitigation at Edge

Volumetric Attacks (Layer 3/4)

Attack types:
- UDP flood: massive volume of UDP packets to saturate bandwidth
- SYN flood: exhaust server connection table with half-open connections
- DNS amplification: spoofed DNS queries generating large responses
- NTP amplification: exploiting NTP monlist for 500x amplification
- Memcached amplification: up to 50,000x amplification factor

CDN defense mechanisms:
1. Distributed absorption:
   - 200+ PoPs each absorb portion of attack traffic
   - Aggregate capacity: 100+ Tbps (exceeds largest known attacks)
   - Attack traffic distributed globally, no single point overwhelmed
   - Largest recorded attacks: 2-3 Tbps (well within CDN capacity)

2. Anycast scrubbing:
   - Attack traffic routed to nearest PoP via anycast
   - Scrubbing applied at network edge (before reaching servers)
   - Clean traffic forwarded to application servers
   - Dirty traffic dropped at line rate using hardware filters

3. BGP-based mitigation:
   - Detect attack via traffic anomaly detection (volume, packet rate)
   - Announce more-specific routes to attract traffic to scrubbing centers
   - Apply filtering rules (ACLs, flowspec)
   - Withdraw routes from overwhelmed PoPs

4. Rate limiting at network layer:
   - Per-IP packet rate limits (e.g., 10,000 pps per source IP)
   - Per-prefix bandwidth limits
   - Protocol-specific limits (UDP vs TCP ratios)
   - Automatic escalation: soft limit → hard limit → blackhole

Capacity planning:
- Design for 10x largest observed attack
- Maintain 50%+ headroom during attack
- Auto-scale scrubbing capacity within 10 seconds
- Geographic diversity: no single region handles >30% of attack

Protocol Attacks (Layer 4)

SYN flood mitigation:
- SYN cookies: stateless SYN handling (no connection table entry)
- SYN proxy: CDN completes handshake before forwarding to origin
- Connection rate limiting: max new connections per IP per second
- Idle connection timeout: aggressive cleanup (30-60 seconds)

TCP state exhaustion defense:
- Connection pooling to origin (limit total connections)
- Per-IP connection limits (e.g., 256 concurrent connections)
- Slowloris protection: minimum data rate requirements
- Keep-alive limits: max requests per connection

SSL/TLS abuse:
- TLS handshake rate limiting per IP
- Incomplete handshake detection and blocking
- Session ticket reuse to reduce handshake cost
- Hardware TLS offload (dedicated crypto accelerators)

Application Layer Attacks (Layer 7)

HTTP flood mitigation:
- Request rate limiting per IP/session/API key
- Challenge-response (CAPTCHA, JavaScript challenges)
- Behavioral analysis (request patterns, timing)
- Machine learning anomaly detection

Specific L7 attack patterns:
- GET floods: high volume of legitimate-looking requests
- POST floods: resource-intensive form submissions
- Slowloris: keep connections open with slow headers
- R-U-Dead-Yet (RUDY): slow POST body transmission
- Cache-busting: random query params to bypass cache

Defense layers:
Layer 1: Rate limiting (IP-based, 1000 req/min default)
Layer 2: Reputation scoring (known bad IPs, ASNs, user agents)
Layer 3: Behavioral analysis (request patterns over time)
Layer 4: Challenge (JavaScript proof-of-work, CAPTCHA)
Layer 5: Block (IP ban for persistent attackers)

Adaptive response:
- Normal: no challenges, full speed
- Elevated: JavaScript challenge for suspicious IPs
- Attack: CAPTCHA for all new visitors, aggressive rate limits
- Emergency: allow-list only, block all unknown traffic

WAF at CDN Layer

OWASP Top 10 Protection

Rule sets covering:
1. Injection (SQL, NoSQL, OS command, LDAP):
   - Pattern matching on query params, headers, body
   - Parameterized query enforcement
   - Example: block requests containing "UNION SELECT", "'; DROP TABLE"

2. Broken Authentication:
   - Brute force detection (failed login rate limiting)
   - Credential stuffing protection
   - Session fixation prevention

3. Sensitive Data Exposure:
   - Block responses containing credit card patterns
   - Prevent SSN/PII leakage in responses
   - Enforce HTTPS (redirect HTTP to HTTPS)

4. XML External Entities (XXE):
   - Block XML with DOCTYPE declarations
   - Disable external entity processing
   - Limit XML parsing depth

5. Broken Access Control:
   - Path traversal detection (../, %2e%2e)
   - Method enforcement (block unexpected HTTP methods)
   - IDOR pattern detection

6. Security Misconfiguration:
   - Block access to sensitive files (.env, .git, wp-config.php)
   - Remove server version headers
   - Enforce security headers in responses

7. Cross-Site Scripting (XSS):
   - Input sanitization patterns
   - Block <script>, javascript:, on* event handlers
   - Content-Security-Policy enforcement

8. Insecure Deserialization:
   - Block known serialized object patterns
   - Content-Type validation
   - Request body size limits

9. Using Components with Known Vulnerabilities:
   - Virtual patching for known CVEs
   - Block exploit payloads for unpatched software
   - Scanner detection (Nessus, Nikto signatures)

10. Insufficient Logging & Monitoring:
    - Log all WAF decisions (allow, block, challenge)
    - Alert on rule trigger rate spikes
    - Correlation with threat intelligence feeds

Custom WAF Rules

Rule language example (pseudo-code):
rule "Block suspicious user agents" {
    condition: request.headers["User-Agent"] matches
        "(sqlmap|nikto|nessus|masscan|zgrab)"
    action: BLOCK
    priority: 1
}

rule "Rate limit API endpoints" {
    condition: request.path starts_with "/api/"
        AND request.rate_per_ip > 100 per minute
    action: CHALLENGE
    priority: 5
}

rule "Geo-block sanctioned countries" {
    condition: request.geo.country IN ["KP", "IR", "SY", "CU"]
    action: BLOCK
    priority: 0
}

rule "Protect admin panel" {
    condition: request.path starts_with "/admin/"
        AND request.ip NOT IN allowed_admin_ips
    action: BLOCK
    priority: 2
}

Rule evaluation order:
1. Allow-list rules (highest priority, bypass all other rules)
2. Block-list rules (known bad actors)
3. Rate limiting rules
4. OWASP managed rules
5. Custom rules (customer-defined)
6. Default action (allow)

Performance impact:
- Rule evaluation: <1ms for typical rule sets (100-500 rules)
- Regex matching: optimized with Hyperscan or RE2
- No measurable latency impact for cache hits
- 1-3ms additional latency for complex rule sets on cache misses

Bot Detection and Management

Bot Classification

Categories:
1. Verified bots (good): Googlebot, Bingbot, monitoring services
   - Action: Allow, possibly serve cached/optimized content
   - Verification: reverse DNS + forward DNS confirmation

2. Unverified bots (unknown): scrapers, SEO tools, custom crawlers
   - Action: Challenge, rate limit, or serve alternate content
   - Detection: user agent analysis, behavior patterns

3. Malicious bots (bad): credential stuffing, DDoS, spam
   - Action: Block, CAPTCHA, or tarpit (slow response)
   - Detection: ML models, threat intelligence, behavior analysis

Detection signals:
- TLS fingerprint (JA3/JA4): bots often have unique TLS stacks
- HTTP/2 fingerprint: frame ordering, settings, priorities
- JavaScript execution: bots often can't execute JS
- Mouse/keyboard patterns: absence indicates automation
- Request timing: too regular = bot, too fast = bot
- Navigation patterns: no referrer, no asset loading
- Cookie handling: bots often don't maintain cookies
- Canvas/WebGL fingerprint: headless browsers detectable

Bot Score Implementation

Scoring model (0.0 = definitely human, 1.0 = definitely bot):

Input features:
- TLS fingerprint match to known bot libraries: +0.3
- No JavaScript execution capability: +0.25
- Request rate > 10 req/s from single IP: +0.2
- User agent matches known bot pattern: +0.15
- No referrer on non-entry pages: +0.1
- IP from hosting/cloud provider ASN: +0.1
- Abnormal header ordering: +0.1
- Failed previous challenges: +0.3

Thresholds:
- Score < 0.3: Human (allow)
- Score 0.3-0.6: Suspicious (monitor, soft challenge)
- Score 0.6-0.8: Likely bot (hard challenge, rate limit)
- Score > 0.8: Definite bot (block or CAPTCHA)

Actions by score:
- Allow: normal request processing
- Challenge: JavaScript challenge (invisible to humans)
- CAPTCHA: visual challenge (blocks most bots)
- Rate limit: slow down requests (1 req/5s)
- Block: return 403 Forbidden
- Tarpit: accept request but respond very slowly

SSL/TLS Best Practices

TLS 1.3 Configuration

Cipher suites (in preference order):
1. TLS_AES_256_GCM_SHA384
2. TLS_CHACHA20_POLY1305_SHA256
3. TLS_AES_128_GCM_SHA256

Key exchange:
- X25519 (preferred): fastest, most secure ECDH
- P-256: widely supported fallback
- Post-quantum: X25519Kyber768 (experimental, future-proofing)

TLS 1.3 advantages for CDN:
- 1-RTT handshake (vs 2-RTT for TLS 1.2)
- 0-RTT resumption (for repeat visitors)
- Simplified cipher suite negotiation
- Forward secrecy mandatory (no RSA key exchange)
- Encrypted handshake (SNI still visible, ECH coming)

0-RTT considerations:
- Enable for GET requests only (replay-safe)
- Disable for POST/PUT (replay attack risk)
- Implement anti-replay mechanisms (single-use tickets)
- Accept 0-RTT data only for idempotent operations

HSTS (HTTP Strict Transport Security)

Header configuration:
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

Implementation at CDN:
- Add HSTS header to all HTTPS responses
- max-age: 1 year minimum for preload eligibility
- includeSubDomains: protect all subdomains
- preload: submit to browser preload lists

Deployment strategy:
1. Start with short max-age (300 seconds) to test
2. Increase to 1 week, monitor for issues
3. Increase to 1 month, add includeSubDomains
4. Set to 1 year, add preload directive
5. Submit to hstspreload.org

CDN-specific considerations:
- Apply HSTS at edge (don't rely on origin)
- Ensure all alternate domains support HTTPS
- Certificate must cover all subdomains
- Cannot easily roll back once preloaded

Certificate Management

Best practices:
- Use ECDSA P-256 certificates (smaller, faster than RSA)
- Dual certificates: ECDSA primary, RSA fallback for old clients
- Short-lived certificates (90 days, auto-renewed)
- OCSP stapling: include OCSP response in TLS handshake
- CT (Certificate Transparency) logging: detect misissued certs
- CAA records: restrict which CAs can issue for your domain

OCSP stapling implementation:
- Fetch OCSP response from CA every 12 hours
- Cache stapled response at edge (valid for 7 days typically)
- Serve stapled response during TLS handshake
- Eliminates client-side OCSP lookup (saves 100-300ms)
- Fallback: if staple unavailable, client does soft-fail OCSP

Certificate pinning (deprecated for web, still used in apps):
- Pin intermediate CA certificate (not leaf)
- Include backup pins for rotation
- Use Expect-CT header instead for web
- Mobile apps: pin in app binary, update via app store

Origin Protection

Origin Cloaking

Problem: If attackers discover origin IP, they bypass CDN protections

Techniques:
1. Never expose origin IP in DNS:
   - Origin uses private IP or internal hostname
   - Only CDN edge servers know origin IP
   - No A records pointing to origin in public DNS

2. Firewall origin to CDN-only:
   - Allow inbound only from CDN IP ranges
   - Block all other inbound traffic
   - CDN publishes IP ranges (update firewall automatically)
   - Example: AWS CloudFront publishes IP ranges in JSON

3. Private connectivity:
   - AWS PrivateLink / VPC peering between CDN and origin
   - Direct Connect / dedicated links
   - No public internet exposure for origin

4. Origin hostname obfuscation:
   - Use random/non-guessable origin hostname
   - Don't use origin.example.com (too obvious)
   - Rotate origin hostname periodically

Authenticated Origin Pulls

Mechanism: CDN presents client certificate when connecting to origin

Implementation:
1. Generate client certificate for CDN
2. Configure origin to require client cert (mTLS)
3. Origin validates: only requests with valid CDN cert are accepted
4. Attackers connecting directly are rejected (no valid cert)

Configuration:
- CDN side: configure client certificate per distribution
- Origin side: nginx/Apache require client cert verification
- Certificate rotation: automated, zero-downtime rotation
- Fallback: if cert validation fails, return 403 (not 5xx)

Additional origin auth methods:
- Shared secret header: X-CDN-Secret: <random-token>
- IP allowlist: only accept from CDN IP ranges
- JWT validation: CDN signs requests with private key
- Mutual TLS: strongest option (certificate-based)

Token Authentication for Content Access

Signed URLs

Format:
https://cdn.example.com/premium/video.mp4
  ?token=abc123def456
  &expires=1705312800
  &ip=203.0.113.0/24
  &signature=hmac_sha256(secret, url+expires+ip)

Parameters:
- token: unique identifier for this access grant
- expires: Unix timestamp when URL becomes invalid
- ip: restrict to client IP or subnet (optional)
- signature: HMAC-SHA256 proving URL was generated by authorized party

Generation (server-side):
  secret_key = "cdn-signing-key-v1"
  url_to_sign = "/premium/video.mp4"
  expiry = current_time + 3600  # 1 hour
  message = f"{url_to_sign}|{expiry}|{client_ip}"
  signature = hmac_sha256(secret_key, message)
  signed_url = f"{base_url}{url_to_sign}?expires={expiry}&sig={signature}"

Validation (at CDN edge):
  1. Extract signature and parameters from URL
  2. Check expiry (reject if expired)
  3. Check IP restriction (reject if mismatch)
  4. Recompute signature with same secret key
  5. Compare signatures (timing-safe comparison)
  6. If valid: serve content; if invalid: return 403

Use cases:
- Premium content (paid video, downloads)
- Time-limited access (24-hour download links)
- Per-user access control without cookies
- Preventing hotlinking from unauthorized sites

Signed Cookies

Advantages over signed URLs:
- Single cookie grants access to multiple resources
- URL remains clean (shareable without leaking access)
- Works with relative URLs and redirects
- Better for streaming (multiple segment requests)

Cookie format:
Set-Cookie: cdn_access=policy_base64.signature; 
  Domain=.cdn.example.com; 
  Path=/premium/; 
  Secure; 
  HttpOnly; 
  SameSite=Strict;
  Expires=Thu, 15 Jan 2025 12:00:00 GMT

Policy (JSON, base64-encoded):
{
  "statement": [{
    "resource": "https://cdn.example.com/premium/*",
    "condition": {
      "date_less_than": 1705312800,
      "ip_address": "203.0.113.0/24"
    }
  }]
}

Validation at edge:
1. Extract cookie from request
2. Decode policy (base64)
3. Verify signature against policy
4. Check conditions (expiry, IP, resource pattern)
5. If valid: serve content; if invalid: redirect to auth page

Geo-Blocking and Compliance

GDPR Data Residency

Requirements:
- EU user data must be processed within EU/EEA (or adequate countries)
- Access logs containing EU user IPs are personal data
- Right to erasure applies to CDN logs
- Data Processing Agreements (DPA) required with CDN provider

CDN compliance implementation:
1. Regional log storage:
   - EU user logs stored only in EU data centers
   - Separate log pipelines per region
   - No cross-border log transfer without legal basis

2. Data minimization:
   - Truncate/hash IP addresses in logs after 24 hours
   - Retain only aggregated analytics long-term
   - Delete raw logs after retention period (30-90 days)

3. Right to erasure:
   - API to delete specific user's data from logs
   - Propagate deletion to all log storage tiers
   - Confirm deletion within 30 days

4. Edge processing:
   - Process EU requests only at EU PoPs
   - Geo-fence: never route EU traffic to non-EU PoPs
   - Configuration: restrict distribution to EU-only price class

Geo-Blocking Implementation

Methods:
1. IP-based geolocation:
   - MaxMind GeoIP2 database (updated weekly)
   - Accuracy: 99.8% at country level, 80% at city level
   - Edge lookup: <0.1ms (in-memory database)

2. Blocking modes:
   - Whitelist: only allow specified countries
   - Blacklist: block specified countries
   - Custom response: return specific page per blocked country

3. Configuration:
   {
     "geo_restriction": {
       "type": "blacklist",
       "countries": ["KP", "IR", "SY", "CU", "RU"],
       "action": "block",
       "custom_error_page": "/geo-blocked.html"
     }
   }

4. Compliance considerations:
   - Sanctions compliance (OFAC, EU sanctions)
   - Content licensing (media rights per territory)
   - Age-restricted content (varies by jurisdiction)
   - Gambling regulations (per-state in US)

Evasion and limitations:
- VPN/proxy users appear in VPN exit country
- Tor exit nodes may be in any country
- IP geolocation databases have errors
- Mobile users may roam across borders
- Mitigation: combine geo-IP with other signals

Content Integrity

Subresource Integrity (SRI)

Purpose: Ensure CDN-delivered scripts/styles haven't been tampered with

HTML implementation:
<script src="https://cdn.example.com/app.js"
        integrity="sha384-oqVuAfXRKap7fdgcCY5uykM6+R9GqQ8K/uxy9rx7HNQlGYl1kPzQho1wx4JwY8wC"
        crossorigin="anonymous"></script>

How it works:
1. Developer computes hash of file content
2. Hash included in HTML integrity attribute
3. Browser fetches file from CDN
4. Browser computes hash of received content
5. If hashes match: execute; if mismatch: block and report

CDN considerations:
- CDN must NOT modify content (no minification, no compression changes)
- Use "no-transform" Cache-Control directive
- Content-Encoding must be consistent (hash is of decoded content)
- Version URLs recommended (hash changes with content)

Limitations:
- Only works for <script> and <link> tags
- Doesn't protect dynamically loaded resources
- Requires updating HTML when CDN content changes
- Not applicable to images or media

Signed URLs and Content Verification

Content signing (origin signs, edge verifies):
1. Origin computes signature over response body
2. Signature included in response header: X-Content-Signature
3. CDN edge can verify content wasn't corrupted in transit
4. Client can verify content wasn't modified by CDN

Implementation:
  # Origin signs content
  signature = ed25519_sign(private_key, response_body)
  response.headers["X-Content-Signature"] = base64(signature)
  response.headers["X-Content-Key-Id"] = "key-2025-01"

  # Edge/client verifies
  public_key = lookup_key("key-2025-01")
  valid = ed25519_verify(public_key, response_body, signature)

Use cases:
- High-security content (financial data, legal documents)
- Regulatory requirements (tamper-evident delivery)
- Supply chain security (software distribution)
- Detecting CDN-level content injection attacks

Cache Poisoning Prevention

Attack: Attacker tricks CDN into caching malicious content

Attack vectors:
1. Host header poisoning:
   - Attacker sends request with manipulated Host header
   - Origin generates response with attacker-controlled URLs
   - CDN caches poisoned response, serves to all users

2. Unkeyed header injection:
   - Headers not in cache key but affecting response
   - X-Forwarded-Host, X-Original-URL, X-Rewrite-URL
   - CDN caches response generated with attacker's headers

3. HTTP request smuggling:
   - Exploit differences in HTTP parsing between CDN and origin
   - Smuggle second request that poisons cache

Prevention:
1. Normalize and validate Host header at edge
2. Include all response-affecting headers in cache key
3. Strip dangerous headers before forwarding to origin
4. Implement request smuggling detection:
   - Reject ambiguous Content-Length/Transfer-Encoding
   - Normalize HTTP parsing (strict mode)
   - Reject requests with multiple Content-Length headers
5. Cache key must include all Vary header values
6. Validate response before caching (no unexpected redirects)
7. Implement cache isolation per customer/distribution

Security Monitoring and Incident Response

Real-Time Threat Detection

Monitoring signals:
- Request rate anomalies (per IP, per path, per distribution)
- Error rate spikes (4xx, 5xx patterns)
- Geographic anomalies (traffic from unusual regions)
- Bot score distribution shifts
- WAF rule trigger rates
- TLS error rates (potential downgrade attacks)
- Cache hit ratio drops (potential cache-busting attack)

Alert thresholds:
- DDoS: >10x normal traffic volume in 60 seconds
- Credential stuffing: >100 failed logins per IP per minute
- Scanning: >50 404s per IP per minute
- Data exfiltration: >1 GB response to single IP in 5 minutes
- WAF bypass attempts: >10 blocked requests per IP per second

Automated response:
1. Detect anomaly (ML model or threshold)
2. Classify threat type
3. Apply mitigation (rate limit, challenge, block)
4. Alert security team
5. Collect forensic data (full request logs)
6. Escalate if mitigation insufficient

Security Headers (CDN-Injected)

Headers added at edge for all responses:
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 0
Referrer-Policy: strict-origin-when-cross-origin
Permissions-Policy: camera=(), microphone=(), geolocation=()
Content-Security-Policy: default-src 'self'; script-src 'self' cdn.example.com

CDN-specific security headers:
X-CDN-Request-Id: unique-request-identifier
X-CDN-Edge: edge-location-identifier
X-CDN-Cache-Status: HIT|MISS|BYPASS

Headers removed at edge (never expose to client):
X-Powered-By: (removed)
Server: (replaced with generic)
X-AspNet-Version: (removed)
X-Debug-*: (removed in production)

Privacy-Preserving CDN Operations

Log Anonymization

Techniques:
1. IP truncation: Store only /24 (IPv4) or /48 (IPv6)
   - 203.0.113.45 → 203.0.113.0
   - Preserves geographic/ASN info, removes individual identity

2. IP hashing: One-way hash with daily rotating salt
   - hash(salt + IP) → pseudonymous identifier
   - Allows session correlation within a day
   - Cannot reverse to original IP

3. Differential privacy: Add noise to aggregated metrics
   - Individual user behavior not extractable
   - Aggregate statistics remain accurate
   - Epsilon parameter controls privacy/utility tradeoff

4. Data minimization:
   - Don't log full User-Agent (just browser family + version)
   - Don't log full URL query strings (may contain PII)
   - Don't log request bodies
   - Truncate referrer URLs

Retention policy:
- Full logs (with IP): 24-72 hours (real-time debugging)
- Anonymized logs: 30-90 days (analytics)
- Aggregated metrics: 2-5 years (capacity planning)
- Delete on request: within 30 days (GDPR right to erasure)

Privacy-Preserving Analytics

Techniques for analytics without tracking:
1. Aggregate-only metrics:
   - Count requests per URL (no per-user breakdown)
   - Bandwidth per region (not per IP)
   - Cache hit ratio (no individual request tracking)

2. On-edge aggregation:
   - Compute statistics at edge, send only aggregates to central
   - No raw request data leaves the edge
   - Reduces data transfer and privacy exposure

3. Federated analytics:
   - Each PoP computes local statistics
   - Central system combines without accessing raw data
   - Similar to federated learning approach

4. Client-side privacy:
   - Support Do Not Track (DNT) header
   - Respect Global Privacy Control (GPC)
   - Minimal cookies (no third-party tracking)
   - No fingerprinting techniques