How it works:
  - Operates on TCP/UDP packets
  - Routes based on IP addresses and port numbers
  - No inspection of application payload
  - Forwards raw TCP streams or UDP datagrams

Decision point: 5-tuple hash (src_ip, src_port, dst_ip, dst_port, protocol)

Packet flow:
  Client -> [SYN] -> LB -> [SYN to selected backend] -> Backend
  Client <- [SYN-ACK] <- LB <- [SYN-ACK] <- Backend
  Client -> [Data] -> LB -> [Data forwarded] -> Backend

How it works:
  - Terminates client TCP connection at LB
  - Parses HTTP/gRPC/WebSocket protocol
  - Makes routing decision based on request content
  - Opens new connection to selected backend

Decision point: URL path, Host header, cookies, headers, method, query params

Request flow:
  Client -> [TLS + HTTP Request] -> LB (terminates TLS, parses HTTP)
  LB -> [New connection + forwarded request] -> Backend
  Backend -> [Response] -> LB -> [Response to client] -> Client

Use L4 when:
  - Non-HTTP protocols (databases, Redis, Kafka, gaming)
  - Maximum performance needed (>1M RPS per instance)
  - Minimal latency requirement (<0.5ms overhead)
  - Backend handles TLS (end-to-end encryption required)
  - Simple round-robin or hash-based distribution sufficient

Use L7 when:
  - HTTP/gRPC traffic requiring content-based routing
  - SSL termination needed (centralized cert management)
  - Multiple services behind single domain (path-based routing)
  - Advanced features needed (WAF, rate limiting, caching)
  - Canary deployments or A/B testing
  - Detailed request-level observability required

Hybrid approach (common in production):
  L4 LB (front) -> L7 LB (middle) -> Backend servers
  - L4 handles raw TCP distribution across L7 instances
  - L7 handles application-level routing and features
  - Example: AWS NLB -> ALB, or Maglev -> Envoy

Examples: F5 BIG-IP, Citrix ADC (NetScaler), A10 Networks

Architecture:
  - Custom ASICs for packet processing
  - Dedicated hardware with specialized network cards
  - Proprietary operating system
  - Appliance form factor (rack-mounted)

Performance characteristics:
  - Throughput: 100+ Gbps per appliance
  - Connections: 10M+ concurrent
  - SSL: Hardware acceleration (dedicated crypto chips)
  - Latency: <100μs added

Cost:
  - Entry level: $50,000-$100,000
  - Mid-range: $100,000-$500,000
  - High-end: $500,000-$2,000,000
  - Annual support: 15-25% of purchase price
  - Typical TCO: $200K-$500K/year per appliance pair

Examples: Nginx, HAProxy, Envoy, Traefik, Caddy, Katran

Architecture:
  - Runs on commodity servers or VMs
  - Standard Linux kernel or kernel bypass (DPDK/XDP)
  - Open-source or commercial licensing
  - Containerized or bare-metal deployment

Performance characteristics:
  - Throughput: 1-40 Gbps per instance (hardware dependent)
  - Connections: 100K-1M per instance
  - SSL: Software crypto (or hardware offload via QAT)
  - Latency: 0.5-5ms added (depending on features)

Cost:
  - Open-source: Free (operational cost only)
  - Commercial (Nginx Plus, HAProxy Enterprise): $5K-$50K/year
  - Infrastructure: $200-$500/month per instance
  - Typical TCO: $50K-$200K/year for equivalent capacity

                        Hardware LB          Software LB
Cost (5-year TCO):      $1M-$5M             $250K-$1M
Max throughput:         100+ Gbps            10-40 Gbps/instance (scales out)
Scaling model:          Scale-up             Scale-out
Provisioning time:      Weeks                Minutes
Flexibility:            Low                  High
Vendor lock-in:         High                 Low
Cloud compatibility:    Poor                 Excellent
Auto-scaling:           No                   Yes
Programmability:        Limited              Extensive

Recommendation:
  - Enterprise/regulated (banks, healthcare): Hardware or managed cloud
  - Cloud-native/startup: Software (Nginx/Envoy/HAProxy)
  - Hyperscale (FAANG): Custom software (Maglev, Katran, GLB)

Architecture:
  Client -> Centralized LB Cluster -> Backend Service A
                                   -> Backend Service B
                                   -> Backend Service C

Characteristics:
  - Dedicated LB infrastructure (separate from application)
  - All traffic flows through LB tier
  - Single point of configuration and control
  - Clear network topology

Architecture:
  Service A [Sidecar Proxy] <-> Service B [Sidecar Proxy]
                             <-> Service C [Sidecar Proxy]
  
  Control Plane (Istio/Linkerd) manages all sidecar configurations

Examples: Istio (Envoy sidecars), Linkerd, Consul Connect

Characteristics:
  - LB logic embedded in each service (sidecar proxy)
  - No centralized LB for east-west traffic
  - Control plane manages configuration
  - Each service instance has its own proxy

North-South traffic (external -> internal):
  - Centralized L7 LB (Nginx/ALB/Envoy) at edge
  - SSL termination, WAF, rate limiting, routing

East-West traffic (service -> service):
  - Service mesh (Envoy sidecars) or client-side LB
  - mTLS, retries, circuit breaking, load balancing

Example (Kubernetes):
  External: Ingress Controller (Nginx/Envoy) -> Service
  Internal: Service Mesh (Istio/Linkerd) for service-to-service

How: Distribute requests sequentially to each backend in order
Complexity: O(1)
State: Single atomic counter

Pros:
  - Simplest to implement and understand
  - Perfectly even distribution (over time)
  - No per-request computation
  - Works well when backends are homogeneous

Cons:
  - Ignores backend capacity differences
  - Ignores current load (may send to overloaded server)
  - Ignores request cost (expensive requests not distributed)

Best for: Homogeneous backends, uniform request costs

How: Each backend gets requests proportional to its weight
Complexity: O(1) with pre-computed schedule
State: Counter + weight table

Example: Server A (weight=3), Server B (weight=1)
  Sequence: A, A, A, B, A, A, A, B, ...

Smooth Weighted Round Robin (Nginx algorithm):
  Avoids bursts: A, A, B, A, A, A, B, A (interleaved)

Pros:
  - Accounts for different backend capacities
  - Still simple and predictable
  - Good for heterogeneous hardware

Cons:
  - Weights are static (don't adapt to real-time load)
  - Manual weight tuning required
  - Doesn't account for request cost variation

How: Send to backend with fewest active connections
Complexity: O(log n) with min-heap, O(n) with linear scan
State: Per-backend connection counter

Pros:
  - Adapts to real-time load
  - Handles slow backends (they accumulate connections)
  - Good for variable request durations
  - Self-correcting (overloaded servers get fewer requests)

Cons:
  - Slightly more overhead than round robin
  - New backends get flooded (0 connections = always selected)
  - Doesn't account for backend capacity differences
  - Connection count != actual load (idle keep-alive connections)

Mitigation: Weighted Least Connections (connections / weight)
Mitigation: Slow start (gradually increase traffic to new backends)

How: Hash request key to position on virtual ring, find nearest backend
Complexity: O(log n) binary search on ring
State: Sorted array of virtual nodes

Hash key options: Client IP, session cookie, request URL, custom header

Pros:
  - Session affinity without storing session state
  - Minimal disruption on backend add/remove (only 1/N keys move)
  - Deterministic (same key always goes to same backend)
  - Good for caching (same content always on same server)

Cons:
  - Uneven distribution with few backends (mitigated with virtual nodes)
  - No load awareness (hot keys overload one backend)
  - Backend removal causes spike on receiving backend
  - Doesn't adapt to real-time load

Improvements:
  - Bounded-load consistent hashing (Google): Cap max load per backend
  - Jump consistent hash: O(1) lookup, but no virtual nodes
  - Maglev hashing: Better distribution, faster lookup table

How: Randomly pick 2 backends, send to the one with fewer connections
Complexity: O(1)
State: Per-backend connection counter

Algorithm:
  1. Select 2 random backends from pool
  2. Compare their current connection counts
  3. Send request to the less-loaded one

Pros:
  - Near-optimal load distribution (exponentially better than random)
  - O(1) per-request (no heap, no ring lookup)
  - Adapts to real-time load
  - Simple to implement
  - Scales well (no contention on shared data structure)

Cons:
  - Slightly less optimal than true least-connections
  - Random selection means non-deterministic behavior
  - No session affinity

Mathematical property:
  - Random: max load = O(log n / log log n) with high probability
  - P2C: max load = O(log log n) with high probability
  - Dramatic improvement with minimal overhead

Used by: Envoy (default), Nginx (optional), many modern LBs

How: Send to backend with lowest average response time
Complexity: O(n) scan or O(log n) with sorted structure
State: Per-backend response time tracking (EMA)

Pros:
  - Routes away from slow backends automatically
  - Accounts for actual backend performance
  - Good for heterogeneous backends

Cons:
  - Response time measurement adds overhead
  - Cold start problem (new backends have no data)
  - Can cause oscillation (all traffic shifts to fastest server)
  - Doesn't distinguish between backend slowness and network latency

Mitigation: Exponential moving average with decay
Mitigation: Combine with P2C (pick 2, choose faster)

How: DNS resolver returns different IPs for same domain name
Implementation: Multiple A records, weighted records, or geo-aware DNS

Pros:
  - No infrastructure needed (just DNS configuration)
  - Works across regions and providers
  - Client connects directly to backend (no proxy hop)
  - Scales infinitely (DNS is distributed)

Cons:
  - DNS caching prevents quick failover (TTL-dependent)
  - Uneven distribution (resolver caching serves many clients)
  - No health checking in basic DNS (need health-checked DNS service)
  - No connection-aware balancing
  - Client may cache stale IP beyond TTL
  - No session persistence

Best for: Geographic distribution, multi-region failover, CDN routing

How: Dedicated LB instance receives all traffic, forwards to backends
Implementation: Reverse proxy (Nginx, HAProxy, Envoy)

Pros:
  - Instant failover (no DNS TTL issues)
  - Connection-aware balancing (least connections, etc.)
  - Health checking with immediate removal
  - Session persistence (cookies, consistent hashing)
  - Request modification (headers, URL rewrite)
  - Centralized metrics and logging

Cons:
  - Additional network hop (latency)
  - LB becomes potential bottleneck
  - LB infrastructure cost
  - Single point of failure (mitigated with HA)

Best for: Application-level routing, SSL termination, fine-grained control

How: Client maintains list of backends, selects one per request
Implementation: gRPC client LB, Netflix Ribbon, custom client library

Architecture:
  Service Registry (Consul/etcd) -> Client Library -> Backend
  Client discovers backends, makes direct connections

Pros:
  - No proxy hop (lower latency)
  - No centralized bottleneck
  - Client can use sophisticated algorithms
  - Reduces infrastructure cost (no LB fleet)
  - Better for gRPC/long-lived connections

Cons:
  - Client complexity (LB logic in every service)
  - Language-specific implementations needed
  - Harder to update LB logic (redeploy all clients)
  - Client must handle service discovery
  - Inconsistent behavior across client implementations
  - Harder to enforce policies centrally

Best for: Internal microservices, gRPC, high-performance paths

How: Dedicated LB infrastructure handles all routing decisions
Implementation: Nginx, HAProxy, AWS ALB, Envoy (as gateway)

Architecture:
  Client -> LB -> Backend
  Client only knows LB address, LB handles discovery and routing

Pros:
  - Simple clients (just connect to LB address)
  - Centralized control and policy enforcement
  - Language-agnostic (any client works)
  - Easier to update LB logic (no client changes)
  - Better observability (single point of measurement)

Cons:
  - Additional network hop
  - LB infrastructure cost and management
  - Potential bottleneck
  - Single point of failure (mitigated with HA)

Best for: External traffic, heterogeneous clients, centralized control

Traffic flow:
  Client -> LB -> Backend (request)
  Backend -> LB -> Client (response)

LB handles both directions of traffic.

Pros:
  - Can modify responses (add headers, compress, cache)
  - Can terminate SSL and re-encrypt
  - Full visibility into request AND response
  - Can implement WAF, response filtering
  - Connection multiplexing possible

Cons:
  - LB bandwidth = inbound + outbound (2x bandwidth requirement)
  - Higher latency (extra hop on response path)
  - LB becomes bandwidth bottleneck for large responses
  - More expensive (need more LB capacity)

Bandwidth impact:
  If responses are 10x larger than requests:
  LB handles 11x the request bandwidth
  Example: 1 GB/s requests + 10 GB/s responses = 11 GB/s through LB

Traffic flow:
  Client -> LB -> Backend (request only through LB)
  Backend -> Client (response bypasses LB entirely)

LB only handles inbound traffic.

Pros:
  - LB bandwidth = inbound only (massive savings for large responses)
  - Lower response latency (one fewer hop)
  - LB can handle 5-10x more traffic
  - Ideal for streaming, media, large file downloads

Cons:
  - Cannot modify responses
  - Cannot terminate SSL at LB (backend must handle TLS)
  - Limited health checking (can't observe response codes)
  - More complex network configuration
  - Backend must accept traffic for LB's VIP address
  - No connection multiplexing

Bandwidth impact:
  Same example: LB only handles 1 GB/s (requests)
  Responses (10 GB/s) go directly from backend to client
  10x reduction in LB bandwidth requirement

Implementation requirements:
  - L2 DSR: Backend on same subnet, loopback interface with VIP
  - L3 DSR: IP-in-IP or GRE tunnel to backend
  - Backend must not ARP for VIP (only LB should)

Use Full Proxy when:
  - SSL termination needed at LB
  - Response modification required (headers, compression)
  - WAF/security inspection of responses
  - Connection multiplexing desired
  - Request/response sizes are similar

Use DSR when:
  - Large responses (streaming, media, downloads)
  - Maximum LB throughput needed
  - Backend can handle SSL
  - Response modification not needed
  - Bandwidth cost is a concern

Hybrid approach:
  - DSR for media/streaming traffic (large responses)
  - Full proxy for API traffic (small responses, need L7 features)
  - Split at L4 LB based on port or protocol

- Custom L4 load balancer using kernel bypass
- Consistent hashing with Maglev hash algorithm
- Handles all Google frontend traffic
- ECMP distribution across Maglev instances
- Connection tracking for session affinity
- 10M+ packets/sec per instance

- XDP-based L4 load balancer
- Runs on commodity hardware
- Handles all Facebook edge traffic
- Consistent hashing (CH) for backend selection
- No per-connection state (stateless with CH)
- Open-sourced, runs on standard Linux

- Fully managed L7 load balancer
- Auto-scales transparently
- Content-based routing (host, path, headers)
- Native integration with AWS services
- WebSocket and HTTP/2 support
- WAF integration
- No infrastructure management

- Modern L7 proxy designed for service mesh
- xDS API for dynamic configuration
- Advanced load balancing (P2C, ring hash, Maglev)
- Built-in observability (metrics, tracing, logging)
- Extensible via WASM filters
- Used as sidecar in Istio service mesh
- Also used as edge proxy (Envoy Gateway)