Networking Protocols: Complete Guide for System Design

Overview

Every system design interview involves services communicating over a network. Understanding protocols at a deep level lets you make informed decisions about latency, reliability, and scalability.

1. TCP vs UDP

TCP (Transmission Control Protocol)

How it works: Connection-oriented, reliable, ordered delivery.

Three-Way Handshake

Cost: 1.5 RTT before any data flows. With TLS, add another 1-2 RTT.

Flow Control (Sliding Window)

• Receiver advertises a window size (how much data it can buffer)
• Sender never sends more than the window allows
• Window shrinks as data arrives, grows as application reads it
• Prevents fast sender from overwhelming slow receiver

Congestion Control

Key insight for interviews: TCP slow start means new connections are slow. This is why connection pooling and HTTP/2 multiplexing matter.

Connection Teardown (Four-Way)

TIME_WAIT problem: High-throughput servers can exhaust ephemeral ports. Solutions: SO_REUSEADDR, connection pooling, or switch to long-lived connections.

UDP (User Datagram Protocol)

How it works: Connectionless, unreliable, unordered. Just sends packets.

• No handshake (0 RTT to start sending)
• No guaranteed delivery or ordering
• No flow/congestion control (application must handle)
• Smaller header (8 bytes vs TCP's 20+ bytes)

When to Use Which

Interview tip: "We'd use TCP here because we need guaranteed delivery of financial transactions" or "UDP for live video because a retransmitted frame arrives too late to be useful."

2. HTTP/1.1 vs HTTP/2 vs HTTP/3

HTTP/1.1 (1997)

Key characteristics:

• Text-based protocol
• One request per TCP connection at a time (head-of-line blocking)
• Workaround: browsers open 6-8 parallel connections per domain
• Keep-Alive reuses connections but still sequential

Head-of-Line (HOL) Blocking:

If Response A is slow, Request C waits even though the server could serve it.

HTTP/2 (2015)

Key improvements:

• Binary framing layer (more efficient parsing)
• Multiplexing: Multiple streams over single TCP connection
• Header compression (HPACK): Reduces redundant headers by 85-90%
• Server push: Server sends resources before client requests them
• Stream prioritization: Client hints which resources matter most

Remaining problem: TCP-level HOL blocking. If a TCP packet is lost, ALL streams wait for retransmission, even unaffected ones.

Performance numbers:

• 50-70% reduction in page load time for asset-heavy pages
• Single connection vs 6-8 connections reduces server memory
• Header compression saves 10-30KB per page load

HTTP/3 (2022) — QUIC

Key innovation: Runs over UDP instead of TCP, implements its own reliability.

Advantages over HTTP/2:

• No HOL blocking: Lost packet only affects its stream
• 0-RTT resumption: Returning clients send data immediately
• Connection migration: Survives IP changes (WiFi → cellular)
• Built-in encryption: TLS 1.3 integrated into handshake

Connection establishment comparison:

Comparison Table

3. WebSocket vs SSE vs Long Polling

Long Polling

Characteristics:

• Compatible with all infrastructure (proxies, load balancers)
• Each response requires a new HTTP request
• Timeout handling needed (30-60s typical)
• Server holds connections open (resource intensive)
• ~100ms latency per message (reconnection overhead)

Server-Sent Events (SSE)

Characteristics:

• Unidirectional (server → client only)
• Built-in reconnection with Last-Event-ID
• Text-based (UTF-8 only)
• Works over standard HTTP (proxy-friendly)
• Automatic reconnection by browser
• Limited to ~6 connections per domain in HTTP/1.1

WebSocket

Characteristics:

• Full-duplex (both directions simultaneously)
• Binary and text frames
• Low overhead per message (2-14 bytes framing vs HTTP headers)
• Persistent connection
• Requires WebSocket-aware load balancers
• No built-in reconnection (application must handle)

Decision Framework

Interview tip: "For a notification system, SSE is simpler and sufficient since we only push from server to client. WebSocket adds complexity we don't need."

4. DNS Resolution Flow

Complete Resolution Process

Record Types

TTL and Caching Strategy

• Short TTL (60-300s): Enables fast failover, more DNS traffic
• Long TTL (3600-86400s): Reduces DNS load, slower failover
• TTL=0: No caching (used during migrations)

System design implications:

• DNS-based load balancing uses short TTLs for health-check responsiveness
• CDN providers use 60s TTL for quick origin switching
• During migrations, lower TTL days in advance, then switch

DNS in System Design

• Global load balancing: GeoDNS routes users to nearest datacenter
• Service discovery: Internal DNS for microservice endpoints
• Failover: Health-checked DNS removes unhealthy endpoints
• Blue-green deployments: Switch DNS to new environment

5. TLS Handshake

TLS 1.2 Handshake (2 RTT)

TLS 1.3 Handshake (1 RTT)

Key improvements in TLS 1.3:

• 1 RTT handshake (vs 2 RTT in 1.2)
• 0-RTT resumption (send data with first message)
• Removed insecure algorithms (RSA key exchange, CBC, RC4, SHA-1)
• Forward secrecy mandatory (ephemeral DH only)
• Encrypted more of the handshake (hides certificate from observers)

Certificate Chain Verification

ALPN (Application-Layer Protocol Negotiation)

• Negotiates application protocol during TLS handshake
• Client sends list: ["h2", "http/1.1"]
• Server picks: "h2"
• Avoids extra round trip for protocol upgrade
• Essential for HTTP/2 and HTTP/3 negotiation

6. gRPC and Protocol Buffers

Protocol Buffers (Protobuf)

Schema definition:

syntax = "proto3";

message User {
  string id = 1;           // field number, not value
  string name = 2;
  string email = 3;
  repeated string roles = 4;
  google.protobuf.Timestamp created_at = 5;
}

message GetUserRequest {
  string user_id = 1;
}

service UserService {
  rpc GetUser(GetUserRequest) returns (User);
  rpc ListUsers(ListUsersRequest) returns (stream User);
  rpc CreateUser(User) returns (User);
}

Binary encoding advantages:

• 3-10x smaller than JSON
• 20-100x faster serialization/deserialization
• Schema evolution with backward/forward compatibility
• Strongly typed (catches errors at compile time)

gRPC Communication Patterns

gRPC vs REST Comparison

When to Use gRPC

Use gRPC for:

• Internal microservice communication (performance critical)
• Polyglot environments (code gen for 10+ languages)
• Streaming data (real-time feeds, event streams)
• Mobile clients with bandwidth constraints

Use REST for:

• Public APIs (browser compatibility, developer familiarity)
• Simple CRUD operations
• When human readability matters for debugging
• Third-party integrations

gRPC Performance Characteristics

Interview tip: "For service-to-service communication within our backend, gRPC gives us type safety, streaming, and 3-5x better performance. For our public API, REST is more accessible to third-party developers."

Quick Reference: Protocol Selection

Interview Cheat Sheet