Networking Protocols: Complete Guide for System Design

Overview

Every system design interview involves services communicating over a network. Understanding protocols at a deep level lets you make informed decisions about latency, reliability, and scalability.

1. TCP vs UDP

TCP (Transmission Control Protocol)

How it works: Connection-oriented, reliable, ordered delivery.

Three-Way Handshake

Client                    Server
  │                         │
  │──── SYN (seq=x) ──────►│   1. Client initiates
  │                         │
  │◄─── SYN-ACK ───────────│   2. Server acknowledges + initiates
  │     (seq=y, ack=x+1)   │
  │                         │
  │──── ACK (ack=y+1) ────►│   3. Client confirms
  │                         │
  │     Connection Open     │

Cost: 1.5 RTT before any data flows. With TLS, add another 1-2 RTT.

Flow Control (Sliding Window)

Receiver advertises a window size (how much data it can buffer)
Sender never sends more than the window allows
Window shrinks as data arrives, grows as application reads it
Prevents fast sender from overwhelming slow receiver

Congestion Control

Algorithm	Behavior
Slow Start	Exponential growth until threshold
Congestion Avoidance	Linear growth after threshold
Fast Retransmit	Retransmit after 3 duplicate ACKs
Fast Recovery	Don't reset to slow start on fast retransmit
BBR (Google)	Model-based, measures bandwidth and RTT

Key insight for interviews: TCP slow start means new connections are slow. This is why connection pooling and HTTP/2 multiplexing matter.

Connection Teardown (Four-Way)

Client                    Server
  │──── FIN ───────────────►│
  │◄─── ACK ────────────────│
  │◄─── FIN ────────────────│
  │──── ACK ───────────────►│
  │                         │
  │  TIME_WAIT (2×MSL)     │   Client waits ~60s before port reuse

TIME_WAIT problem: High-throughput servers can exhaust ephemeral ports. Solutions: SO_REUSEADDR, connection pooling, or switch to long-lived connections.

UDP (User Datagram Protocol)

How it works: Connectionless, unreliable, unordered. Just sends packets.

No handshake (0 RTT to start sending)
No guaranteed delivery or ordering
No flow/congestion control (application must handle)
Smaller header (8 bytes vs TCP's 20+ bytes)

When to Use Which

Use Case	Protocol	Why
Web APIs	TCP	Need reliability, ordering
Video streaming	UDP	Tolerate loss, need low latency
Gaming	UDP	Real-time, stale data useless
DNS queries	UDP	Small payload, speed matters
File transfer	TCP	Must have complete, ordered data
VoIP	UDP	Real-time, retransmission too slow
IoT telemetry	UDP	Lightweight, high volume

Interview tip: "We'd use TCP here because we need guaranteed delivery of financial transactions" or "UDP for live video because a retransmitted frame arrives too late to be useful."

2. HTTP/1.1 vs HTTP/2 vs HTTP/3

HTTP/1.1 (1997)

Key characteristics:

Text-based protocol
One request per TCP connection at a time (head-of-line blocking)
Workaround: browsers open 6-8 parallel connections per domain
Keep-Alive reuses connections but still sequential

Head-of-Line (HOL) Blocking:

Connection 1: [Request A]────────[Response A]────[Request C]────[Response C]
Connection 2: [Request B]──[Response B]──────────[idle]─────────────────────

If Response A is slow, Request C waits even though the server could serve it.

HTTP/2 (2015)

Key improvements:

Binary framing layer (more efficient parsing)
Multiplexing: Multiple streams over single TCP connection
Header compression (HPACK): Reduces redundant headers by 85-90%
Server push: Server sends resources before client requests them
Stream prioritization: Client hints which resources matter most

Single TCP Connection:
┌─────────────────────────────────────────────┐
│ Stream 1: [Headers][Data][Data]             │
│ Stream 2: [Headers][Data]                   │
│ Stream 3: [Headers][Data][Data][Data]       │
│ (interleaved frames on the wire)            │
└─────────────────────────────────────────────┘

Remaining problem: TCP-level HOL blocking. If a TCP packet is lost, ALL streams wait for retransmission, even unaffected ones.

Performance numbers:

50-70% reduction in page load time for asset-heavy pages
Single connection vs 6-8 connections reduces server memory
Header compression saves 10-30KB per page load

HTTP/3 (2022) — QUIC

Key innovation: Runs over UDP instead of TCP, implements its own reliability.

┌──────────────────────────────────────┐
│           HTTP/3 (Application)        │
├──────────────────────────────────────┤
│           QUIC (Transport)            │
│  • Stream multiplexing               │
│  • Per-stream flow control           │
│  • Connection migration              │
│  • 0-RTT connection establishment    │
├──────────────────────────────────────┤
│           UDP (Network)               │
└──────────────────────────────────────┘

Advantages over HTTP/2:

No HOL blocking: Lost packet only affects its stream
0-RTT resumption: Returning clients send data immediately
Connection migration: Survives IP changes (WiFi → cellular)
Built-in encryption: TLS 1.3 integrated into handshake

Connection establishment comparison:

HTTP/1.1 + TLS 1.2:  3 RTT (TCP + TLS + Request)
HTTP/2 + TLS 1.3:    2 RTT (TCP + TLS/Request combined)
HTTP/3 (new):        1 RTT (QUIC handshake includes crypto)
HTTP/3 (resumption): 0 RTT (send data with first packet)

Comparison Table

Feature	HTTP/1.1	HTTP/2	HTTP/3
Transport	TCP	TCP	QUIC (UDP)
Multiplexing	No	Yes	Yes
HOL Blocking	Application + TCP	TCP only	None
Header Compression	None	HPACK	QPACK
Connection Setup	2-3 RTT	2 RTT	0-1 RTT
Connection Migration	No	No	Yes
Encryption	Optional	Effectively required	Mandatory

3. WebSocket vs SSE vs Long Polling

Long Polling

Client                         Server
  │── GET /updates ───────────►│
  │                            │  (holds connection open)
  │                            │  ... waits for data ...
  │◄── 200 OK + data ─────────│  (responds when data available)
  │                            │
  │── GET /updates ───────────►│  (immediately reconnects)
  │                            │  ... waits again ...

Characteristics:

Compatible with all infrastructure (proxies, load balancers)
Each response requires a new HTTP request
Timeout handling needed (30-60s typical)
Server holds connections open (resource intensive)
~100ms latency per message (reconnection overhead)

Server-Sent Events (SSE)

Client                         Server
  │── GET /stream ────────────►│
  │   Accept: text/event-stream│
  │                            │
  │◄── HTTP 200 ──────────────│
  │    Content-Type:           │
  │    text/event-stream       │
  │                            │
  │◄── data: message 1\n\n ───│  (server pushes)
  │◄── data: message 2\n\n ───│  (server pushes)
  │◄── data: message 3\n\n ───│  (server pushes)
  │         ...                │

Characteristics:

Unidirectional (server → client only)
Built-in reconnection with Last-Event-ID
Text-based (UTF-8 only)
Works over standard HTTP (proxy-friendly)
Automatic reconnection by browser
Limited to ~6 connections per domain in HTTP/1.1

WebSocket

Client                         Server
  │── HTTP Upgrade Request ───►│
  │   Upgrade: websocket       │
  │   Connection: Upgrade      │
  │                            │
  │◄── 101 Switching ─────────│
  │    Protocols               │
  │                            │
  │◄──► Full-duplex binary ◄──►│  (bidirectional frames)
  │◄──► communication    ◄────►│

Characteristics:

Full-duplex (both directions simultaneously)
Binary and text frames
Low overhead per message (2-14 bytes framing vs HTTP headers)
Persistent connection
Requires WebSocket-aware load balancers
No built-in reconnection (application must handle)

Decision Framework

Requirement	Best Choice	Why
Real-time chat	WebSocket	Bidirectional, low latency
Live sports scores	SSE	Server-push only, auto-reconnect
Stock ticker	WebSocket	High frequency, bidirectional
News feed updates	SSE	Infrequent server pushes
Collaborative editing	WebSocket	Bidirectional, binary data
Simple notifications	SSE	One-way, simple implementation
Legacy system support	Long Polling	Works everywhere
IoT device commands	WebSocket	Bidirectional, persistent

Interview tip: "For a notification system, SSE is simpler and sufficient since we only push from server to client. WebSocket adds complexity we don't need."

4. DNS Resolution Flow

Complete Resolution Process

User types "www.example.com"
         │
         ▼
┌─────────────────┐     Cache hit?
│  Browser Cache  │────────────────► Done (TTL-based)
└────────┬────────┘
         │ Cache miss
         ▼
┌─────────────────┐     Cache hit?
│    OS Cache     │────────────────► Done
│  (stub resolver)│
└────────┬────────┘
         │ Cache miss
         ▼
┌─────────────────┐     Cache hit?
│ Recursive DNS   │────────────────► Done
│ (ISP/8.8.8.8)  │
└────────┬────────┘
         │ Cache miss (iterative queries begin)
         ▼
┌─────────────────┐
│  Root Server    │──► "Ask .com TLD server at 192.5.6.30"
│  (13 clusters)  │
└────────┬────────┘
         ▼
┌─────────────────┐
│  TLD Server     │──► "Ask example.com NS at 205.251.192.1"
│  (.com, .org)   │
└────────┬────────┘
         ▼
┌─────────────────────┐
│ Authoritative Server │──► "www.example.com = 93.184.216.34"
│ (example.com)        │
└──────────────────────┘

Record Types

Type	Purpose	Example
A	IPv4 address	example.com → 93.184.216.34
AAAA	IPv6 address	example.com → 2606:2800:220:1:...
CNAME	Alias to another name	www → example.com
MX	Mail server	example.com → mail.example.com
NS	Nameserver delegation	example.com → ns1.example.com
TXT	Arbitrary text	SPF, DKIM, domain verification
SRV	Service location	_http._tcp.example.com

TTL and Caching Strategy

Short TTL (60-300s): Enables fast failover, more DNS traffic
Long TTL (3600-86400s): Reduces DNS load, slower failover
TTL=0: No caching (used during migrations)

System design implications:

DNS-based load balancing uses short TTLs for health-check responsiveness
CDN providers use 60s TTL for quick origin switching
During migrations, lower TTL days in advance, then switch

DNS in System Design

Global load balancing: GeoDNS routes users to nearest datacenter
Service discovery: Internal DNS for microservice endpoints
Failover: Health-checked DNS removes unhealthy endpoints
Blue-green deployments: Switch DNS to new environment

5. TLS Handshake

TLS 1.2 Handshake (2 RTT)

Client                              Server
  │                                   │
  │── ClientHello ───────────────────►│  Supported ciphers, random
  │                                   │
  │◄── ServerHello ──────────────────│  Chosen cipher, random
  │◄── Certificate ──────────────────│  Server's X.509 cert
  │◄── ServerKeyExchange ────────────│  DH parameters
  │◄── ServerHelloDone ──────────────│
  │                                   │
  │── ClientKeyExchange ─────────────►│  Client's DH public key
  │── ChangeCipherSpec ──────────────►│  "Switching to encrypted"
  │── Finished ──────────────────────►│  Encrypted verification
  │                                   │
  │◄── ChangeCipherSpec ─────────────│
  │◄── Finished ─────────────────────│
  │                                   │
  │◄══► Encrypted Application Data ◄═►│

TLS 1.3 Handshake (1 RTT)

Client                              Server
  │                                   │
  │── ClientHello ───────────────────►│  + key_share (DH public key)
  │    + supported_versions           │  + signature_algorithms
  │    + key_share                    │
  │                                   │
  │◄── ServerHello ──────────────────│  + key_share
  │◄── EncryptedExtensions ──────────│  (encrypted from here)
  │◄── Certificate ──────────────────│
  │◄── CertificateVerify ───────────│
  │◄── Finished ─────────────────────│
  │                                   │
  │── Finished ──────────────────────►│
  │                                   │
  │◄══► Encrypted Application Data ◄═►│

Key improvements in TLS 1.3:

1 RTT handshake (vs 2 RTT in 1.2)
0-RTT resumption (send data with first message)
Removed insecure algorithms (RSA key exchange, CBC, RC4, SHA-1)
Forward secrecy mandatory (ephemeral DH only)
Encrypted more of the handshake (hides certificate from observers)

Certificate Chain Verification

┌─────────────────────┐
│   Root CA (trusted)  │  Pre-installed in OS/browser
│   Self-signed        │  ~150 root CAs trusted globally
└──────────┬──────────┘
           │ signs
           ▼
┌─────────────────────┐
│  Intermediate CA     │  Issued by Root CA
│                      │  Used for day-to-day signing
└──────────┬──────────┘
           │ signs
           ▼
┌─────────────────────┐
│  Server Certificate  │  Your domain's certificate
│  (leaf cert)         │  Contains public key + domain name
└─────────────────────┘

ALPN (Application-Layer Protocol Negotiation)

Negotiates application protocol during TLS handshake
Client sends list: ["h2", "http/1.1"]
Server picks: "h2"
Avoids extra round trip for protocol upgrade
Essential for HTTP/2 and HTTP/3 negotiation

6. gRPC and Protocol Buffers

Protocol Buffers (Protobuf)

Schema definition:

syntax = "proto3";

message User {
  string id = 1;           // field number, not value
  string name = 2;
  string email = 3;
  repeated string roles = 4;
  google.protobuf.Timestamp created_at = 5;
}

message GetUserRequest {
  string user_id = 1;
}

service UserService {
  rpc GetUser(GetUserRequest) returns (User);
  rpc ListUsers(ListUsersRequest) returns (stream User);
  rpc CreateUser(User) returns (User);
}

Binary encoding advantages:

3-10x smaller than JSON
20-100x faster serialization/deserialization
Schema evolution with backward/forward compatibility
Strongly typed (catches errors at compile time)

gRPC Communication Patterns

┌──────────────────────────────────────────────────────┐
│                    gRPC Modes                          │
├──────────────────────────────────────────────────────┤
│                                                       │
│  1. Unary RPC (request-response)                     │
│     Client ──[Request]──► Server ──[Response]──►     │
│                                                       │
│  2. Server Streaming                                  │
│     Client ──[Request]──► Server ──[R1][R2][R3]──►   │
│                                                       │
│  3. Client Streaming                                  │
│     Client ──[R1][R2][R3]──► Server ──[Response]──►  │
│                                                       │
│  4. Bidirectional Streaming                           │
│     Client ◄──[messages]──► Server                   │
│                                                       │
└──────────────────────────────────────────────────────┘

gRPC vs REST Comparison

Aspect	gRPC	REST
Protocol	HTTP/2	HTTP/1.1 or HTTP/2
Payload	Protobuf (binary)	JSON (text)
Contract	.proto file (strict)	OpenAPI (optional)
Streaming	Native (4 modes)	Limited (SSE, WebSocket)
Browser support	Via grpc-web proxy	Native
Code generation	Built-in	Third-party tools
Latency	Lower (binary + HTTP/2)	Higher (text + overhead)
Debugging	Harder (binary)	Easier (human-readable)
Load balancing	L7 required (HTTP/2)	L4 or L7

When to Use gRPC

Use gRPC for:

Internal microservice communication (performance critical)
Polyglot environments (code gen for 10+ languages)
Streaming data (real-time feeds, event streams)
Mobile clients with bandwidth constraints

Use REST for:

Public APIs (browser compatibility, developer familiarity)
Simple CRUD operations
When human readability matters for debugging
Third-party integrations

gRPC Performance Characteristics

Benchmark: 1000 requests, 1KB payload

REST/JSON:
  Serialization:   ~500μs
  Payload size:    ~1,200 bytes
  Total latency:   ~2ms

gRPC/Protobuf:
  Serialization:   ~50μs
  Payload size:    ~400 bytes
  Total latency:   ~0.5ms

Interview tip: "For service-to-service communication within our backend, gRPC gives us type safety, streaming, and 3-5x better performance. For our public API, REST is more accessible to third-party developers."

Quick Reference: Protocol Selection

Need reliable delivery?
├── Yes → TCP-based
│   ├── Request-response? → HTTP (REST or gRPC)
│   ├── Server push only? → SSE
│   ├── Bidirectional real-time? → WebSocket
│   └── High-performance internal? → gRPC
└── No → UDP-based
    ├── Real-time media? → RTP/WebRTC
    ├── Fast queries? → DNS, QUIC
    └── IoT/lightweight? → MQTT over UDP, CoAP

Interview Cheat Sheet

When interviewer asks...	Key points to mention
"How do services communicate?"	gRPC for internal, REST for external, async via message queues
"How to handle real-time updates?"	WebSocket for bidirectional, SSE for server-push, consider scale
"Why is the first request slow?"	TCP handshake + TLS handshake + DNS resolution = cold start
"How to reduce latency?"	Connection pooling, HTTP/2 multiplexing, 0-RTT with TLS 1.3
"How does HTTPS work?"	TLS handshake, certificate verification, symmetric key exchange
"HTTP/2 vs HTTP/3?"	HTTP/3 eliminates TCP HOL blocking, enables connection migration