Interviewer: "Design a messaging system like Facebook Messenger"

You: "Great! Let me clarify the requirements:
- Are we building for global scale like Facebook's 1.3B users?
- Do we need both individual and group messaging?
- Should we support media sharing (images, videos, files)?
- What's the acceptable message delivery latency?
- Do we need features like read receipts and online presence?
- Any specific compliance requirements like GDPR?"

[Client Apps] → [Load Balancer] → [API Gateway] → [Message Service] → [Database]
                                                      ↓
                                               [WebSocket Service]

Interviewer: "Why choose WebSocket over HTTP for real-time messaging?"

Strong Answer:
"WebSocket provides several advantages for messaging:
1. Full-duplex communication - both client and server can initiate
2. Lower latency - no HTTP request/response overhead
3. Persistent connections - avoid connection establishment cost
4. Real-time updates - instant message delivery and typing indicators

However, WebSocket has challenges:
1. Connection management complexity at scale
2. Load balancing requires sticky sessions
3. Some firewalls/proxies block WebSocket
4. Higher server resource usage per connection

For fallback, I'd implement Server-Sent Events or long polling for environments that block WebSocket."

Interviewer: "How do you ensure message ordering in group chats?"

Strong Answer:
"Message ordering is critical for user experience. I'd use:

1. **Sequence Numbers**: Assign incrementing sequence numbers per conversation
2. **Single Writer**: Route all messages for a conversation through one server
3. **Message Queue**: Use Kafka with conversation_id as partition key
4. **Client-side Ordering**: Clients sort messages by sequence number
5. **Gap Detection**: Clients detect missing sequence numbers and request them

For global ordering across all conversations, I'd use:
- Vector clocks for causal ordering
- Lamport timestamps for total ordering
- But this adds complexity, so I'd only implement if required"

Interviewer: "How would you scale the database for billions of messages?"

Strong Answer:
"I'd use a multi-database approach:

1. **Message Storage (Cassandra)**:
   - Partition by conversation_id for locality
   - Time-series data model with TIMEUUID
   - 3x replication across data centers
   - Handles high write volume efficiently

2. **User Data (PostgreSQL)**:
   - Master-replica setup for read scaling
   - Shard by user_id if needed
   - ACID compliance for critical user data

3. **Caching (Redis)**:
   - Recent messages cached for fast access
   - User sessions and presence data
   - Distributed cache with consistent hashing

4. **Search (Elasticsearch)**:
   - Full-text search across message content
   - Separate index for better performance"

Interviewer: "How do you handle 100 million concurrent WebSocket connections?"

Strong Answer:
"Connection scaling requires several strategies:

1. **Horizontal Scaling**:
   - 2,000-5,000 connections per server
   - 20,000+ WebSocket servers globally
   - Auto-scaling based on connection count

2. **Load Balancing**:
   - Consistent hashing for sticky sessions
   - Health checks and failover mechanisms
   - Geographic routing to nearest data center

3. **Connection Pooling**:
   - Reuse connections across conversations
   - Connection multiplexing where possible
   - Graceful connection migration during updates

4. **Resource Optimization**:
   - 8KB memory per connection
   - Efficient serialization (Protocol Buffers)
   - Connection compression and heartbeat optimization"

"For real-time messaging, I'd use event-driven architecture:

1. **Message Events**: message_sent, message_delivered, message_read
2. **Presence Events**: user_online, user_offline, user_typing
3. **System Events**: connection_established, connection_lost

Benefits:
- Loose coupling between services
- Easy to add new features
- Natural fit for real-time systems
- Scalable event processing"

"For high-scale messaging, I'd separate read and write operations:

Write Side (Commands):
- Send message
- Create conversation
- Update user profile

Read Side (Queries):
- Get message history
- Search messages
- Get conversation list

This allows:
- Independent scaling of reads vs writes
- Optimized data models for each operation
- Better performance and availability"

"I'd implement multi-layer caching:

L1 (Application Cache):
- In-memory cache for frequently accessed data
- 1-minute TTL to balance freshness and performance

L2 (Distributed Cache - Redis):
- Recent messages per conversation
- User presence and session data
- 1-hour TTL with cache warming

L3 (Database Query Cache):
- Query result caching
- Invalidated on data changes"

"Database optimization strategies:

1. **Indexing**:
   - Composite indexes on (conversation_id, timestamp)
   - Partial indexes for active conversations only

2. **Partitioning**:
   - Time-based partitioning for message tables
   - Hash partitioning for user tables

3. **Connection Pooling**:
   - PgBouncer for PostgreSQL connections
   - Connection pool sizing based on load

4. **Read Replicas**:
   - Route read queries to replicas
   - Async replication for better write performance"

"Typing indicators require real-time updates with specific characteristics:

1. **Client Side**:
   - Detect typing events (keypress, focus)
   - Debounce to avoid excessive updates (500ms)
   - Send typing_start when user begins typing
   - Send typing_stop after 3 seconds of inactivity

2. **Server Side**:
   - Broadcast typing events to conversation participants
   - Use Redis with 5-second TTL for typing state
   - Don't persist typing events (ephemeral data)

3. **Optimization**:
   - Rate limit typing events (max 1 per second)
   - Only show typing for active conversation
   - Batch multiple typing updates

This provides responsive UX while minimizing server load."

"Offline message delivery requires multiple strategies:

1. **Message Queuing**:
   - Store messages in persistent queue (Kafka/RabbitMQ)
   - Partition by user_id for ordered delivery
   - Retry logic with exponential backoff

2. **Push Notifications**:
   - APNs for iOS, FCM for Android
   - Rich notifications with message preview
   - Badge count updates

3. **Synchronization**:
   - When user comes online, sync missed messages
   - Use sequence numbers to detect gaps
   - Incremental sync to minimize data transfer

4. **Storage**:
   - Offline messages stored in database
   - TTL for cleanup (30 days)
   - Compression for large message backlogs

This ensures reliable message delivery regardless of connectivity."

"E2E encryption requires careful key management and protocol design:

1. **Protocol Choice**:
   - Signal Protocol for proven security
   - Double Ratchet for forward secrecy
   - X3DH for initial key exchange

2. **Key Management**:
   - Identity keys stored locally
   - Pre-keys uploaded to server
   - Session keys derived per conversation

3. **Message Flow**:
   - Encrypt message with session key
   - Include ephemeral key for ratcheting
   - Server routes encrypted envelope
   - Recipient decrypts with session key

4. **Challenges**:
   - Key verification (QR codes, safety numbers)
   - Multi-device synchronization
   - Backup and recovery
   - Performance impact

This provides strong security while maintaining usability."