Design WhatsApp

Problem Statement#

"Design a chat system like WhatsApp. Users should be able to send messages to each other and to groups, in real time."

That's it. One sentence. The rest is yours to figure out.

Requirements Gathering#

This is where the interview actually begins. A strong candidate doesn't start drawing boxes. They ask questions. Not to stall, but because the answers fundamentally change the design.

Here's how a good requirements conversation looks:

Candidate: "Are we designing 1:1 messaging, group messaging, or both? And what's the max group size?"

Interviewer: "Both. Groups up to about 1,000 members."

Candidate: "Do we need delivery and read receipts, the sent, delivered, read distinction?"

Interviewer: "Yes. The double-tick and blue-tick behavior is in scope."

Candidate: "What scale? Daily active users and message volume?"

Interviewer: "500 million DAU, 100 billion messages per day."

Candidate: "How fast must delivery be when both users are online?"

Interviewer: "Sub-second. Under 500 milliseconds end to end."

Candidate: "Do we persist history? If someone reinstalls the app, do they get their messages back?"

Interviewer: "Yes. Messages are durable and retrievable. Never lose a confirmed message."

Candidate: "Text only, or media too, images, video, documents?"

Interviewer: "Text, images, video, documents. But you can treat media as an upload-then-reference flow."

Candidate: "Out of scope: end-to-end encryption, voice and video calls, reactions and threaded replies?"

Interviewer: "Correct, out of scope. Just reliable, ordered message delivery."

Functional and Non-Functional Requirements#

Functional Requirements#

1. Users can send and receive text messages in real time (1:1 conversations)
2. Messages show delivery status: sent, delivered, read (the famous double tick)
3. Users can message groups of up to ~1,000 members
4. Messages are durable and retrievable, so a reinstalled client can load history
5. Users can send media: images, video, documents
6. Online/offline presence indicator for contacts

Non-Functional Requirements#

Out of scope: End-to-end encryption (Signal Protocol), voice/video calls, reactions and threaded replies, WhatsApp Business API

Capacity Estimates#

Work through the math. Don't just state the numbers.

Message throughput:
  100B messages/day ÷ 86,400 sec = ~1.16M messages/sec

Concurrent connections (assume 20% of DAU online at once):
  500M × 0.20 = 100M concurrent WebSocket connections

Chat servers (each holds ~100k persistent connections):
  100M ÷ 100k = ~1,000 chat servers

Storage (text only, avg 100 bytes/message):
  100B × 100B = 10 TB/day of message data
  At replication factor 3: ~30 TB/day written

Group fan-out amplification:
  A 1,000-member group message = up to 1,000 delivery attempts
  This is the write-amplification number to watch

The number that should make you pause: 100M concurrent persistent connections. You can't poll your way to that: the connection model is the core of this design.

High-Level Design#

Two things drive everything: holding millions of live connections, and routing a message from one connection to another reliably. Keep them separate in your head.

[Client A] ⇄ WebSocket ⇄ [Chat Server A] → [Kafka] → [Chat Server B] ⇄ WebSocket ⇄ [Client B]
                              ↓                              ↓
                        [Message Store: Cassandra]   [Presence: Redis]
                              ↓
                        [Media Service → S3]

Write path: Client A sends over its WebSocket to Chat Server A → server persists to Cassandra (status: sent) → publishes to Kafka → ACKs Client A immediately (single tick). A router stage consumes from Kafka and, for each recipient, asks the presence service which server currently holds that recipient's socket (user:{id} → server_id in Redis) and forwards the message only to that one server, which pushes it over the recipient's WebSocket. The forward is directed, not broadcast: you never fan a message out to all 1,000 chat servers to find the one that owns the connection. If presence says the recipient is offline, delivery is skipped and the reconnect scan handles it.

Read path (history / offline): On reconnect, a client sends its last-seen message ID; the server range-scans Cassandra for anything newer across its conversations and delivers it over the fresh connection.

The key insight: the ACK to the sender is decoupled from delivery to the recipient. The sender learns the server durably has the message (single tick) before the recipient is even reachable. Kafka is only the real-time routing layer; Cassandra is the source of truth, so a message dropped from the queue is never lost.

Deep Dives#

1. Connection Model: WebSocket vs Long Polling vs SSE#

This is the first fork in the road. You need bidirectional, low-latency, persistent connections for 100M clients.

HTTP long polling: client requests, server holds it open until a message arrives, client immediately re-requests. Wastes connections, high overhead per message, not truly real-time. A fallback, not a design.

Server-Sent Events (SSE): server pushes to client over one persistent HTTP stream, but it's one-directional. The client can't send over the same channel, so you'd need a second channel for outbound messages. Wrong shape for chat.

WebSocket (recommended): full-duplex persistent connection. One handshake, then low-overhead frames in both directions. Client and server both send at will. This is what every serious chat app uses (WhatsApp historically used a customized XMPP over the same idea).

The connection stays open while the app is foregrounded. Chat servers are therefore stateful (each holds ~100k live sockets) but the per-connection state is tiny, so they scale horizontally behind a connection-aware load balancer.

2. Message Flow and Delivery Receipts#

This is the most nuanced part of the design and where strong candidates separate themselves. WhatsApp has four tick states: clock (sending), single tick (server received), double tick (delivered to device), and blue tick (read). Each requires a precise acknowledgement traveling back up the chain.

1:1 message flow:

1.  Client A → Chat Server A over WebSocket
2.  Chat Server A persists to Cassandra (status: sent)
3.  Chat Server A publishes to Kafka topic "messages"
4.  Chat Server A → ACK to Client A          → single tick ✓
5.  Chat Server B consumes from Kafka
6.  Chat Server B → delivers to Client B over WebSocket
7.  Client B → delivery ACK to Chat Server B
8.  Chat Server B updates status: delivered in Cassandra
9.  Chat Server B → status event back via Kafka
10. Chat Server A → pushes status to Client A → double tick ✓✓
11. Client B opens the chat → sends read receipt
12. Read receipt propagates to Client A       → blue ticks

The key insight: step 4 happens before step 6. The sender gets immediate confirmation the server durably holds the message, independent of whether the recipient is reachable. Delivery and read are later, separate hops.

3. Offline Message Queueing#

When the recipient is offline, Chat Server B can't deliver over a socket that doesn't exist. The message already sits durably in Cassandra with status sent. When Client B reconnects:

1. Client B connects to any chat server (load balanced)
2. Client B sends its last-seen message ID
3. Server range-scans Cassandra for messages after that ID across all of Client B's conversations
4. Server delivers the backlog over the new WebSocket, in order
5. Client B ACKs, statuses flip to delivered, senders see double ticks

The key: messages are never stored only in memory or only in Kafka. Cassandra is the durable source of truth; the queue is just the fast path for online delivery. If a message is never consumed from Kafka, it's still safely in Cassandra waiting for the reconnect scan.

4. Group Messaging and Fan-Out#

A group message to 500 people is fundamentally different from a 1:1 message. First, separate two things people conflate, storage fan-out and delivery fan-out, because they get different answers here.

Storage: one shared log, not per-member copies. The message is written once to the conversation's partition (messages keyed by conversation_id, per the data model). All members read that same partition; per-member read state lives in conversation_members.last_read_id. That's storage fan-out on read: storing 1,000 copies of every group message would be wasteful and it's not what the schema does.

Delivery: fan out to each online member. Real-time push is per-member: the router resolves each member's server via presence and forwards to whoever is currently connected. This is bounded work because group size is capped, so it stays tractable.

# STORAGE: always one write to the shared conversation log.
append_to_conversation_log(conversation_id, message_id)   # sender ACKed here

# DELIVERY: async, per online member, bounded by the ~1,000 cap.
enqueue_delivery_fanout(message_id, member_ids)   # router → presence → push

Do the delivery fan-out asynchronously: the sender gets its single tick after the one durable log write, and a background job drives the up-to-1,000 directed pushes. For unbounded broadcast channels (millions of subscribers) even delivery fan-out on write breaks, and you flip fully to pull: subscribers fetch the shared log on read, and you drop per-subscriber receipts.

5. Message Ordering#

Within a conversation, messages must arrive in order. Two hazards: clock skew between servers, and concurrent messages from both participants.

Be precise about what solves what here, because a common answer ("just use TIMEUUID") is only half right. A TIMEUUID's high bits are a wall-clock reading (100ns ticks since the Gregorian epoch), generated on the same potentially-skewed server, so TIMEUUID does not by itself immunize you against clock skew. What it buys is global uniqueness and a deterministic tiebreak, and, critically, a single, consistent total order that every reader of the partition agrees on.

conversation_id   (partition key)
message_id        (clustering column, TIMEUUID / UUIDv7, DESC)
sender_id
content
status

Two levers give you correct ordering, not just consistent ordering:

1. Stamp the ID at one authoritative point: the recipient-facing server (or a per-conversation sequencer) that owns the partition, rather than trusting each client/sender's clock. That removes cross-sender skew from the ordering decision.
2. For strict correctness (financial-grade "message N always precedes N+1"), assign a per-conversation monotonic sequence number from that single owner. It costs a counter but eliminates skew entirely; TIMEUUID is the pragmatic default when "consistent order everyone agrees on" is good enough.

Partitioning by conversation_id keeps a whole conversation on one partition, so a range scan returns it already ordered.

Data Model#

-- Messages (Cassandra, partitioned by conversation for ordered range scans)
messages (
  conversation_id  UUID,        -- PARTITION KEY
  message_id       TIMEUUID,    -- CLUSTERING KEY DESC (time-ordered)
  sender_id        UUID,
  content          TEXT,        -- null for media
  media_url        TEXT,        -- null for text
  message_type     VARCHAR,     -- text | image | video | doc
  status           VARCHAR,     -- sent | delivered | read
  created_at       TIMESTAMPTZ,
  PRIMARY KEY (conversation_id, message_id)
)

-- Conversations (Postgres)
conversations (
  id          UUID PRIMARY KEY,
  type        VARCHAR,          -- direct | group
  created_at  TIMESTAMPTZ
)

-- Membership + read tracking (Postgres)
conversation_members (
  conversation_id  UUID,
  user_id          UUID,
  joined_at        TIMESTAMPTZ,
  last_read_id     TIMEUUID,    -- drives read receipts
  PRIMARY KEY (conversation_id, user_id)
)

-- Presence (Redis)
-- Key: user:{id} → server_id (which chat server holds the socket)
-- TTL refreshed by heartbeat; expiry = offline

API Design#

WebSocket: wss://chat.example.com/ws
─────────────────────────────────────────────
Client → Server (send):
  {
    "type": "message",
    "temp_id": "client-uuid",       // idempotency for retries
    "conversation_id": "conv-uuid",
    "content": "Hey!",
    "message_type": "text"
  }

Server → Client (ACK, single tick):
  {
    "type": "ack",
    "temp_id": "client-uuid",
    "message_id": "server-timeuuid",
    "status": "sent"
  }

Server → Client (incoming message):
  {
    "type": "message",
    "message_id": "server-timeuuid",
    "conversation_id": "conv-uuid",
    "sender_id": "user-uuid",
    "content": "Hey!",
    "created_at": "2026-07-02T10:00:00Z"
  }

REST (history / reconnect):
GET /api/v1/conversations/:id/messages?before=<timeuuid>&limit=50
Authorization: Bearer <token>
Response 200:
  { "messages": [ ... ], "has_more": true, "next_cursor": "<timeuuid>" }

Note: sender_id and identity always come from the auth token server-side, never from a client-supplied field. The client's temp_id is only an idempotency hint; the server assigns the authoritative message_id.

Failure Scenarios and Edge Cases#

Code#

Message Send: Persist, Route, and ACK#

public class ChatServer {
    private final CassandraSession cassandra;
    private final KafkaProducer<String, MessageEvent> kafka;
    private final ConnectionRegistry connections;   // sockets held on THIS server
    private final PresenceService presence;          // user:{id} → server_id (Redis)
    private final IdempotencyCache idCache;          // temp_id → message_id (Redis, TTL)
    private final Router router;                      // directed forward to a server

    // Runs on the SENDER's chat server: persist + hand off for routing.
    public void onSend(Session session, InboundMessage in) {
        // 1. Dedup on the client's temp_id, NOT on message_id, and no LWT.
        UUID messageId = idCache.resolveOrCreate(in.tempId(), UUIDs::timeBased);
        if (idCache.wasExisting()) {                 // a retry we already handled
            session.send(new Ack(in.tempId(), messageId, "sent"));
            return;
        }

        // 2. Persist: a plain write. Re-inserting the same (conv, msg) is a
        //    harmless overwrite, so we never pay for IF NOT EXISTS (Paxos).
        cassandra.execute(insert("messages")
            .value("conversation_id", in.conversationId())
            .value("message_id", messageId)
            .value("sender_id", session.userId())
            .value("content", in.content())
            .value("status", "sent"));

        // 3. Hand off to the router. Kafka decouples ingest from delivery.
        kafka.send(new ProducerRecord<>("messages", in.conversationId().toString(),
            new MessageEvent(messageId, in.conversationId(), session.userId(),
                             in.content())));

        // 4. ACK the sender immediately → single tick.
        session.send(new Ack(in.tempId(), messageId, "sent"));
    }

    // ROUTER (separate consumer group): for each recipient, look up the ONE
    // server holding their socket and forward only there, never broadcast to
    // all 1,000 servers.
    public void onKafkaMessage(MessageEvent event) {
        for (UUID recipient : membersOf(event.conversationId())) {
            String serverId = presence.serverFor(recipient);   // null = offline
            if (serverId != null) {
                router.forward(serverId, recipient, event);     // directed
            }
            // offline recipients get it via the reconnect range-scan
        }
    }

    // On the OWNING server: deliver to the local socket.
    public void onDirectedDelivery(UUID recipient, MessageEvent event) {
        Session s = connections.get(recipient);
        if (s != null) s.send(event.toOutbound());
    }
}

Trade-off Summary#

Follow-up Questions#

Basic

Q: Why WebSocket instead of HTTP polling?

Chat is bidirectional and latency-sensitive at massive connection counts. Long polling re-establishes a request per message, huge overhead and not truly real-time. SSE is push-only, so the client can't send over the same channel. WebSocket is one handshake then cheap full-duplex frames, which is exactly the shape of a conversation.

Q: Why does the sender get a single tick before the recipient receives anything?

Because the ACK to the sender is decoupled from delivery. The single tick means "the server durably has your message": it fires as soon as Cassandra write plus Kafka publish succeed. Delivery to the recipient's device (double tick) and read (blue tick) are later, separate acknowledgements that depend on the recipient being reachable.

Q: What happens when the recipient is offline?

Nothing is lost. The message is already durable in Cassandra with status sent. When the recipient reconnects, they send their last-seen ID and the server range-scans for everything newer and delivers it in order. Kafka is only the online fast path; Cassandra is the source of truth.

Q: How is group messaging different from 1:1?

A group send fans out to every member. WhatsApp uses async fan-out on write: the sender is ACKed after one durable write, and a background job delivers to the other members' inboxes. It works because group size is capped, so worst-case fan-out is a bounded constant, unlike an unbounded social feed.

Minute-by-Minute Interview Playbook#

0 to 5 min: Requirements Ask the questions above. Scope 1:1 first, then groups. Nail down receipts, scale, durability, and out-of-scope (no E2E encryption, no calls). Write FR/NFR on the board.

5 to 8 min: Capacity estimates Do the math out loud. The 100M concurrent connections number justifies the entire connection-model discussion; the 1.16M writes/sec justifies Cassandra over Postgres.

8 to 18 min: HLD Draw the WebSocket connection model first, then the path Client → Chat Server → Kafka → Chat Server → Client, with Cassandra hanging off the write. Explain why WebSocket over polling. Get the happy path right before edge cases.

18 to 38 min: Deep dives Lead with the delivery-receipt flow: draw all three tick states step by step. This is the highest-value section. Then offline queueing ("what if the recipient is offline?"), then group fan-out with a concrete threshold, then ordering (TIMEUUID, not wall clock).

38 to 45 min: Failure scenarios and wrap-up Cover the server-crash retry (idempotent on message_id) and Kafka lag (nothing lost, Cassandra is truth). Summarize in one sentence: "WebSocket connections, Kafka routing, Cassandra as the durable source of truth, with the sender ACK decoupled from delivery so nothing is lost and the sender always gets instant feedback."

Green flags

• Reaches for WebSocket immediately and can say why, not just that
• Walks the delivery-receipt flow with all three tick states
• Decouples the sender ACK from recipient delivery deliberately
• Treats group fan-out as a distinct, bounded problem
• Handles the crash/retry duplicate with idempotency keys

Red flags

• Uses HTTP polling for real-time delivery
• Never addresses offline users
• Treats group messages identically to 1:1
• Uses a relational DB as the primary message store at 100B/day
• Orders by wall-clock timestamp and ignores clock skew

Further Reading#