# Sharding and partitioning

- Source: https://intervueclub.com/course/sharding-and-partitioning
- Author: IntervueClub
- Part of: HLD 101 (https://intervueclub.com/course), Storing state
- Last updated: 2026-10-04

When one database has to be split: what actually forces it, how to pick a partition key from your queries, where hot shards come from, and how to add shards to a live system without moving most of the data.

## A primary that can't keep up

Say you're building Instagram, and posts, likes and comments live in one Postgres primary with a read replica. In load tests the primary applies about 5,000 writes a second before its write-ahead log and disk fall behind. A normal day is 3,000 writes a second. Then a launch pushes it to 8,000.

**Question:** The team adds a second read replica. Does the queue of writes waiting on the primary get shorter?

**Answer:**
No. A replica has to apply every write the primary applies, so it adds read capacity and nothing for writes. The primary still receives 8,000 writes a second and applies 5,000, so the backlog grows by 3,000 every second regardless of how many replicas sit behind it.

![Writes arrive at a primary database faster than it can apply them. A queue of writes builds up in front of it. A second read replica is added and the queue keeps growing.](https://intervueclub.com/animations/sharding/one-primary-saturates.light.png)

*Replicas copy every write, so they can't drain a write backlog.*

Three things can run out on a single database, and only one of them forces a split:

| What ran out | Cheaper fix to try first |
|---|---|
| Disk space | A bigger volume, archiving or tiering old rows |
| Working set no longer fits in RAM | More memory, or a cache in front of the hot rows |
| Write throughput | None. This is the one that forces partitioning |

There is also a cheap move between "one table" and "many machines": **vertical partitioning**, which splits columns instead of rows. If every page load reads a user's name and avatar but almost never their 2 KB bio, moving `bio` to its own table can shrink the hot table several times over. It buys time and adds no cross-machine queries.

*The rest of this concept is free to read after signing in: https://intervueclub.com/course/sharding-and-partitioning*