Design LeetCode

Problem Statement#

"Design a coding practice platform like LeetCode. Users pick a problem, write a solution in the browser, hit submit, and get a verdict back. The platform also runs timed contests with a live leaderboard."

Two sentences. Almost everything hard about this problem is hiding behind the word submit.

Requirements Gathering#

Do not start drawing boxes. Start by finding out which parts of this are actually hard, because the answers change the design a lot.

Here is how the conversation should go:

Candidate: "Are we building the practice side, the contest side, or both?"

Interviewer: "Both. They share the same submission path."

Candidate: "How many languages do we support?"

Interviewer: "Start with five or six. Python, Java, C++, JavaScript, Go."

Candidate: "Is there a difference between hitting Run and hitting Submit?"

Interviewer: "Yes. Run executes the two or three sample cases the user can see. Submit runs the full hidden suite, around a hundred cases."

Candidate: "When a solution is wrong, what does the user get back?"

Interviewer: "The verdict, the first failing case with expected versus actual output, and the runtime. The rest of the hidden tests stay hidden."

Candidate: "How fast does the verdict need to come back?"

Interviewer: "A few seconds. Past ten seconds people start refreshing and resubmitting."

Candidate: "How big is the platform? Problems, users, submissions per day?"

Interviewer: "About 4,000 problems, a few hundred thousand daily actives, roughly half a million submissions a day."

Candidate: "And contests?"

Interviewer: "Weekly. 90 minutes, four problems, up to 100,000 participants."

Candidate: "How is a contest scored, and how fresh does the leaderboard have to be?"

Interviewer: "Points for each problem solved, with a time penalty that grows with wrong submissions. A few seconds of staleness is fine."

Candidate: "Can I assume users are already logged in, and leave out payments, editorials, and discussions?"

Interviewer: "Yes. Assume the user id comes from the session."

Functional and Non-Functional Requirements#

Functional Requirements#

1. Users browse a paginated, filterable list of problems
2. Users open a problem and get the starter stub in their chosen language
3. Users run their code against the visible sample cases and get output quickly
4. Users submit against the full hidden suite and get a verdict: Accepted, Wrong Answer, Time Limit Exceeded, Memory Limit Exceeded, Runtime Error, or Compile Error, plus the first failing case and the runtime
5. Contest submissions are scored and show up on a near live leaderboard

Non-Functional Requirements#

Out of scope: authentication, payments, premium tiers, editorials, discussion threads, plagiarism detection.

Capacity Estimates#

Do this out loud. The numbers decide the shape of the system.

Steady state (normal practice traffic):
  500k submissions/day  = ~6/sec average
  evening peak ~5x      = ~30/sec
  Small. A couple of machines could keep up.

Contest burst (this is what sizes the system):
  100k participants x ~8 submissions each = ~800k submissions
  over 90 min (5,400s)                    = ~150/sec average
  but arrivals bunch up: the first and last ten minutes carry ~40% of the volume
  peak                                    = ~600/sec

Execution fleet (the actual cost):
  ~100 hidden cases per submission, ~1.5s of CPU, budget 2s worst case
  600/sec x 2s        = ~1,200 executions in flight at any moment
  1 execution pins ~1 vCPU
  1,200 vCPUs / 16 per node = ~75 judge nodes at peak, ~4 at steady state

Storage:
  4,000 problems x ~200 KB of test data = well under 1 GB
  500k submissions/day x ~2 KB          = ~1 GB/day, ~350 GB/year
  One Postgres handles this for years.

The number worth pausing on is 75 nodes at peak against 4 at steady state, roughly a 20x swing, on the most expensive resource in the system. Nothing else here is difficult: the data is tiny, the read traffic is cacheable, the write rate is low. The design exists to absorb that swing safely and to run hostile code without getting owned.

High-Level Design#

Two independent things are happening. Browsing problems is a read-heavy, mostly static website. Judging code is a compute job that takes seconds and must be contained. Keep them apart.

Submit path. The API writes a submissions row with status queued, pushes a job onto the queue, and returns 202 with the submission id straight away. It does not wait for the code to run.

Judge path. A worker pulls the job, fetches the problem's test suite (from object storage, cached on local disk), starts a fresh sandbox, compiles and runs the code, compares the output, then writes the verdict to Postgres and to Redis. If the submission belongs to a contest, it also updates the leaderboard.

Result path. The client polls GET /submissions/:id every second or so. That endpoint reads Redis, not Postgres.

Browse path. Problem lists and problem statements barely change. Serve them from a CDN with a long TTL and purge on edit. This traffic should never reach the database.

The one idea that carries the rest of the design: the queue is the shock absorber. It lets the API accept 600 submissions a second while the fleet is still scaling up, and it turns "a worker crashed" into "the job gets redelivered" instead of "the user lost their submission."

Deep Dives#

1. A submission is a job, not a request#

The obvious design is POST /submit holding the HTTP connection open until the verdict is ready. It works on your laptop and falls apart in production.

Holding the connection means 600 requests a second each staying open for two to five seconds, so a few thousand connections parked on your API tier doing nothing but waiting. Every proxy in the path has an idle timeout you are now fighting. A slow judge node turns into a user-facing 504. And there is nowhere to put the work, so the fleet has to already be at peak size the moment the contest starts.

Returning 202 { submission_id, status: "queued" } fixes all of that at once. The remaining question is how the result gets back to the user.

Decision: polling, at a one second interval, backing off after ten seconds. Add SSE for contests if the latency is worth it.

2. Running untrusted code#

Someone will submit while True: os.fork(). Someone will try to read your filesystem, open a socket to your cloud metadata endpoint, or simply print the expected answers if you were careless enough to leave them where the code can read them. Assume all of it and design for it.

Decision: a fresh container per submission, one image per language, drawn from a warm pool. It hits the latency budget, it is simple to operate, and the security story is solid once configured properly. I would name microVMs as the next step and say why: this platform invites the whole internet to run code, so paying a few tens of milliseconds for kernel-level separation is a trade worth making as the fleet grows.

Now the configuration, which is where the real answer lives:

• No network at all. Not a firewall rule, no interface (--network=none). This blocks exfiltration and, more importantly, the cloud metadata endpoint at 169.254.169.254, which is how a lot of container escapes turn into stolen credentials.
• Read-only root filesystem, plus a small size-capped tmpfs at /tmp for compiler output.
• cgroup limits: one CPU, 256 MB of memory, and a pid cap around 64 so fork bombs hit a wall instead of the host.
• A wall-clock timeout enforced by the worker, set above the CPU limit, killing the whole cgroup rather than one pid. Never trust the code to stop itself.
• A seccomp profile that denies everything the language runtime does not need: no ptrace, no mount, no module loading.
• Non-root user, all capabilities dropped, no-new-privileges.
• A fresh container every time. Never reuse one. A previous submission can leave a file, a background thread, or a poisoned cache behind.
• Expected outputs never go inside the sandbox. Pipe the inputs in, read what the program printed, compare on the worker. If you mount the answer file next to the user's code, somebody will read it and print it, and their solution will be "Accepted" in 0ms.

That last point catches people out more often than the exotic ones.

3. One test suite, six languages#

The trap is writing test cases per problem per language. At 4,000 problems and six languages that is 24,000 suites to keep in sync, and every new language means redoing all of it. Nobody can maintain that.

Write one language-neutral spec per problem, and one harness per language.

The spec is data:

{
  "entry":  { "name": "maxDepth", "params": [{ "name": "root", "type": "tree" }],
              "returns": "int" },
  "limits": { "time_ms": 2000, "memory_mb": 256 },
  "compare": "exact",
  "cases": [
    { "input": [[3, 9, 20, null, null, 15, 7]], "expected": 3 },
    { "input": [[1, null, 2]],                  "expected": 2 }
  ]
}

Each parameter has a declared type, and each type has one agreed serialisation. A binary tree is a level-order array with null for missing children. A linked list is a flat array. A graph is an adjacency list. The serialisation is written once, in the spec format, and every language just has to agree with it.

The harness is the per-language piece. It ships inside each runtime image, reads the spec on stdin, builds the real objects (a TreeNode in Python, a TreeNode in Java, a TreeNode* in C++), calls the user's function, serialises whatever comes back, and prints one JSON line per case. Six small harnesses, written once, used by every problem forever.

Two details that matter:

Run all cases in one process. Starting a JVM a hundred times costs far more than the code you are measuring. One process, a loop over the cases, and you pay startup once. Submissions stop at the first failure (that is the case you report); Run executes all the samples so the user sees everything.

Comparison is per problem, not global. Exact match covers most cases. Floats need an epsilon. "Return the values in any order" needs a set comparison. Problems with several valid answers need a small validator function instead of a fixed expected value, which is the one place a per-problem checker is unavoidable. And for in-place problems, the thing to check is the array the user mutated, not the return value.

4. The contest leaderboard#

100,000 people refreshing a leaderboard while submissions land continuously. Scoring is points per problem solved, with a time penalty that grows with wrong attempts, so ranking is points descending, then penalty ascending.

Decision: the sorted set. The one puzzle it creates is that a ZSET ranks on a single number, and our ranking has two dimensions. Pack them:

score = points * 10^7 + (10^7 - penalty_seconds)

A 90 minute contest caps penalty at a few thousand seconds, so 10^7 leaves plenty of headroom, and a double holds integers exactly up to about 9 x 10^15, far more than we need. Higher is better in both dimensions, so a single ZREVRANGE gives the correct ranking, tie-break included.

Keep the supporting state in Redis alongside it, hash-tagged so a cluster keeps them on one slot:

{contest:42}:u:1337:solved   SET   problem ids this user has solved
{contest:42}:u:1337:totals   HASH  points, penalty
{contest:42}:board           ZSET  user_id -> packed score

On the read side, cache the top 100 in the API process for two seconds. 100,000 pollers then collapse into one Redis call every two seconds. A user's own rank cannot be shared like that, but ZREVRANK is one cheap logarithmic lookup.

5. Surviving the contest burst#

Four judge nodes on a Tuesday, seventy-five on Sunday at 10:30. Getting that wrong shows up as a three minute wait for a verdict, right when the platform is most visible.

Autoscale on queue depth, not CPU. CPU on a judge node is pinned at 100% by design; that is the machine doing its job. The signal that matters is the age of the oldest message in the queue. Target something like "oldest message under two seconds" and let the fleet size follow.

Pre-warm. Contests are scheduled, so scale up fifteen minutes early. Reactive scaling from 4 nodes to 75 takes minutes you do not have, and cold nodes still need to pull runtime images.

Two queues, two pools. Contest submissions and practice submissions get their own queues with separate (or weighted) worker pools. Contest traffic must not sit behind a backlog of practice submissions, and someone practising on a Sunday morning should not wait three minutes because a contest is running.

Cap concurrency per user. One pending submission per user per problem. It kills accidental double-submits, stops a user's own retry loop from eating capacity, and costs one Redis key to enforce.

Degrade honestly. If the queue does back up, show the truth ("queued, about 20 seconds") rather than a spinner that means nothing. And if you need capacity badly, disabling Run during the final minutes of a contest is a reasonable lever: samples are a convenience, submissions are the product.

Data Model#

-- Problems (Postgres). Small, read-heavy, cached hard at the CDN.
problems (
  id           UUID PRIMARY KEY,
  slug         VARCHAR UNIQUE NOT NULL,      -- "max-depth-of-binary-tree"
  title        VARCHAR NOT NULL,
  difficulty   VARCHAR NOT NULL,             -- easy | medium | hard
  tags         TEXT[],
  statement_md TEXT NOT NULL,
  code_stubs   JSONB NOT NULL,               -- { "python": "...", "java": "..." }
  suite_key    VARCHAR NOT NULL,             -- object-store key, versioned
  suite_version INTEGER NOT NULL
)
-- Index: (difficulty), GIN on tags     powers list filtering
-- Index: (slug)                        the detail page lookup

-- Submissions (Postgres). The only table that really grows.
submissions (
  id            UUID PRIMARY KEY,
  user_id       UUID NOT NULL,
  problem_id    UUID NOT NULL,
  contest_id    UUID,                        -- NULL for practice
  language      VARCHAR NOT NULL,
  code          TEXT NOT NULL,
  status        VARCHAR NOT NULL,            -- queued | running | judged | error
  verdict       VARCHAR,                     -- accepted | wrong_answer | tle | mle | re | ce
  failed_case   INTEGER,
  runtime_ms    INTEGER,
  memory_kb     INTEGER,
  created_at    TIMESTAMPTZ NOT NULL,
  judged_at     TIMESTAMPTZ
)
-- Index: (user_id, created_at DESC)              "my submissions"
-- Index: (contest_id, user_id, problem_id)       scoring and rejudge replay
-- Partial index: (created_at) WHERE status IN ('queued','running')   stuck-job sweeper
-- Partitioned by month on created_at

Test suites live in object storage, not Postgres. They are immutable, they are read by every worker on every submission, and they are the one thing you want cached on local disk keyed by suite_key + suite_version. Bumping the version is also how a rejudge invalidates every worker's cache at once.

Redis holds only things that can be rebuilt:

sub:{submission_id}                 JSON status blob, TTL 1h    (polling)
{contest:42}:board                  ZSET  user -> packed score
{contest:42}:u:{uid}:solved         SET   solved problem ids
{contest:42}:u:{uid}:totals         HASH  points, penalty
pending:{user_id}:{problem_id}      string, TTL 60s             (per-user concurrency cap)

The code column is write-once and can get large. At 350 GB a year Postgres is fine; if it grows past that, store the code in object storage and keep a pointer.

API Design#

GET /api/v1/problems?cursor=&difficulty=medium&tags=tree
─────────────────────────────────────────────
Response 200: { "problems": [ { "slug": "max-depth-of-binary-tree",
                                "title": "Max Depth of Binary Tree",
                                "difficulty": "easy",
                                "tags": ["tree", "dfs"],
                                "solved": true } ],
                "next_cursor": "eyJpZCI6..." }
Cursor pagination, not offsets. Cacheable at the CDN, per-user "solved" merged client-side.

GET /api/v1/problems/:slug?language=python
─────────────────────────────────────────────
Response 200: { "slug": "...", "title": "...", "statement_md": "...",
                "code_stub": "class Solution:\n    def maxDepth(self, root):\n        ",
                "samples": [ { "input": "[3,9,20,null,null,15,7]", "output": "3" } ] }

POST /api/v1/problems/:slug/submissions
─────────────────────────────────────────────
Request:  { "language": "python", "code": "...", "contest_id": "uuid-or-null" }
Response 202: { "submission_id": "uuid", "status": "queued" }
Response 429: { "error": "submission_pending",
                "message": "Your previous submission is still running." }

GET /api/v1/submissions/:id
─────────────────────────────────────────────
Response 200 (pending): { "status": "running" }
Response 200 (done):    { "status": "judged", "verdict": "wrong_answer",
                          "failed_case": 14,
                          "expected": "3", "actual": "2",
                          "runtime_ms": 48, "memory_kb": 17200 }

GET /api/v1/contests/:id/leaderboard?cursor=&limit=100
─────────────────────────────────────────────
Response 200: { "rows": [ { "rank": 1, "user": "alice", "points": 400,
                            "penalty_seconds": 3120 } ],
                "me": { "rank": 8421, "points": 200, "penalty_seconds": 4890 },
                "frozen": false }

Two things to say out loud. The user id comes from the session, never from the request body, or anyone can submit as anyone. And 202 on submit is a deliberate contract: the API is promising the submission is durably queued, not that it has run.

Failure Scenarios and Edge Cases#

Code#

The judge worker#

The part worth writing out: consume a job, run it in a locked-down sandbox, turn the result into a verdict.

FINAL = {"judged", "error"}

def worker_loop(queue, db, redis, sandbox, suites):
    for msg in queue.consume():                       # at-least-once delivery
        sub = db.submission(msg.submission_id)
        if sub.status in FINAL:                       # duplicate delivery, already done
            msg.ack()
            continue
        verdict = judge(sub, sandbox, suites)
        db.finalize(sub.id, verdict)                  # UPDATE ... WHERE status <> 'judged'
        redis.setex(f"sub:{sub.id}", 3600, verdict.json())
        if sub.contest_id:
            update_leaderboard(redis, sub, verdict)   # idempotent, see the Lua script
        msg.ack()

def judge(sub, sandbox, suites):
    suite = suites.load(sub.problem_id)               # cached on local disk by version
    box = sandbox.create(                             # fresh box, never reused
        image=IMAGES[sub.language],
        network=None, read_only=True, tmpfs={"/tmp": "32m"},
        cpus=1.0, memory_mb=suite.memory_mb, pids=64,
        seccomp="judge-profile.json", user="nobody", drop_caps="ALL",
    )
    try:
        build = box.run(COMPILE[sub.language], timeout_s=10)   # compiling is untrusted too
        if build.exit_code != 0:
            return Verdict.compile_error(build.stderr[:4000])

        # All cases in one process: pay interpreter/JVM startup once, not 100 times.
        run = box.run(HARNESS[sub.language],
                      stdin=suite.inputs_json,        # expected outputs never enter the box
                      timeout_s=suite.time_ms / 1000 + 1,   # wall clock above the CPU budget
                      max_stdout_bytes=8 * 1024 * 1024)
        if run.killed_oom:     return Verdict.mle()
        if run.timed_out:      return Verdict.tle()
        if run.exit_code != 0: return Verdict.runtime_error(run.stderr[:4000])

        for i, actual in enumerate(parse_json_lines(run.stdout)):
            if not suite.matches(i, actual):          # exact | epsilon | unordered | custom
                return Verdict.wrong_answer(case=i, expected=suite.expected(i), actual=actual)
        return Verdict.accepted(runtime_ms=run.user_time_ms, memory_kb=run.peak_memory_kb)
    finally:
        box.destroy()                                 # tear down even on an exception

The leaderboard update#

This runs on Redis as one script so the whole update is atomic, and it is safe to replay because the SADD gates everything after it.

-- KEYS: 1 solved-set, 2 user-totals hash, 3 board zset  (same hash tag, one slot)
-- ARGV: 1 problem_id, 2 user_id, 3 points, 4 penalty_seconds
if redis.call('SADD', KEYS[1], ARGV[1]) == 0 then
  return 0                          -- already solved: a redelivered job changes nothing
end
local points  = redis.call('HINCRBY', KEYS[2], 'points',  ARGV[3])
local penalty = redis.call('HINCRBY', KEYS[2], 'penalty', ARGV[4])

-- Pack two ranking dimensions into one score: points desc, then penalty asc.
redis.call('ZADD', KEYS[3], points * 10000000 + (10000000 - penalty), ARGV[2])
return 1

Trade-off Summary#

Follow-up Questions#

Basic

Q: Why return 202 instead of just waiting for the result?

Judging takes two to five seconds, and during a contest there are hundreds of submissions a second. Holding those connections open ties up the API tier, fights every proxy timeout in the path, and turns a slow worker into a user-facing error. Queuing the job means the API stays fast, the work is durable, and a crashed worker results in a redelivery rather than a lost submission.

Q: Why a container per submission rather than a VM?

A VM boots in seconds, which spends the entire latency budget before the user's code starts. A container from a warm pool starts in tens of milliseconds and, once you strip the network, mount the root read-only, apply cgroup limits and a seccomp profile, is contained well enough for a disposable judge node with nothing worth stealing on it. If you want kernel-level separation without the boot cost, that is what Firecracker microVMs are for.

Q: How do you stop an infinite loop?

Three layers. A CPU quota through cgroups, so the process cannot consume more than one core. A wall-clock timeout enforced by the worker outside the sandbox, which SIGKILLs the whole cgroup rather than one pid. And a pid limit, so a fork bomb hits a wall instead of the host. The code never gets a say in whether it stops.

Q: Why not just query the submissions table for the leaderboard?

Because 100,000 people refreshing every few seconds means a full aggregation per refresh over hundreds of thousands of rows, to answer a question whose answer barely changed. A Redis sorted set is updated once per accepted submission, and after that a page of the board is one ZREVRANGE and a user's own rank is one ZREVRANK.

Q: Why not write test cases separately for each language?

4,000 problems times six languages is 24,000 suites to keep in sync, and every new language repeats the whole job. Writing one language-neutral spec per problem and one harness per language reduces the per-language work to a single small program, written once.

Minute-by-Minute Interview Playbook#

0 to 5 min: Requirements

Ask the questions above. The ones that matter are Run versus Submit, the five second verdict budget, and contest scale. Confirm auth and payments are out. Write the requirements down, and put "untrusted code isolation" at the top of the non-functional list where the interviewer can see it.

5 to 10 min: Capacity estimates

Do the math out loud and land on the two numbers that matter: about 600 submissions a second at contest peak, and roughly 1,200 concurrent executions, which is about 75 judge nodes against 4 at steady state. Say the quiet part too: this platform is small, and most of it is a normal web app. That earns you credit rather than losing it.

10 to 18 min: High-level design

Draw the two paths separately. Browse is cached and boring. Submit goes API to queue to worker to sandbox, with 202 and polling. State up front that a submission is a job, not a request, and say why.

18 to 40 min: Deep dives

Lead with the sandbox. This is the part the interviewer wants, so do not wait to be asked. Name the options (language sandbox, VM, container, microVM), pick containers, and then list the actual configuration: no network, read-only root, cgroup CPU and memory and pid limits, seccomp, non-root, fresh box each time, and expected outputs kept outside the box. Then the multi-language test harness, because it proves you have thought past the boxes. Then the leaderboard with the packed sorted-set score. Then autoscaling on queue depth if there is time.

40 to 45 min: Failure and wrap-up

Cover worker death and redelivery, why the leaderboard update is idempotent, and the rejudge path. Close in one sentence: "Treat every submission as a durable job, run it in a throwaway sandbox with no network and hard resource caps, judge it against one language-neutral test spec, and keep the leaderboard in a sorted set that can always be rebuilt from Postgres."

Green flags

• Reaches for a queue and 202 without being pushed
• Talks about the sandbox as a security problem and names real controls, not just "we use Docker"
• Realises the expected outputs must not be readable from inside the sandbox
• Notices polling load is bounded by in-flight submissions, not by online users
• Designs one test spec and a harness per language instead of per-language suites
• Makes the leaderboard update idempotent because the queue is at-least-once
• Says plainly that this is a small system and resists over-engineering it

Red flags

• Runs user code in the API process, or relies on a language-level sandbox
• Holds the HTTP connection open until judging finishes
• Says "Docker" and stops, with no mention of network, CPU, memory, pids, or syscalls
• Reuses containers between submissions to save start time
• Recomputes the leaderboard with a GROUP BY on every refresh
• Ignores duplicate delivery and increments scores directly
• Designs for a billion users when the interviewer said a few hundred thousand

Further Reading#