# Design LeetCode - Source: https://intervueclub.com/problems/distributed-systems/design-leetcode - Author: Dhananjay Aggarwal - Category: Distributed Systems - Difficulty: medium - Asked at: Google, Amazon, Microsoft, LinkedIn, Uber - Last updated: 2026-09-19 A complete system design walkthrough for a coding practice platform like LeetCode. Covers why a submission is a job and not a request, sandboxing untrusted code with containers and microVMs, running one test suite against six languages, a Redis sorted set leaderboard for 100k contest users, autoscaling the judge fleet on queue depth, failure scenarios, and everything an interviewer actually evaluates. ## Problem Statement > "Design a coding practice platform like LeetCode. Users pick a problem, write a solution in the browser, hit submit, and get a verdict back. The platform also runs timed contests with a live leaderboard." Two sentences. Almost everything hard about this problem is hiding behind the word *submit*. Most of this system is an ordinary CRUD app: a list of problems, a detail page, a submissions table. Then there is one component that runs code written by strangers on your servers, and that component is the entire interview. In this design you'll learn: 1. Why a submission has to be a background job instead of a plain HTTP request, and how the client gets the result back 2. How to actually run untrusted code safely (containers, microVMs, seccomp, cgroups) and the mistakes that leak your test data 3. How one set of test cases per problem can be run against six different languages without writing six test suites 4. How to keep a leaderboard live for 100,000 people without melting your database A useful thing to notice early: this platform is small. A few hundred thousand users, a few thousand problems. Do not over-engineer it. ## Requirements Gathering Do not start drawing boxes. Start by finding out which parts of this are actually hard, because the answers change the design a lot. Here is how the conversation should go: --- **Candidate:** "Are we building the practice side, the contest side, or both?" **Interviewer:** "Both. They share the same submission path." --- **Candidate:** "How many languages do we support?" **Interviewer:** "Start with five or six. Python, Java, C++, JavaScript, Go." --- **Candidate:** "Is there a difference between hitting Run and hitting Submit?" **Interviewer:** "Yes. Run executes the two or three sample cases the user can see. Submit runs the full hidden suite, around a hundred cases." --- **Candidate:** "When a solution is wrong, what does the user get back?" **Interviewer:** "The verdict, the first failing case with expected versus actual output, and the runtime. The rest of the hidden tests stay hidden." --- **Candidate:** "How fast does the verdict need to come back?" **Interviewer:** "A few seconds. Past ten seconds people start refreshing and resubmitting." --- **Candidate:** "How big is the platform? Problems, users, submissions per day?" **Interviewer:** "About 4,000 problems, a few hundred thousand daily actives, roughly half a million submissions a day." --- **Candidate:** "And contests?" **Interviewer:** "Weekly. 90 minutes, four problems, up to 100,000 participants." --- **Candidate:** "How is a contest scored, and how fresh does the leaderboard have to be?" **Interviewer:** "Points for each problem solved, with a time penalty that grows with wrong submissions. A few seconds of staleness is fine." --- **Candidate:** "Can I assume users are already logged in, and leave out payments, editorials, and discussions?" **Interviewer:** "Yes. Assume the user id comes from the session." --- ## Functional and Non-Functional Requirements ### Functional Requirements 1. Users browse a paginated, filterable list of problems 2. Users open a problem and get the starter stub in their chosen language 3. Users run their code against the visible sample cases and get output quickly 4. Users submit against the full hidden suite and get a verdict: Accepted, Wrong Answer, Time Limit Exceeded, Memory Limit Exceeded, Runtime Error, or Compile Error, plus the first failing case and the runtime 5. Contest submissions are scored and show up on a near live leaderboard ### Non-Functional Requirements | Requirement | Target | |---|---| | Isolation | Untrusted code must not escape the sandbox, reach the network, or see another submission. **Hard constraint** | | Verdict latency | p95 under 5s from submit to verdict | | Durability | Once the API accepts a submission it is never silently dropped | | Consistency | Browse and leaderboard favour availability; the verdict itself must be correct and stable | | Contest load | 100,000 participants, bursts around 600 submissions/sec | **Out of scope:** authentication, payments, premium tiers, editorials, discussion threads, plagiarism detection. ### Capacity Estimates Do this out loud. The numbers decide the shape of the system. ``` Steady state (normal practice traffic): 500k submissions/day = ~6/sec average evening peak ~5x = ~30/sec Small. A couple of machines could keep up. Contest burst (this is what sizes the system): 100k participants x ~8 submissions each = ~800k submissions over 90 min (5,400s) = ~150/sec average but arrivals bunch up: the first and last ten minutes carry ~40% of the volume peak = ~600/sec Execution fleet (the actual cost): ~100 hidden cases per submission, ~1.5s of CPU, budget 2s worst case 600/sec x 2s = ~1,200 executions in flight at any moment 1 execution pins ~1 vCPU 1,200 vCPUs / 16 per node = ~75 judge nodes at peak, ~4 at steady state Storage: 4,000 problems x ~200 KB of test data = well under 1 GB 500k submissions/day x ~2 KB = ~1 GB/day, ~350 GB/year One Postgres handles this for years. ``` The number worth pausing on is **75 nodes at peak against 4 at steady state**, roughly a 20x swing, on the most expensive resource in the system. Nothing else here is difficult: the data is tiny, the read traffic is cacheable, the write rate is low. The design exists to absorb that swing safely and to run hostile code without getting owned. --- ## High-Level Design Two independent things are happening. Browsing problems is a read-heavy, mostly static website. Judging code is a compute job that takes seconds and must be contained. Keep them apart. ```mermaid flowchart LR C["Client
Monaco editor"] API["API Service"] DB[("Postgres
problems · submissions")] S3[("Object Store
test suites")] Q["Submission Queue
SQS / Kafka"] W["Judge Workers
autoscaled"] BOX["Sandbox
one per submission"] R[("Redis
status · leaderboard")] C -->|"GET /problems"| API C -->|"POST /submissions"| API C -->|"GET /submissions/:id"| API API --> DB API --> Q API --> R Q --> W W --> BOX W --> S3 W --> DB W --> R ``` **Submit path.** The API writes a `submissions` row with status `queued`, pushes a job onto the queue, and returns `202` with the submission id straight away. It does not wait for the code to run. **Judge path.** A worker pulls the job, fetches the problem's test suite (from object storage, cached on local disk), starts a fresh sandbox, compiles and runs the code, compares the output, then writes the verdict to Postgres and to Redis. If the submission belongs to a contest, it also updates the leaderboard. **Result path.** The client polls `GET /submissions/:id` every second or so. That endpoint reads Redis, not Postgres. **Browse path.** Problem lists and problem statements barely change. Serve them from a CDN with a long TTL and purge on edit. This traffic should never reach the database. The one idea that carries the rest of the design: **the queue is the shock absorber.** It lets the API accept 600 submissions a second while the fleet is still scaling up, and it turns "a worker crashed" into "the job gets redelivered" instead of "the user lost their submission." --- ## Deep Dives ### 1. A submission is a job, not a request The obvious design is `POST /submit` holding the HTTP connection open until the verdict is ready. It works on your laptop and falls apart in production. Holding the connection means 600 requests a second each staying open for two to five seconds, so a few thousand connections parked on your API tier doing nothing but waiting. Every proxy in the path has an idle timeout you are now fighting. A slow judge node turns into a user-facing `504`. And there is nowhere to put the work, so the fleet has to already be at peak size the moment the contest starts. Returning `202 { submission_id, status: "queued" }` fixes all of that at once. The remaining question is how the result gets back to the user. #### Client polls GET /submissions/:id (Recommended) The client asks every second until the status is final. The payload is a few hundred bytes and comes from Redis. The load is smaller than people expect, because **a user only polls while they have a submission in flight.** The number of pollers is bounded by pending submissions (about 1,200 at contest peak), not by the 100,000 people on the site. That is a few thousand requests a second against Redis, which is nothing. #### Server-sent events (Situational) One long-lived connection per pending submission, verdict pushed the instant it lands. Saves maybe a second of perceived latency and a handful of requests. Worth adding later for contests, where a second matters and the page is already open. Not worth the connection-management work at launch. #### WebSocket for verdicts (Avoided) A full duplex channel to deliver one small message, once, in response to an action the user just took. You now own connection state, reconnects, and sticky routing for no real gain. Reach for WebSockets when the server pushes unprompted updates, which is not this. **Decision:** polling, at a one second interval, backing off after ten seconds. Add SSE for contests if the latency is worth it. ### **For senior interviews ↓** at-least-once delivery, and why the result write must be idempotent Every real queue delivers at least once. A worker that finishes judging and dies before acking will have its job redelivered, and the submission gets judged twice. That is fine as long as nothing downstream double-counts. Make the finalise write conditional: `UPDATE submissions SET verdict = ..., status = 'judged' WHERE id = ? AND status <> 'judged'`. A duplicate run either writes the same verdict or hits the guard. The leaderboard is the part that actually breaks. If the worker does `INCRBY score`, a redelivery silently gives someone extra points. Model it as **state, not increments**: add the problem to a per-user set of solved problems and only score when the `SADD` returns 1 (see deep dive 4). Replays then become no-ops. Also size the queue's visibility timeout above your worst-case execution time, or a slow submission gets redelivered while it is still running and you pay for it twice. And run a sweeper over `status IN ('queued','running') AND created_at < now() - interval '2 minutes'` to catch jobs that vanished entirely, because a submission stuck on a spinner forever is worse than an error. --- ### 2. Running untrusted code Someone will submit `while True: os.fork()`. Someone will try to read your filesystem, open a socket to your cloud metadata endpoint, or simply print the expected answers if you were careless enough to leave them where the code can read them. Assume all of it and design for it. #### Run it in the API process with a language-level sandbox (Avoided) Restricted `eval`, import allowlists, a custom classloader. Every sandbox of this kind has eventually been escaped, and none of them contain CPU or memory: one infinite loop takes an API node down with it. This is the answer that ends the interview early. #### A fresh VM per submission (Situational) Genuinely strong isolation, its own kernel, nothing shared. But a full VM takes seconds to boot, which spends your entire latency budget before a line of user code runs, and the per-submission cost is high. Keeping a pool of warm VMs helps, though you then have to scrub each one between submissions, which is the part people underestimate. #### A container per submission (Recommended) One image per language, pre-pulled on every judge node. Start time is tens of milliseconds from a warm pool, the cost is low, and the isolation is good once you actually lock it down. The honest caveat: containers share the host kernel, so a kernel bug is a real escape path. That is an acceptable risk when the blast radius is a disposable judge node holding no secrets. #### A microVM per submission (Firecracker or gVisor) (Recommended) Firecracker boots a stripped VM in roughly 125ms with its own kernel; gVisor intercepts syscalls in userspace instead. Container-like economics with a much smaller escape surface. This is what AWS Lambda runs under, and it exists precisely because "run a stranger's code cheaply and safely" is a common problem. **Decision:** a fresh container per submission, one image per language, drawn from a warm pool. It hits the latency budget, it is simple to operate, and the security story is solid once configured properly. I would name microVMs as the next step and say why: this platform invites the whole internet to run code, so paying a few tens of milliseconds for kernel-level separation is a trade worth making as the fleet grows. Now the configuration, which is where the real answer lives: - **No network at all.** Not a firewall rule, no interface (`--network=none`). This blocks exfiltration and, more importantly, the cloud metadata endpoint at `169.254.169.254`, which is how a lot of container escapes turn into stolen credentials. - **Read-only root filesystem**, plus a small size-capped `tmpfs` at `/tmp` for compiler output. - **cgroup limits:** one CPU, 256 MB of memory, and a pid cap around 64 so fork bombs hit a wall instead of the host. - **A wall-clock timeout enforced by the worker**, set above the CPU limit, killing the whole cgroup rather than one pid. Never trust the code to stop itself. - **A seccomp profile** that denies everything the language runtime does not need: no `ptrace`, no `mount`, no module loading. - **Non-root user, all capabilities dropped, `no-new-privileges`.** - **A fresh container every time.** Never reuse one. A previous submission can leave a file, a background thread, or a poisoned cache behind. - **Expected outputs never go inside the sandbox.** Pipe the inputs in, read what the program printed, compare on the worker. If you mount the answer file next to the user's code, somebody will read it and print it, and their solution will be "Accepted" in 0ms. That last point catches people out more often than the exotic ones. ### **For senior interviews ↓** blast radius, and compilation is untrusted too Isolation is not only about the sandbox. Assume one submission eventually escapes and ask what it reaches. Judge nodes belong in their own account or VPC, with an instance role that can do nothing interesting, no route to the primary database, and no shared credentials. Workers pull test data through a signed URL or a narrow read-only service, so a compromised node has nothing worth stealing and no path to the rest of the platform. Recycle nodes on a schedule so a quiet foothold does not become permanent. **Compilation is code execution too.** A C++ template recursion can hang a compiler for minutes and eat all the memory on the box; `#include` of odd paths can leak file contents into an error message. Compile inside the same sandbox, with its own shorter timeout and its own memory cap, and treat a compile timeout as Compile Error rather than an infrastructure failure. One more: outputs are attacker-controlled. Cap stdout (a program printing gigabytes should be killed, not buffered), strip control characters, and truncate stderr before it is stored or rendered. --- ### 3. One test suite, six languages The trap is writing test cases per problem per language. At 4,000 problems and six languages that is 24,000 suites to keep in sync, and every new language means redoing all of it. Nobody can maintain that. Write **one language-neutral spec per problem**, and **one harness per language**. The spec is data: ```json { "entry": { "name": "maxDepth", "params": [{ "name": "root", "type": "tree" }], "returns": "int" }, "limits": { "time_ms": 2000, "memory_mb": 256 }, "compare": "exact", "cases": [ { "input": [[3, 9, 20, null, null, 15, 7]], "expected": 3 }, { "input": [[1, null, 2]], "expected": 2 } ] } ``` Each parameter has a declared type, and each type has one agreed serialisation. A binary tree is a level-order array with `null` for missing children. A linked list is a flat array. A graph is an adjacency list. The serialisation is written once, in the spec format, and every language just has to agree with it. The harness is the per-language piece. It ships inside each runtime image, reads the spec on stdin, builds the real objects (a `TreeNode` in Python, a `TreeNode` in Java, a `TreeNode*` in C++), calls the user's function, serialises whatever comes back, and prints one JSON line per case. Six small harnesses, written once, used by every problem forever. Two details that matter: **Run all cases in one process.** Starting a JVM a hundred times costs far more than the code you are measuring. One process, a loop over the cases, and you pay startup once. Submissions stop at the first failure (that is the case you report); Run executes all the samples so the user sees everything. **Comparison is per problem, not global.** Exact match covers most cases. Floats need an epsilon. "Return the values in any order" needs a set comparison. Problems with several valid answers need a small validator function instead of a fixed expected value, which is the one place a per-problem checker is unavoidable. And for in-place problems, the thing to check is the array the user mutated, not the return value. ### **For senior interviews ↓** why the same submission passes on one node and fails on another Time Limit Exceeded is the least deterministic verdict you will ship, and users notice immediately when it flaps. Measure inside the harness, around the user's function call only, so process start and JIT warmup are not charged to them. Then stop the noise at the source: pin one vCPU per execution and do not oversubscribe judge nodes. If two submissions share a core, one of them gets an unfair verdict. Set the limit generously, three to five times the reference solution, so borderline noise cannot flip a verdict. A limit tight enough to separate a good solution from a slightly slower good solution is a limit tight enough to fail the same code twice out of ten runs. Instruction counting (via `perf` or a userspace interpreter) gives near-perfect determinism if you need it, at the cost of much more machinery. Most platforms do not bother. Finally, be honest about "faster than 92% of submissions". Runtimes are not comparable across languages, and the distribution shifts as hardware changes. It is a fun UX number, not a measurement, and a good candidate says so rather than defending it. --- ### 4. The contest leaderboard 100,000 people refreshing a leaderboard while submissions land continuously. Scoring is points per problem solved, with a time penalty that grows with wrong attempts, so ranking is *points descending, then penalty ascending*. #### Group by over the submissions table on every request (Avoided) A `GROUP BY user_id` over hundreds of thousands of contest rows, once per viewer, every few seconds. The database is doing full aggregations to answer a question whose answer barely changed since the last one. This falls over long before 100k users. #### Recompute the whole board every few seconds into a cache (Situational) One background job aggregates and writes the ranked list to a cache; everyone reads the cache. Simple, and honestly fine for a few thousand participants. It breaks down on the personal question. The board is one blob, so answering "what is *my* rank" for the person sitting at position 47,000 means scanning it. And at 100k participants that blob is megabytes being rebuilt constantly. #### Redis sorted set, updated as verdicts land (Recommended) The board is a ZSET keyed by user id. A worker updates it the moment an Accepted verdict is written. Reads are `ZREVRANGE` for a page and `ZREVRANK` for one user's position, both cheap at any size. **Decision:** the sorted set. The one puzzle it creates is that a ZSET ranks on a single number, and our ranking has two dimensions. Pack them: ``` score = points * 10^7 + (10^7 - penalty_seconds) ``` A 90 minute contest caps penalty at a few thousand seconds, so `10^7` leaves plenty of headroom, and a double holds integers exactly up to about `9 x 10^15`, far more than we need. Higher is better in both dimensions, so a single `ZREVRANGE` gives the correct ranking, tie-break included. Keep the supporting state in Redis alongside it, hash-tagged so a cluster keeps them on one slot: ``` {contest:42}:u:1337:solved SET problem ids this user has solved {contest:42}:u:1337:totals HASH points, penalty {contest:42}:board ZSET user_id -> packed score ``` On the read side, cache the top 100 in the API process for two seconds. 100,000 pollers then collapse into one Redis call every two seconds. A user's own rank cannot be shared like that, but `ZREVRANK` is one cheap logarithmic lookup. ### **For senior interviews ↓** freezing, rejudging, and treating Redis as disposable **Redis is a cache here, not the source of truth.** The `submissions` table is authoritative. That is not a formality: it is what makes everything below possible, because the board can always be rebuilt from scratch. **Freeze the last 15 minutes.** Stop publishing updates near the end of a contest. It is a tradition worth keeping (nobody can reverse-engineer who is about to overtake them) and it happens to cut your read load exactly when submission load peaks. **Rejudging.** A bad test case gets spotted 40 minutes into a contest more often than you would like. Fix the suite, replay every affected submission through the queue, recompute the board into a **shadow key**, then `RENAME` it over the live one so viewers never see a half-built leaderboard. This works only because the submissions table kept everything. **Losing Redis mid-contest** should be an inconvenience, not an incident. Rebuild from the submissions table, which takes seconds at this size, and serve a "leaderboard updating" state in the meantime. Verdicts keep flowing the whole time, because judging never depended on the board. --- ### 5. Surviving the contest burst Four judge nodes on a Tuesday, seventy-five on Sunday at 10:30. Getting that wrong shows up as a three minute wait for a verdict, right when the platform is most visible. **Autoscale on queue depth, not CPU.** CPU on a judge node is pinned at 100% by design; that is the machine doing its job. The signal that matters is the age of the oldest message in the queue. Target something like "oldest message under two seconds" and let the fleet size follow. **Pre-warm.** Contests are scheduled, so scale up fifteen minutes early. Reactive scaling from 4 nodes to 75 takes minutes you do not have, and cold nodes still need to pull runtime images. **Two queues, two pools.** Contest submissions and practice submissions get their own queues with separate (or weighted) worker pools. Contest traffic must not sit behind a backlog of practice submissions, and someone practising on a Sunday morning should not wait three minutes because a contest is running. **Cap concurrency per user.** One pending submission per user per problem. It kills accidental double-submits, stops a user's own retry loop from eating capacity, and costs one Redis key to enforce. **Degrade honestly.** If the queue does back up, show the truth ("queued, about 20 seconds") rather than a spinner that means nothing. And if you need capacity badly, disabling Run during the final minutes of a contest is a reasonable lever: samples are a convenience, submissions are the product. --- ## Data Model ```sql -- Problems (Postgres). Small, read-heavy, cached hard at the CDN. problems ( id UUID PRIMARY KEY, slug VARCHAR UNIQUE NOT NULL, -- "max-depth-of-binary-tree" title VARCHAR NOT NULL, difficulty VARCHAR NOT NULL, -- easy | medium | hard tags TEXT[], statement_md TEXT NOT NULL, code_stubs JSONB NOT NULL, -- { "python": "...", "java": "..." } suite_key VARCHAR NOT NULL, -- object-store key, versioned suite_version INTEGER NOT NULL ) -- Index: (difficulty), GIN on tags powers list filtering -- Index: (slug) the detail page lookup -- Submissions (Postgres). The only table that really grows. submissions ( id UUID PRIMARY KEY, user_id UUID NOT NULL, problem_id UUID NOT NULL, contest_id UUID, -- NULL for practice language VARCHAR NOT NULL, code TEXT NOT NULL, status VARCHAR NOT NULL, -- queued | running | judged | error verdict VARCHAR, -- accepted | wrong_answer | tle | mle | re | ce failed_case INTEGER, runtime_ms INTEGER, memory_kb INTEGER, created_at TIMESTAMPTZ NOT NULL, judged_at TIMESTAMPTZ ) -- Index: (user_id, created_at DESC) "my submissions" -- Index: (contest_id, user_id, problem_id) scoring and rejudge replay -- Partial index: (created_at) WHERE status IN ('queued','running') stuck-job sweeper -- Partitioned by month on created_at ``` Test suites live in object storage, not Postgres. They are immutable, they are read by every worker on every submission, and they are the one thing you want cached on local disk keyed by `suite_key + suite_version`. Bumping the version is also how a rejudge invalidates every worker's cache at once. Redis holds only things that can be rebuilt: ``` sub:{submission_id} JSON status blob, TTL 1h (polling) {contest:42}:board ZSET user -> packed score {contest:42}:u:{uid}:solved SET solved problem ids {contest:42}:u:{uid}:totals HASH points, penalty pending:{user_id}:{problem_id} string, TTL 60s (per-user concurrency cap) ``` The `code` column is write-once and can get large. At 350 GB a year Postgres is fine; if it grows past that, store the code in object storage and keep a pointer. --- ## API Design ``` GET /api/v1/problems?cursor=&difficulty=medium&tags=tree ───────────────────────────────────────────── Response 200: { "problems": [ { "slug": "max-depth-of-binary-tree", "title": "Max Depth of Binary Tree", "difficulty": "easy", "tags": ["tree", "dfs"], "solved": true } ], "next_cursor": "eyJpZCI6..." } Cursor pagination, not offsets. Cacheable at the CDN, per-user "solved" merged client-side. GET /api/v1/problems/:slug?language=python ───────────────────────────────────────────── Response 200: { "slug": "...", "title": "...", "statement_md": "...", "code_stub": "class Solution:\n def maxDepth(self, root):\n ", "samples": [ { "input": "[3,9,20,null,null,15,7]", "output": "3" } ] } POST /api/v1/problems/:slug/submissions ───────────────────────────────────────────── Request: { "language": "python", "code": "...", "contest_id": "uuid-or-null" } Response 202: { "submission_id": "uuid", "status": "queued" } Response 429: { "error": "submission_pending", "message": "Your previous submission is still running." } GET /api/v1/submissions/:id ───────────────────────────────────────────── Response 200 (pending): { "status": "running" } Response 200 (done): { "status": "judged", "verdict": "wrong_answer", "failed_case": 14, "expected": "3", "actual": "2", "runtime_ms": 48, "memory_kb": 17200 } GET /api/v1/contests/:id/leaderboard?cursor=&limit=100 ───────────────────────────────────────────── Response 200: { "rows": [ { "rank": 1, "user": "alice", "points": 400, "penalty_seconds": 3120 } ], "me": { "rank": 8421, "points": 200, "penalty_seconds": 4890 }, "frozen": false } ``` Two things to say out loud. The user id comes from the session, never from the request body, or anyone can submit as anyone. And `202` on submit is a deliberate contract: the API is promising the submission is durably queued, not that it has run. --- ## Failure Scenarios and Edge Cases ### **For senior interviews ↓** what breaks and how the system recovers **A worker dies mid-execution** The message is never acked, the visibility timeout expires, another worker picks it up. The user waits a bit longer and still gets a verdict. This is the main reason the queue is there, and it is why the visibility timeout must be longer than the worst-case execution, otherwise healthy slow jobs get redelivered and run twice. **The same submission gets judged twice** At-least-once delivery guarantees this eventually. The finalise write is conditional on `status <> 'judged'`, and the leaderboard update is gated by a `SADD` that returns 0 on replay, so the second run changes nothing. **A submission crashes the sandbox every time** Some inputs break a runtime, and that job will be retried forever if you let it. Cap redeliveries (three is plenty), route the message to a dead letter queue, mark the submission `error` with "internal error, please resubmit", and alert. An honest error beats an infinite retry loop, for the user and for the fleet. **The queue backs up at contest start** Depth-based autoscaling plus pre-warming should stop it, but if it happens: show real queue position instead of a spinner, keep the contest queue isolated from practice, and shed Run requests before you shed Submits. **Redis goes down** Status polling falls back to reading Postgres directly, which is slower but correct. The leaderboard serves a "rebuilding" state and gets reconstructed from the submissions table in seconds. Nothing is lost, because Redis never held anything authoritative. **A test case turns out to be wrong during a contest** Fix the suite, bump `suite_version` (which invalidates every worker's local cache), replay the affected submissions through the queue, rebuild the board into a shadow key, and `RENAME` it into place. Announce it. This is only survivable because every submission's source code was stored. **User code tries to read the expected outputs** It cannot, because they were never inside the sandbox. Inputs go in on stdin, output comes back on stdout, and the comparison happens on the worker. Worth stating explicitly in the interview, since it is a design decision and not a configuration flag. **A program prints gigabytes to stdout** Cap output and kill the process when it exceeds the cap. Unbounded stdout is a memory exhaustion attack on the worker, not on the sandbox. **Time Limit Exceeded flapping on the same code** One vCPU pinned per execution, no oversubscription on judge nodes, and a limit set well above the reference solution. If a verdict depends on which machine the job landed on, the verdict is broken. --- ## Code ### The judge worker The part worth writing out: consume a job, run it in a locked-down sandbox, turn the result into a verdict. **Python** ```python FINAL = {"judged", "error"} def worker_loop(queue, db, redis, sandbox, suites): for msg in queue.consume(): # at-least-once delivery sub = db.submission(msg.submission_id) if sub.status in FINAL: # duplicate delivery, already done msg.ack() continue verdict = judge(sub, sandbox, suites) db.finalize(sub.id, verdict) # UPDATE ... WHERE status <> 'judged' redis.setex(f"sub:{sub.id}", 3600, verdict.json()) if sub.contest_id: update_leaderboard(redis, sub, verdict) # idempotent, see the Lua script msg.ack() def judge(sub, sandbox, suites): suite = suites.load(sub.problem_id) # cached on local disk by version box = sandbox.create( # fresh box, never reused image=IMAGES[sub.language], network=None, read_only=True, tmpfs={"/tmp": "32m"}, cpus=1.0, memory_mb=suite.memory_mb, pids=64, seccomp="judge-profile.json", user="nobody", drop_caps="ALL", ) try: build = box.run(COMPILE[sub.language], timeout_s=10) # compiling is untrusted too if build.exit_code != 0: return Verdict.compile_error(build.stderr[:4000]) # All cases in one process: pay interpreter/JVM startup once, not 100 times. run = box.run(HARNESS[sub.language], stdin=suite.inputs_json, # expected outputs never enter the box timeout_s=suite.time_ms / 1000 + 1, # wall clock above the CPU budget max_stdout_bytes=8 * 1024 * 1024) if run.killed_oom: return Verdict.mle() if run.timed_out: return Verdict.tle() if run.exit_code != 0: return Verdict.runtime_error(run.stderr[:4000]) for i, actual in enumerate(parse_json_lines(run.stdout)): if not suite.matches(i, actual): # exact | epsilon | unordered | custom return Verdict.wrong_answer(case=i, expected=suite.expected(i), actual=actual) return Verdict.accepted(runtime_ms=run.user_time_ms, memory_kb=run.peak_memory_kb) finally: box.destroy() # tear down even on an exception ``` **Java** ```java public final class JudgeWorker { private static final Set FINAL = Set.of("judged", "error"); public void loop() { for (Message msg : queue.consume()) { // at-least-once delivery Submission sub = db.submission(msg.submissionId()); if (FINAL.contains(sub.status())) { // duplicate delivery, already done msg.ack(); continue; } Verdict v = judge(sub); db.finalize(sub.id(), v); // UPDATE ... WHERE status <> 'judged' redis.setex("sub:" + sub.id(), 3600, v.toJson()); if (sub.contestId() != null) updateLeaderboard(sub, v); // idempotent msg.ack(); } } private Verdict judge(Submission sub) { Suite suite = suites.load(sub.problemId()); // cached on local disk by version Sandbox box = sandbox.create(SandboxSpec.builder() .image(IMAGES.get(sub.language())) .network(false).readOnlyRoot(true).tmpfs("/tmp", "32m") .cpus(1.0).memoryMb(suite.memoryMb()).pidLimit(64) .seccomp("judge-profile.json").user("nobody").dropAllCaps() .build()); // fresh box, never reused try { Result build = box.run(COMPILE.get(sub.language()), Duration.ofSeconds(10)); if (build.exitCode() != 0) // compiling is untrusted too return Verdict.compileError(truncate(build.stderr(), 4000)); // All cases in one process: pay JVM startup once, not 100 times. Result run = box.run(HARNESS.get(sub.language()), suite.inputsJson(), // expected outputs stay outside Duration.ofMillis(suite.timeMs() + 1000), 8 * 1024 * 1024); if (run.killedOom()) return Verdict.mle(); if (run.timedOut()) return Verdict.tle(); if (run.exitCode() != 0) return Verdict.runtimeError(truncate(run.stderr(), 4000)); List outputs = parseJsonLines(run.stdout()); for (int i = 0; i < outputs.size(); i++) { if (!suite.matches(i, outputs.get(i))) // exact | epsilon | unordered | custom return Verdict.wrongAnswer(i, suite.expected(i), outputs.get(i)); } return Verdict.accepted(run.userTimeMs(), run.peakMemoryKb()); } finally { box.destroy(); // tear down even on an exception } } } ``` **C++** ```cpp static const std::set kFinal = {"judged", "error"}; void JudgeWorker::loop() { for (Message& msg : queue_.consume()) { // at-least-once delivery Submission sub = db_.submission(msg.submissionId()); if (kFinal.count(sub.status)) { // duplicate delivery, already done msg.ack(); continue; } Verdict v = judge(sub); db_.finalize(sub.id, v); // UPDATE ... WHERE status <> 'judged' redis_.setex("sub:" + sub.id.str(), 3600, v.toJson()); if (sub.contestId) updateLeaderboard(sub, v); // idempotent msg.ack(); } } Verdict JudgeWorker::judge(const Submission& sub) { Suite suite = suites_.load(sub.problemId); // cached on local disk by version SandboxSpec spec{ .image = kImages.at(sub.language), .network = false, .readOnlyRoot = true, .tmpfs = {{"/tmp", "32m"}}, .cpus = 1.0, .memoryMb = suite.memoryMb, .pidLimit = 64, .seccomp = "judge-profile.json", .user = "nobody", .dropAllCaps = true, }; Sandbox box = sandbox_.create(spec); // fresh box, never reused Defer cleanup([&] { box.destroy(); }); // tear down on every path Result build = box.run(kCompile.at(sub.language), /*timeout_s=*/10); if (build.exitCode != 0) // compiling is untrusted too return Verdict::compileError(truncate(build.stderr, 4000)); // All cases in one process: pay process start once, not 100 times. Result run = box.run(kHarness.at(sub.language), suite.inputsJson, // expected outputs stay outside suite.timeMs / 1000.0 + 1.0, // wall clock above the CPU budget 8 * 1024 * 1024); if (run.killedOom) return Verdict::mle(); if (run.timedOut) return Verdict::tle(); if (run.exitCode != 0) return Verdict::runtimeError(truncate(run.stderr, 4000)); auto outputs = parseJsonLines(run.stdout); for (size_t i = 0; i < outputs.size(); ++i) { if (!suite.matches(i, outputs[i])) // exact | epsilon | unordered | custom return Verdict::wrongAnswer(i, suite.expected(i), outputs[i]); } return Verdict::accepted(run.userTimeMs, run.peakMemoryKb); } ``` ### The leaderboard update This runs on Redis as one script so the whole update is atomic, and it is safe to replay because the `SADD` gates everything after it. ```lua -- KEYS: 1 solved-set, 2 user-totals hash, 3 board zset (same hash tag, one slot) -- ARGV: 1 problem_id, 2 user_id, 3 points, 4 penalty_seconds if redis.call('SADD', KEYS[1], ARGV[1]) == 0 then return 0 -- already solved: a redelivered job changes nothing end local points = redis.call('HINCRBY', KEYS[2], 'points', ARGV[3]) local penalty = redis.call('HINCRBY', KEYS[2], 'penalty', ARGV[4]) -- Pack two ranking dimensions into one score: points desc, then penalty asc. redis.call('ZADD', KEYS[3], points * 10000000 + (10000000 - penalty), ARGV[2]) return 1 ``` --- ## Trade-off Summary | Decision | Chosen | Alternative | Why | |---|---|---|---| | Submit handling | `202` plus a queued job | Synchronous request holding the connection | Seconds-long connections do not survive a 600/sec burst, and a queue gives you retries and a buffer | | Result delivery | Client polling | WebSocket or SSE | Only users with a submission in flight poll, so the load is tiny; WebSockets are machinery for a single small message | | Execution isolation | Container per submission, locked down | VM per submission, or a language sandbox | Container start fits the latency budget; language sandboxes always get escaped; full VMs boot too slowly | | Next step on isolation | Firecracker microVMs | Stay on containers | A separate kernel per submission for roughly container cost, worth it as the fleet grows | | Container reuse | Never reuse | Warm pool of reused containers | Leftover files and processes leak between users, which is both a correctness and a security bug | | Test cases | One neutral spec plus a harness per language | A suite per problem per language | 4,000 suites instead of 24,000, and adding a language is one harness | | Leaderboard | Redis sorted set with a packed score | SQL `GROUP BY` per request, or a recomputed blob | O(log N) rank for any user, and no repeated aggregation | | Leaderboard truth | Postgres submissions table | Redis | Makes rejudging and rebuilding possible; Redis stays disposable | | Autoscaling signal | Queue depth and message age | CPU utilisation | Judge CPU is always pinned, so it says nothing; queue age is the real backlog | | Queues | Separate contest and practice queues | One shared queue | Contests must not queue behind practice traffic, and the reverse | --- ## Follow-up Questions **Basic** **Q: Why return `202` instead of just waiting for the result?** Judging takes two to five seconds, and during a contest there are hundreds of submissions a second. Holding those connections open ties up the API tier, fights every proxy timeout in the path, and turns a slow worker into a user-facing error. Queuing the job means the API stays fast, the work is durable, and a crashed worker results in a redelivery rather than a lost submission. **Q: Why a container per submission rather than a VM?** A VM boots in seconds, which spends the entire latency budget before the user's code starts. A container from a warm pool starts in tens of milliseconds and, once you strip the network, mount the root read-only, apply cgroup limits and a seccomp profile, is contained well enough for a disposable judge node with nothing worth stealing on it. If you want kernel-level separation without the boot cost, that is what Firecracker microVMs are for. **Q: How do you stop an infinite loop?** Three layers. A CPU quota through cgroups, so the process cannot consume more than one core. A wall-clock timeout enforced by the worker outside the sandbox, which SIGKILLs the whole cgroup rather than one pid. And a pid limit, so a fork bomb hits a wall instead of the host. The code never gets a say in whether it stops. **Q: Why not just query the submissions table for the leaderboard?** Because 100,000 people refreshing every few seconds means a full aggregation per refresh over hundreds of thousands of rows, to answer a question whose answer barely changed. A Redis sorted set is updated once per accepted submission, and after that a page of the board is one `ZREVRANGE` and a user's own rank is one `ZREVRANK`. **Q: Why not write test cases separately for each language?** 4,000 problems times six languages is 24,000 suites to keep in sync, and every new language repeats the whole job. Writing one language-neutral spec per problem and one harness per language reduces the per-language work to a single small program, written once. ### **Senior follow-ups ↓** **Q: The same submission passed on one run and got TLE on the next. What is wrong?** Something is making execution time non-deterministic: shared cores from oversubscribed judge nodes, a noisy neighbour, or startup and JIT warmup being charged to the user. Fix it by pinning one vCPU per execution with no oversubscription, timing inside the harness around the user's function call only, and setting limits three to five times the reference solution so ordinary variance cannot flip a verdict. A time limit tight enough to fail the same code intermittently is a broken limit. **Q: A test case turns out to be wrong 40 minutes into a contest. What do you do?** Fix the suite and bump its version, which invalidates every worker's local cache. Replay the affected submissions through the queue (this works because the source code of every submission is stored). Rebuild the leaderboard into a shadow key from the corrected results, then `RENAME` it over the live board so nobody sees a half-built ranking. Announce the rejudge. The reason any of this is possible is that Postgres is the source of truth and Redis is only a cache. **Q: Your queue is at-least-once. How do you stop a double-judged submission from double-counting on the leaderboard?** Never increment. Model scoring as state: a per-user set of solved problems, and a totals hash derived from it. The leaderboard script adds the problem to the set and does nothing further if `SADD` returns 0, so a replay is a no-op. The verdict write is guarded the same way, with `UPDATE ... WHERE status <> 'judged'`. **Q: How do you stop a submission from reading the expected outputs or exfiltrating your test data?** The expected outputs never enter the sandbox. Inputs are piped in on stdin, the program's output is read back out, and the comparison happens on the worker. The sandbox has no network interface at all, so there is nowhere to send anything even if it did get hold of it. Beyond that, judge nodes live in their own account with an instance role that grants nothing useful and no route to the primary database, so escaping the sandbox gets an attacker a disposable machine and not much else. **Q: How would you support problems with more than one valid answer?** Replace the fixed expected value with a per-problem checker: a small function that takes the input and the user's output and returns valid or invalid. It runs on the worker, not in the sandbox, so it is trusted code. The spec format already carries a `compare` field, so this is just another comparator mode alongside exact, epsilon, and unordered. Keep checkers rare, since every one of them is a piece of per-problem code someone has to maintain. --- ## Minute-by-Minute Interview Playbook **0 to 5 min: Requirements** Ask the questions above. The ones that matter are Run versus Submit, the five second verdict budget, and contest scale. Confirm auth and payments are out. Write the requirements down, and put "untrusted code isolation" at the top of the non-functional list where the interviewer can see it. **5 to 10 min: Capacity estimates** Do the math out loud and land on the two numbers that matter: about 600 submissions a second at contest peak, and roughly 1,200 concurrent executions, which is about 75 judge nodes against 4 at steady state. Say the quiet part too: this platform is small, and most of it is a normal web app. That earns you credit rather than losing it. **10 to 18 min: High-level design** Draw the two paths separately. Browse is cached and boring. Submit goes API to queue to worker to sandbox, with `202` and polling. State up front that a submission is a job, not a request, and say why. **18 to 40 min: Deep dives** Lead with the sandbox. This is the part the interviewer wants, so do not wait to be asked. Name the options (language sandbox, VM, container, microVM), pick containers, and then list the actual configuration: no network, read-only root, cgroup CPU and memory and pid limits, seccomp, non-root, fresh box each time, and expected outputs kept outside the box. Then the multi-language test harness, because it proves you have thought past the boxes. Then the leaderboard with the packed sorted-set score. Then autoscaling on queue depth if there is time. **40 to 45 min: Failure and wrap-up** Cover worker death and redelivery, why the leaderboard update is idempotent, and the rejudge path. Close in one sentence: "Treat every submission as a durable job, run it in a throwaway sandbox with no network and hard resource caps, judge it against one language-neutral test spec, and keep the leaderboard in a sorted set that can always be rebuilt from Postgres." **Green flags** - Reaches for a queue and `202` without being pushed - Talks about the sandbox as a security problem and names real controls, not just "we use Docker" - Realises the expected outputs must not be readable from inside the sandbox - Notices polling load is bounded by in-flight submissions, not by online users - Designs one test spec and a harness per language instead of per-language suites - Makes the leaderboard update idempotent because the queue is at-least-once - Says plainly that this is a small system and resists over-engineering it **Red flags** - Runs user code in the API process, or relies on a language-level sandbox - Holds the HTTP connection open until judging finishes - Says "Docker" and stops, with no mention of network, CPU, memory, pids, or syscalls - Reuses containers between submissions to save start time - Recomputes the leaderboard with a `GROUP BY` on every refresh - Ignores duplicate delivery and increments scores directly - Designs for a billion users when the interviewer said a few hundred thousand --- ## Further Reading - [Firecracker microVMs](https://firecracker-microvm.github.io/): The microVM that boots in around 125ms and gives each workload its own kernel. Built for exactly this problem: running untrusted code cheaply and safely. - [gVisor](https://gvisor.dev/docs/): A userspace kernel that intercepts syscalls from the sandboxed process. The middle ground between container speed and VM isolation. - [Docker seccomp profiles](https://docs.docker.com/engine/security/seccomp/): How to restrict the syscalls a container can make. The concrete version of 'we locked down the sandbox'. - [Redis Sorted Sets](https://redis.io/docs/latest/develop/data-types/sorted-sets/): ZADD, ZREVRANGE and ZREVRANK, the three commands the entire contest leaderboard is built from. - [Related: Design a Rate Limiter](https://intervueclub.com/problems/databases/design-a-rate-limiter): The per-user submission cap and contest-entry throttling are the same distributed counter problem, solved properly. - [Related: Design Ticketmaster](https://intervueclub.com/problems/realtime/design-ticketmaster): Another system where a scheduled event creates a 20x traffic spike, and the design is mostly about absorbing it.