A rate limiter for a public API: a system design interview question

An API behind several servers needs fair per-customer limits that hold across all of them. It is a mid-level question of about 40 minutes, and it starts from a system that already works, which is how many real design interviews begin.

How do I approach the a rate limiter for a public api question?

To design a rate limiter, pick an algorithm such as token bucket, keep each customer's counters in a fast shared store so every server agrees, make the check-and-decrement atomic, return a 429 with a retry hint, and decide whether the limiter fails open or closed when its store is down.

Practise this design interview

The situation

A company sells a public API. Requests reach a load balancer, which spreads them over eight identical API servers, and each server talks to the same database. There is no limit today: one customer's runaway script recently slowed everyone down.

What you are asked to design

Design rate limiting so each customer gets a fair quota (for example 100 requests a minute, with short bursts allowed), the limit holds across all eight servers, and a rejected request tells the caller when to retry. The limiter must not become the new bottleneck or a single point of failure.

Questions worth asking first

What are we limiting by: API key, user, IP address, or endpoint?

Is it a hard limit or can the system be a little generous under failure?

Do different customer plans have different limits, and how quickly can a limit change?

What should happen if the limiter's own storage is unavailable?

Where interviewers dig

Choosing an algorithm: fixed window, sliding window, token bucket and what each does at the edges.

A shared counter store and making check-and-decrement atomic.

Where the limiter runs (gateway, middleware, sidecar) and the failure behaviour.

What a strong answer covers

Compares at least two algorithms and picks one with reasons; token bucket suits bursts, a sliding window gives smoother fairness.

Keeps the counters in a fast shared store, makes the read-modify-write atomic (a script or an atomic increment), and notes the extra network hop per request.

Returns a clear rejection (HTTP 429 with a retry-after hint) and rate-limit headers.

Decides what happens when the store fails (fail open or fail closed) and says why; considers per-server local limits as a fallback or an optimisation.

Traps that cost points

Counting per server, so the real limit is eight times the stated one.

A read-then-write on the counter, which lets concurrent requests slip past the limit.

A fixed window that allows twice the limit around the boundary, without noticing.

How the practice works here

It works on a laptop or desktop, not on a phone: the canvas and the interviewer's view of your drawing need the screen space.

You talk with the interviewer by voice, and the canvas already shows the company's system as it is today. You extend it, and the interviewer sees what you draw. A button pauses your microphone when you want to think or sketch in silence; the interviewer waits.

When the time is up you get a report: a hire or no-hire call for the level, ratings for problem navigation, solution design, technical depth and communication with a quote from your own answer for each, what to fix, and the points a strong answer covered that yours did not. Your first interview on giveinterview, of any kind, is free, and every interview report is free. After that a design interview costs credits at the same per-minute rate as a role interview, and playing duels earns credits you can spend on it.

Frequently asked questions

How long is this design interview?

About 40 minutes, the length of a typical mid-level design round.

Can I do it on my phone?

No. System design practice needs a computer screen for the canvas, so it is for laptops and desktops only.

Do I have to draw?

Yes. The canvas starts with the current system and the interviewer sees what you add, as at a whiteboard. You can pause your microphone to sketch and explain afterwards.

Is it free?

Your first interview on giveinterview, of any kind, is free, and every interview report is free. After that a design interview costs credits at the same per-minute rate as a role interview, and playing duels earns credits you can spend on it.