Load Balancing
A host seating restaurant guests
May 18, 2026 · 4 min read
The problem: one server, all the traffic
If every visitor to your app is handled by a single server, that server eventually runs out of room — CPU, memory, open connections — while everything else you've paid for sits idle. Past a certain point, it doesn't get slower gracefully; it stops responding at all.
Like this
Imagine a restaurant with ten tables, but the host keeps seating every new party at table one because it's closest to the door. Table one is overwhelmed. Nine tables are empty.
Like this
Imagine a restaurant with ten tables, but the host keeps seating every new party at table one because it's closest to the door. Table one is overwhelmed. Nine tables are empty.
What a load balancer actually does
A load balancer sits in front of a group of identical servers and decides, for each incoming request, which one should handle it — based on who's least busy, or simply taking turns. The client never talks to a specific server directly; it talks to the load balancer, which routes the request onward.
Like this
The host walking the floor, watching which tables have room, and seating each new party accordingly — that's the load balancer. Guests don't pick their table; the host routes them.
Like this
The host walking the floor, watching which tables have room, and seating each new party accordingly — that's the load balancer. Guests don't pick their table; the host routes them.
A few common strategies
Round robin just takes turns, server one, two, three, one again. Least connections sends the next request to whichever server currently has the fewest active requests. Fancier setups factor in server health, response time, or geography.
Like this
A good host doesn't just rotate blindly — they notice table three just got their food and freed up attention, and seat the next walk-in there instead of table one, which is still mid-order.
Like this
A good host doesn't just rotate blindly — they notice table three just got their food and freed up attention, and seat the next walk-in there instead of table one, which is still mid-order.
The catch: state
If a server remembers something about you between requests — like your logged-in session — and the next request lands on a different server that doesn't know you, things break. This is why systems either keep servers stateless (session data lives elsewhere, shared) or use “sticky sessions” to keep a user on the same server.
Like this
If your waiter from table one is the only one who remembers you asked for no onions, getting reseated at table five means explaining your order all over again.
Like this
If your waiter from table one is the only one who remembers you asked for no onions, getting reseated at table five means explaining your order all over again.
Got a concept you want explained like this?
Ask me about itRelated explainers
New explainers, straight to your inbox
One email whenever a new concept goes up. No spam, unsubscribe anytime.