Concurrency Models Compared: Processes, Threads, Event Loops, Coroutines

Two different things: concurrency and parallelism

Concurrency is the ability to deal with many things at once (a structural property); parallelism is actually executing at the same instant (which needs multiple cores). A single core can still be highly concurrent, but only multiple cores give true parallelism. Before choosing, ask whether the work is CPU-bound or IO-bound.

Four models compared

Selection order

1. CPU-bound: multiple processes (or threads in a language with real parallelism) spread across cores;
2. IO-bound with many connections: an event loop or coroutines, so few threads handle many waiting connections;
3. Strong isolation needed (untrusted code, crash-prone modules): processes;
4. Lots of shared state: threads, but keep lock granularity small.

Four common traps

• Blocking calls inside an event loop: one synchronous IO stalls the whole loop and every request with it;
• Thread pool exhaustion: when a downstream slows, all threads wait and new requests queue — the failure amplifies;
• Lock contention: a coarse lock serialises your concurrency; shrink critical sections or go lock-free;
• Underestimating connections: each costs memory and a file descriptor, so set limits and timeouts.

Estimate how much concurrency you need

Use Little's law: concurrency ≈ throughput × response time. If the target is 1000 QPS at an average of 200ms, roughly 200 requests are in flight at once. Derive thread counts, pool sizes and queue limits from that number instead of guessing.

Backpressure is not optional

Every concurrency model needs backpressure: bounded queues, controlled timeouts, fast failure past capacity. Without it, rising load converts latency into memory growth and ends in a crash rather than graceful degradation.

Measure distributions, not averages

Judge concurrency by P50 / P95 / P99 and the timeout rate, not the mean. A healthy average with a terrible P99 usually means queuing, lock contention or stray slow requests dragging the tail.

Real-world scenarios: three selection calls

1. "File uploads occasionally corrupt": UDP guarantees neither order nor retransmission, so use TCP for files; if you need low latency plus reliability, add acks and retransmission at the application layer (like QUIC).
2. "Real-time voice stutters with high latency": dropping a frame beats waiting for a retransmit, so media usually uses UDP with packet-loss concealment and a jitter buffer.
3. "Memory spikes as connections grow": every connection has a cost — set limits, timeouts and backpressure so queued requests do not turn into OOM.

Common questions

Are coroutines faster than threads? They win on creation and switch cost and memory footprint; single-core throughput is not automatically higher, and real CPU work still needs multiple cores. Can an event loop use many cores? Yes — one event-loop process per core (the usual multi-worker model) with a master distributing connections. Why did adding threads make it slower? Usually lock contention, context switching, or a shared queue becoming the bottleneck.

Deadlock and livelock

• The four conditions: mutual exclusion, hold-and-wait, no preemption, circular wait — break any one, and the common fix is a global lock ordering that removes the cycle;
• Livelock: threads keep running without progress, e.g. endlessly backing off for each other; randomise the backoff instead of a fixed order;
• Starvation: low-priority work never gets resources — review scheduling and lock fairness;
• Diagnosis: dump thread stacks periodically (jstack on the JVM) and read the wait-for graph rather than guessing afterwards.