API Protocols & Real-Time Communication
Deep dive comparison of REST, GraphQL, gRPC, WebSockets, and SSE.
1. Protocol Comparison (REST vs. GraphQL vs. gRPC)
| Feature | REST | GraphQL | gRPC |
|---|---|---|---|
| Data Format | JSON/XML (Text) | JSON (Text) | Protobuf (Binary) |
| Payload Size | Large (Over-fetching) | Minimal (Client defined) | Smallest (Binary) |
| Performance | Medium | Medium (Parsing overhead) | High (HTTP/2) |
| Strict Typing | Optional (OpenAPI) | Native (Schema) | Native (Protobuf) |
| Browser Support | Native | Native | Requires gRPC-Web |
| Use Case | Public APIs | Complex UI/Mobile | Microservices |
2. Real-Time: WebSockets vs. SSE (Server-Sent Events)
WebSockets
- Type: Full-duplex (Bi-directional).
- Protocol:
ws://orwss://(runs over TCP; starts as an HTTP/1.1 request, then upgrades to the WebSocket protocol). - Use Case: Chat apps, collaborative editing (Google Docs), gaming.
- Complexity: Higher (requires heartbeats, custom auth).
SSE (Server-Sent Events)
- Type: Mono-directional (Server -> Client).
- Protocol: Standard HTTP.
- Use Case: Live stock tickers, news feeds, AI text-streaming (ChatGPT).
- Simplicity: High (Automatic reconnection, native browser API).
Code Example (SSE in Python/FastAPI):
from datetime import datetime
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
import asyncio
app = FastAPI()
async def event_generator():
while True:
yield f"data: The time is {datetime.now()}\n\n"
await asyncio.sleep(1)
@app.get("/events")
async def stream():
return StreamingResponse(event_generator(), media_type="text/event-stream")
Real-Time Protocol Comparison Interview Questions
Q7: WebSocket vs Server-Sent Events (SSE) — when do you choose each?
WebSocket: full-duplex (client and server push), lower latency, binary + text. SSE: one-way server→client over regular HTTP, auto-reconnect, built-in event fields, works through proxies. Choose SSE when updates flow only from server (live scores, feed ticks); choose WebSocket for chat, collaborative editing, or client→server real-time needs.
Q8: When should you use gRPC over REST?
gRPC: typed binary (protobuf), bidirectional streaming, multiplexing over HTTP/2, codegen clients. Best for internal service-to-service RPC, streaming pipelines, and polyglot backend systems. REST: human-debuggable, cacheable, universal clients/browsers, simple discovery. Use gRPC internally, REST/gRPC-web at the edge unless grpc-gateway is justified.
Q9: REST vs GraphQL — how do you decide?
REST: resource-oriented endpoints, simple caching, easy versioning, predictable. GraphQL: single endpoint, client-driven selection, avoids over/under-fetching, strong typing + introspection. Choose GraphQL for mobile apps with multiple views and complex nested data; choose REST for simple CRUD, public APIs, and cache-heavy read paths (cache at CDN).
Q10: What are the trade-offs of Message Queues vs direct HTTP calls?
Queues: async, buffered, decoupled, retry/backoff, no caller blocking — but latency, at-least-once duplicates, harder debugging, eventual consistency. HTTP: synchronous, immediate feedback, simple — but couples producer/consumer and fails under bursts. Use queues for background work, fan-out, and resilience; HTTP for request/response semantics.
Q11: What is the difference between WebSocket and raw TCP socket?
WebSocket is an application-level protocol over TCP with an HTTP upgrade handshake, framing, ping/pong, and browser support. Raw TCP socket is transport-level with no framing or protocol — you implement your own message boundary, reconnection, and security. Choose WebSocket for web clients; raw TCP only for custom low-level protocols.
Q12: What is long polling and when is it appropriate?
Long polling: client makes a request, server holds it open until data is available or timeout, then responds; client immediately reconnects. Simulates push over request/response. Simpler than WebSocket, works everywhere, but adds per-connection server overhead and latency. Appropriate for low-frequency server events when streaming isn't available.
Q13: How do you choose between MQTT and WebSocket for IoT/real-time?
MQTT: lightweight pub/sub over TCP with QoS levels, designed for constrained devices and unstable networks; broker-based. WebSocket: full-duplex, better for rich browser apps with complex messages. Choose MQTT for sensors/devices and many-to-many pub/sub; WebSocket for interactive web UIs.
Q14: What is Server-Sent Events vs Polling in terms of cost?
Polling: repeated full HTTP requests, wasted bandwidth, latency = poll interval. SSE: one persistent connection, server pushes immediately, near-zero idle cost. Choose SSE when events are frequent or latency matters; polling only for rare events or when servers can't stream.
Q15: What is the difference between push and pull message consumption?
Push (RabbitMQ basic.consume): broker sends messages to consumers, limited by consumer prefetch — lower latency, but consumers must handle backpressure. Pull (Kafka): consumer fetches batches when ready — better throughput control, lag-based monitoring, replayable offsets. Kafka and SQS are pull-based (SQS consumers poll with ReceiveMessage, ideally with long polling); RabbitMQ pushes.
Protocol Trade-offs in Depth
Q16: What are the four gRPC call types, and when would you use each?
Technical Answer:
| Type | Shape | Typical use |
|---|---|---|
| Unary | one request → one response | Ordinary RPC: GetUser, CreateOrder |
| Server streaming | one request → stream of responses | Large result sets sent in pages, progress updates, watching a resource |
| Client streaming | stream of requests → one response | Uploading chunks or batches of metrics, then one summary |
| Bidirectional streaming | stream ↔ stream, independent | Chat, live collaboration, interactive sessions between services |
All four run as a single HTTP/2 stream per call, so many calls share one connection.
syntax = "proto3";
package telemetry.v1;
service Telemetry {
rpc GetDevice(GetDeviceRequest) returns (Device); // unary
rpc WatchDevice(GetDeviceRequest) returns (stream DeviceEvent); // server streaming
rpc UploadReadings(stream Reading) returns (UploadSummary); // client streaming
rpc Control(stream ControlCommand) returns (stream ControlAck); // bidirectional
}
message GetDeviceRequest { string device_id = 1; }
message Device { string device_id = 1; string name = 2; }
message DeviceEvent { string device_id = 1; string state = 2; int64 ts_ms = 3; }
message Reading { string device_id = 1; double value = 2; int64 ts_ms = 3; }
message UploadSummary { int64 accepted = 1; int64 rejected = 2; }
message ControlCommand { string device_id = 1; string command = 2; }
message ControlAck { string device_id = 1; bool ok = 2; }
Long-lived streams have operational costs: they pin a client to one backend (load-balancing impact), need deadlines and keepalives, and must be resumable when a connection drops.
🔍 Interviewer Notes: They want the candidate to match each type to a real use case and to know streaming calls still need deadlines, flow control and reconnect logic.
❌ Common Mistakes: Using bidirectional streaming where unary calls would do, which complicates load balancing and retries. Running streams without deadlines.
Q17: What is the N+1 problem in GraphQL, and how does DataLoader solve it?
Technical Answer:
GraphQL resolves fields one at a time. For a query like posts { author { name } }, the posts resolver runs one query, then the author resolver runs once per post — 1 + N database queries. DataLoader collects every key requested within one tick of the event loop and calls a single batch function (SELECT ... WHERE id IN (...)), with a per-request cache so the same key is only loaded once.
A minimal, runnable asyncio version of the pattern:
import asyncio
from collections.abc import Awaitable, Callable, Hashable, Sequence
class DataLoader:
"""Batches load(key) calls made in the same event-loop tick into one batch call."""
def __init__(self, batch_fn: Callable[[Sequence[Hashable]], Awaitable[Sequence[object]]]):
self._batch_fn = batch_fn
self._cache: dict[Hashable, asyncio.Future] = {}
self._queue: list[Hashable] = []
def load(self, key: Hashable) -> asyncio.Future:
if key in self._cache:
return self._cache[key]
fut = asyncio.get_running_loop().create_future()
self._cache[key] = fut
self._queue.append(key)
if len(self._queue) == 1: # first key in this tick: schedule a flush
asyncio.get_running_loop().call_soon(lambda: asyncio.ensure_future(self._dispatch()))
return fut
async def _dispatch(self) -> None:
keys, self._queue = self._queue, []
try:
values = await self._batch_fn(keys)
for key, value in zip(keys, values, strict=True):
self._cache[key].set_result(value)
except Exception as exc:
for key in keys:
self._cache.pop(key).set_exception(exc)
AUTHORS = {1: "Ada", 2: "Linus", 3: "Grace"}
queries_run = 0
async def batch_get_authors(ids: Sequence[int]) -> list[str | None]:
global queries_run
queries_run += 1 # stands in for: SELECT id, name FROM authors WHERE id IN (...)
return [AUTHORS.get(i) for i in ids] # results must be in the same order as ids
async def main() -> None:
loader = DataLoader(batch_get_authors) # create one loader per request
posts = [{"id": p, "author_id": a} for p, a in [(10, 1), (11, 2), (12, 1), (13, 3)]]
names = await asyncio.gather(*(loader.load(p["author_id"]) for p in posts))
print(names, "queries:", queries_run) # ['Ada', 'Linus', 'Ada', 'Grace'] queries: 1
if __name__ == "__main__":
asyncio.run(main())
Two rules: create loaders per request (so one user's cache never serves another user), and the batch function must return results in the same order as the keys, with a placeholder for missing ones.
🔍 Interviewer Notes: Expect the candidate to explain why GraphQL causes N+1 (independent field resolvers), the batching-per-tick idea, and the per-request scoping rule.
❌ Common Mistakes: A global DataLoader shared across requests (data leaks across users and stale cache). Returning batch results in database order instead of key order.
Q18: How does caching differ between REST, GraphQL and gRPC?
Technical Answer:
- REST gets HTTP caching for free: each resource has its own URL,
GETresponses are cacheable by browsers, CDNs and proxies, controlled withCache-Control,ETag/If-None-MatchandLast-Modified. Invalidation is per URL. - GraphQL typically sends
POST /graphqlwith the query in the body. Intermediaries do not cache POST, and every query shape is different, so HTTP caching mostly disappears. Options: persisted queries sent asGETwith a query hash (CDN-cacheable), client-side normalised caches keyed by type and ID (Apollo, Relay), and server-side caching per resolver or per data source. - gRPC uses HTTP/2
POSTfor every call, so standard HTTP caches do not apply. Caching is done in the application (a cache in front of the data source or a client-side cache keyed on the request message).
import hashlib
import json
def etag_for(payload: dict) -> str:
"""Strong ETag from a canonical JSON body (REST conditional GET)."""
body = json.dumps(payload, sort_keys=True, separators=(",", ":")).encode()
return '"' + hashlib.sha256(body).hexdigest()[:32] + '"'
def conditional_get(payload: dict, if_none_match: str | None) -> tuple[int, dict, bytes]:
tag = etag_for(payload)
headers = {"ETag": tag, "Cache-Control": "private, max-age=0, must-revalidate"}
if if_none_match == tag:
return 304, headers, b"" # client copy is still valid
return 200, headers, json.dumps(payload).encode()
if __name__ == "__main__":
user = {"id": 7, "name": "Ada"}
status, headers, _ = conditional_get(user, None)
print(status, headers["ETag"])
print(conditional_get(user, headers["ETag"])[0]) # 304
🔍 Interviewer Notes: They are testing whether you know GraphQL and gRPC give up HTTP-level caching and what the workarounds are. Persisted queries are the expected senior answer for GraphQL at the edge.
❌ Common Mistakes: Claiming GraphQL "can't be cached." Forgetting Vary or private when responses depend on the user, which lets a shared cache serve one user's data to another.
Q19: How do error models differ across REST, GraphQL and gRPC?
Technical Answer:
- REST: the HTTP status code is the primary signal (400, 401, 403, 404, 409, 422, 429, 5xx), with a structured body. RFC 9457 "Problem Details" (
application/problem+json) is a standard body format. - GraphQL: a response can be partially successful. The server typically returns HTTP 200 with both
data(fields that resolved) and anerrorsarray (each withmessage,pathand optionalextensionssuch as a code). Clients must checkerrorseven on 200. - gRPC: each call ends with a status code from a fixed set (
OK,INVALID_ARGUMENT,NOT_FOUND,ALREADY_EXISTS,PERMISSION_DENIED,UNAUTHENTICATED,RESOURCE_EXHAUSTED,FAILED_PRECONDITION,UNAVAILABLE,DEADLINE_EXCEEDED,INTERNAL, and others) plus a message, sent in HTTP/2 trailers.
When a gateway translates between them, use one mapping table and keep it consistent:
from http import HTTPStatus
# Common gRPC -> HTTP mapping used by gateways (for example grpc-gateway)
GRPC_TO_HTTP = {
"OK": HTTPStatus.OK,
"INVALID_ARGUMENT": HTTPStatus.BAD_REQUEST,
"FAILED_PRECONDITION": HTTPStatus.BAD_REQUEST,
"OUT_OF_RANGE": HTTPStatus.BAD_REQUEST,
"UNAUTHENTICATED": HTTPStatus.UNAUTHORIZED,
"PERMISSION_DENIED": HTTPStatus.FORBIDDEN,
"NOT_FOUND": HTTPStatus.NOT_FOUND,
"ALREADY_EXISTS": HTTPStatus.CONFLICT,
"ABORTED": HTTPStatus.CONFLICT,
"RESOURCE_EXHAUSTED": HTTPStatus.TOO_MANY_REQUESTS,
"CANCELLED": 499, # client closed request (non-standard)
"UNIMPLEMENTED": HTTPStatus.NOT_IMPLEMENTED,
"UNAVAILABLE": HTTPStatus.SERVICE_UNAVAILABLE,
"DEADLINE_EXCEEDED": HTTPStatus.GATEWAY_TIMEOUT,
"INTERNAL": HTTPStatus.INTERNAL_SERVER_ERROR,
"UNKNOWN": HTTPStatus.INTERNAL_SERVER_ERROR,
"DATA_LOSS": HTTPStatus.INTERNAL_SERVER_ERROR,
}
RETRYABLE_GRPC = {"UNAVAILABLE", "RESOURCE_EXHAUSTED", "ABORTED"} # retry with backoff
def graphql_error(message: str, path: list[str | int], code: str) -> dict:
return {"message": message, "path": path, "extensions": {"code": code}}
if __name__ == "__main__":
print(int(GRPC_TO_HTTP["DEADLINE_EXCEEDED"])) # 504
print({"data": {"user": {"name": "Ada", "orders": None}},
"errors": [graphql_error("orders service unavailable", ["user", "orders"], "UNAVAILABLE")]})
🔍 Interviewer Notes: The GraphQL partial-success model is the key point many candidates miss. For gRPC, knowing which codes are safe to retry shows production experience.
❌ Common Mistakes: Returning 200 with {"error": ...} in REST. Treating any GraphQL 200 as success. Mapping every gRPC failure to UNKNOWN or INTERNAL, which makes retry logic impossible.
Q20: How do you evolve schemas safely in REST, GraphQL and gRPC?
Technical Answer: The shared rule: additive changes are safe; removing, renaming or changing the meaning of a field is breaking.
- REST/JSON: add optional fields; clients must ignore unknown fields. For breaking changes, use a new version (
/v2/...or a media-type version) and run both in parallel during migration. - GraphQL: evolve one schema without versions. Add fields, mark old ones
@deprecated(reason: "..."), track field usage from query logs, and remove only when usage reaches zero. - gRPC/Protobuf: the wire format uses field numbers, not names. Never reuse or change a field number or its type; when deleting, mark the number and name
reservedso nobody reuses them by accident. New fields are invisible to old clients and default to zero values for old servers.
syntax = "proto3";
package shop.v1;
message Order {
reserved 3; // was: string coupon = 3; removed in favour of discounts
reserved "coupon";
string id = 1;
int64 total_cents = 2;
repeated Discount discounts = 4; // additive: old clients ignore it
string currency = 5; // additive: old senders leave it empty
}
message Discount {
string code = 1;
int64 amount_cents = 2;
}
type Order {
id: ID!
totalCents: Int!
coupon: String @deprecated(reason: "Use discounts")
discounts: [Discount!]!
}
🔍 Interviewer Notes: Expect "field numbers are the contract" for protobuf and "deprecate and measure usage" for GraphQL. Senior candidates mention consumer-driven contract tests or schema-registry checks in CI to block breaking changes.
❌ Common Mistakes: Renumbering protobuf fields. Changing a field's type in place. Removing a GraphQL field without checking whether any client still queries it.
WebSockets, SSE & Long Polling Internals
Q21: How does the WebSocket opening handshake work?
Technical Answer:
A WebSocket starts as an ordinary HTTP/1.1 GET with upgrade headers. If the server agrees, it answers 101 Switching Protocols, and from then on the same TCP connection carries WebSocket frames instead of HTTP.
GET /chat HTTP/1.1
Host: chat.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13
Origin: https://app.example.com
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
Sec-WebSocket-Accept proves the server actually understood the WebSocket request (it is not a security mechanism): it is the Base64 of the SHA-1 of the client key concatenated with a fixed GUID from RFC 6455.
import base64
import hashlib
WS_GUID = "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" # fixed value from RFC 6455
def websocket_accept(sec_websocket_key: str) -> str:
digest = hashlib.sha1((sec_websocket_key + WS_GUID).encode("ascii")).digest()
return base64.b64encode(digest).decode("ascii")
if __name__ == "__main__":
# Example key from RFC 6455
print(websocket_accept("dGhlIHNhbXBsZSBub25jZQ==")) # s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
After the upgrade, data travels as frames (text, binary, ping, pong, close). Frames from client to server must be masked, which stops browser-originated traffic from being mistaken for other protocols by caching proxies.
Because the handshake is HTTP, proxies and load balancers must forward the Upgrade and Connection headers explicitly (NGINX does not by default).
🔍 Interviewer Notes: They want "HTTP upgrade → 101 → same TCP connection." Knowing Origin must be checked by the server (browsers do not enforce same-origin on WebSockets) is a strong security signal.
❌ Common Mistakes: Saying WebSocket "upgrades from HTTP to TCP" (it was always TCP; it upgrades from HTTP to the WebSocket protocol). Treating Sec-WebSocket-Key as authentication.
Q22: How does the SSE wire format work, and how does a client resume after a disconnect?
Technical Answer:
SSE is a text/event-stream HTTP response that never ends. Each event is a block of field: value lines followed by a blank line:
data:— the payload (multipledata:lines are joined with newlines).event:— an event type; the browser dispatches it toaddEventListener(type, ...).id:— the event ID. The browser remembers the last one.retry:— reconnection delay in milliseconds.- Lines starting with
:are comments, commonly used as keep-alive pings.
When the connection drops, the browser's EventSource reconnects automatically and sends the Last-Event-ID header, so the server can replay what was missed.
import asyncio
import json
from fastapi import FastAPI, Header
from fastapi.responses import StreamingResponse
app = FastAPI()
EVENTS: list[dict] = [] # stands in for a durable, ordered event log
def format_sse(event_id: int, event_type: str, data: dict) -> str:
return f"id: {event_id}\nevent: {event_type}\ndata: {json.dumps(data)}\n\n"
async def event_stream(last_id: int):
yield "retry: 3000\n\n" # tell the browser to wait 3 s before reconnecting
cursor = last_id
while True:
new = [e for e in EVENTS if e["id"] > cursor]
for e in new:
yield format_sse(e["id"], e["type"], e["data"])
cursor = e["id"]
if not new:
yield ": keep-alive\n\n" # comment line keeps proxies from timing out
await asyncio.sleep(1)
@app.get("/events")
async def events(last_event_id: int = Header(default=0, alias="Last-Event-ID")):
return StreamingResponse(
event_stream(last_event_id),
media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"}, # disable NGINX buffering
)
@app.post("/publish")
async def publish(payload: dict):
event = {"id": len(EVENTS) + 1, "type": "order.updated", "data": payload}
EVENTS.append(event)
return {"id": event["id"]}
Limitations: SSE is text only and one-directional. Under HTTP/1.1, browsers limit open connections per origin (six in common browsers), which several SSE tabs can exhaust; HTTP/2 multiplexes streams over one connection and removes that problem.
🔍 Interviewer Notes: Last-Event-ID resume is the feature that makes SSE production-grade; candidates should mention it. Knowing about proxy buffering and the HTTP/1.1 per-origin connection limit shows hands-on experience.
❌ Common Mistakes: Forgetting the blank line that terminates each event. Sending events without IDs, so reconnects silently lose data. Leaving response buffering on in a reverse proxy, so events arrive in bursts.
Q23: How do you implement long polling correctly on the server?
Technical Answer: The client asks "anything newer than cursor X?" The server answers immediately if there is, otherwise it holds the request until new data arrives or a timeout passes, then returns an empty response. The client immediately asks again with the latest cursor.
Correctness details:
- Always use a cursor (last-seen ID or version) so nothing is lost between polls.
- Return before any proxy or load balancer idle timeout (25-30 s is common).
- Do not block a thread per waiting request; use async waiting.
- Clients add jitter and back off on errors.
import asyncio
from fastapi import FastAPI, Response
app = FastAPI()
messages: list[dict] = []
new_message = asyncio.Condition()
@app.get("/poll")
async def poll(after: int = 0, timeout: float = 25.0):
def pending() -> list[dict]:
return [m for m in messages if m["id"] > after]
async with new_message:
try:
await asyncio.wait_for(new_message.wait_for(lambda: bool(pending())), timeout)
except TimeoutError:
return Response(status_code=204) # nothing new; client polls again
return {"messages": pending(), "cursor": messages[-1]["id"]}
@app.post("/send")
async def send(body: dict):
async with new_message:
messages.append({"id": len(messages) + 1, **body})
new_message.notify_all() # wake every waiting poller
return {"ok": True}
client server
GET /poll?after=41 ----------> no data -> wait (up to 25 s)
POST /send arrives -> notify
<---------- 200 [42]
GET /poll?after=42 ----------> no data -> wait ... timeout
<---------- 204
GET /poll?after=42 ----------> ...
🔍 Interviewer Notes: They want the cursor, the timeout below proxy limits, and non-blocking waits. Candidates who mention that each waiting client still holds a connection (so memory and file descriptors matter) understand the cost.
❌ Common Mistakes: No cursor (messages arriving between two polls are lost). Holding requests for minutes, so load balancers cut them with 504s. Thread-per-request servers that run out of threads.
Q24: Why do WebSocket connections need heartbeats, and how do you set them up?
Technical Answer: Two problems that TCP alone does not solve quickly:
- Half-open connections: if a phone loses signal or a laptop sleeps, no FIN or RST is sent. The server thinks the connection is alive and keeps buffering messages for it. TCP only notices when it tries to send and retransmissions time out, which can take many minutes.
- Idle timeouts in middleboxes: load balancers, proxies and NAT devices drop connections that are silent for longer than their idle timeout.
WebSocket has control frames for this: one side sends ping, the other must reply pong. Send pings more often than the shortest idle timeout on the path, and close the connection if a pong does not come back in time.
import asyncio
import websockets
async def handler(ws) -> None:
async for message in ws:
await ws.send(f"echo: {message}")
async def main() -> None:
async with websockets.serve(
handler,
"0.0.0.0",
8765,
ping_interval=20, # send a ping every 20 s (below a 60 s LB idle timeout)
ping_timeout=20, # no pong within 20 s -> connection considered dead and closed
):
await asyncio.Future() # run forever
if __name__ == "__main__":
asyncio.run(main())
Browsers' WebSocket API does not expose ping/pong to JavaScript (the browser answers server pings automatically). If the client needs to detect a dead server, send an application-level heartbeat message and expect a reply.
🔍 Interviewer Notes: Look for half-open connection detection and middlebox idle timeouts as the two reasons, and the rule "ping interval < smallest idle timeout." Knowing browsers cannot send pings from JavaScript is a good detail.
❌ Common Mistakes: Relying on TCP keepalive (default idle time is hours). Pinging every second from millions of clients (needless load and mobile battery drain). Never cleaning up server state for dead connections.
Q25: How should a real-time client reconnect after a disconnect?
Technical Answer: Reconnect with exponential backoff plus jitter, and resume from where you left off:
- Delay grows (1 s, 2 s, 4 s … up to a cap), so a struggling server is not hammered.
- Jitter (randomness) spreads clients out; without it, every client disconnected by the same event reconnects at the same instant (thundering herd).
- Reset the backoff after a connection has been healthy for a while, not merely after connecting.
- On reconnect, send the last processed sequence number or event ID so the server can replay missed messages.
- Distinguish permanent failures (authentication rejected,
4xxxapplication close codes you define as fatal) from transient ones.
import asyncio
import random
import websockets
URL = "wss://realtime.example.com/stream"
def full_jitter(attempt: int, base: float = 1.0, cap: float = 60.0) -> float:
"""'Full jitter': random delay between 0 and the exponential backoff value."""
return random.uniform(0, min(cap, base * 2 ** attempt))
async def consume(handle) -> None:
attempt = 0
last_seq = 0
while True:
try:
async with websockets.connect(f"{URL}?after={last_seq}") as ws:
connected_at = asyncio.get_running_loop().time()
async for raw in ws:
last_seq = handle(raw, last_seq) # returns the new last processed seq
if asyncio.get_running_loop().time() - connected_at > 30:
attempt = 0 # healthy for 30 s: reset backoff
except websockets.InvalidStatus as exc:
if exc.response.status_code in (401, 403):
raise # permanent: re-authenticate instead
except (OSError, websockets.InvalidHandshake, websockets.ConnectionClosed):
pass # transient: fall through to backoff
delay = full_jitter(attempt)
attempt += 1
await asyncio.sleep(delay)
🔍 Interviewer Notes: Jitter is the must-have word. Senior candidates also mention resume-from-cursor and server-side protection (connection-rate limits) because client behaviour cannot be fully trusted.
❌ Common Mistakes: Immediate reconnect in a tight loop. Exponential backoff without jitter. Reconnecting forever on authentication errors. Losing messages because the client does not send a resume cursor.
Q26: How do you authenticate WebSocket connections from a browser?
Technical Answer:
The browser WebSocket constructor cannot set custom headers such as Authorization. Common approaches:
- Cookies: the handshake is an HTTP request, so same-site cookies are sent automatically. You must check the
Originheader, or any website can open a socket using the victim's cookies (Cross-Site WebSocket Hijacking). - Short-lived ticket: the client calls an authenticated REST endpoint to get a one-time ticket valid for a few seconds, then connects with
?ticket=.... Query strings can end up in logs, which is acceptable only because the ticket is single-use and expires quickly. - Authenticate in the first message: accept the socket, require an auth message within a short deadline, close otherwise.
Also re-check authorization over the life of the connection: tokens expire and permissions change while a socket stays open for hours.
import hashlib
import hmac
import secrets
import time
SECRET = secrets.token_bytes(32)
ALLOWED_ORIGINS = {"https://app.example.com"}
_used_nonces: set[str] = set() # use a shared store with expiry (for example Redis) in production
def issue_ticket(user_id: str, ttl: int = 30) -> str:
expires = int(time.time()) + ttl
nonce = secrets.token_urlsafe(12)
msg = f"{user_id}.{expires}.{nonce}"
sig = hmac.new(SECRET, msg.encode(), hashlib.sha256).hexdigest()
return f"{msg}.{sig}"
def verify_handshake(ticket: str, origin: str | None) -> str:
if origin not in ALLOWED_ORIGINS:
raise PermissionError("bad origin") # blocks cross-site hijacking
user_id, expires, nonce, sig = ticket.rsplit(".", 3)
expected = hmac.new(SECRET, f"{user_id}.{expires}.{nonce}".encode(), hashlib.sha256).hexdigest()
if not hmac.compare_digest(sig, expected):
raise PermissionError("bad signature")
if int(expires) < time.time():
raise PermissionError("ticket expired")
if nonce in _used_nonces:
raise PermissionError("ticket already used")
_used_nonces.add(nonce)
return user_id
if __name__ == "__main__":
t = issue_ticket("user-42")
print(verify_handshake(t, "https://app.example.com")) # user-42
try:
verify_handshake(t, "https://app.example.com")
except PermissionError as e:
print("second use:", e) # ticket already used
🔍 Interviewer Notes: The Origin check and the "no custom headers from browsers" constraint are what they are listening for. Re-validating long-lived sessions is the senior-level point.
❌ Common Mistakes: Putting a long-lived JWT in the query string. Cookie auth without an Origin check. Authenticating once at connect and never again for a connection that lives for days.
Scaling Real-Time Systems
Q27: How do you scale WebSocket servers horizontally?
Technical Answer: A WebSocket connection is pinned to whichever server accepted it. Once you have more than one server, a message for user B may be produced on server 1 while B is connected to server 3. You need:
- A fan-out layer (pub/sub): every server subscribes to the channels its connected users care about; producers publish to a channel; each server forwards to its local sockets. Redis Pub/Sub, NATS, Kafka (with a consumer per server) or a managed service are typical.
- Load balancing of new connections: least-connections works better than round robin because connections are long-lived. Sticky sessions are not needed for a pure WebSocket (the TCP connection already stays on one server), but they are needed for libraries that fall back to multi-request transports such as long polling (for example Socket.IO), where several HTTP requests must reach the same server.
- Connection-aware deploys: draining a server disconnects all its clients, so roll gradually and rely on client backoff with jitter.
- Per-server limits: file descriptors, memory per connection, and kernel buffer sizes, not request throughput, are usually the first bottleneck.
import asyncio
import json
from collections import defaultdict
import redis.asyncio as redis
import websockets
r = redis.Redis()
local: dict[str, set] = defaultdict(set) # room -> sockets connected to THIS server
async def redis_listener() -> None:
pubsub = r.pubsub()
await pubsub.psubscribe("room:*")
async for msg in pubsub.listen():
if msg["type"] != "pmessage":
continue
room = msg["channel"].decode().removeprefix("room:")
for ws in list(local.get(room, ())):
try:
await ws.send(msg["data"].decode())
except websockets.ConnectionClosed:
local[room].discard(ws)
async def handler(ws) -> None:
room = ws.request.path.strip("/") or "lobby"
local[room].add(ws)
try:
async for text in ws: # client message -> publish to every server
await r.publish(f"room:{room}", json.dumps({"room": room, "text": text}))
finally:
local[room].discard(ws)
async def main() -> None:
async with websockets.serve(handler, "0.0.0.0", 8765):
await redis_listener()
if __name__ == "__main__":
asyncio.run(main())
client A --ws--> server 1 --PUBLISH room:42--> Redis --> server 1 --> local sockets in room 42
\-> server 2 --> local sockets in room 42
\> server 3 --> local sockets in room 42
🔍 Interviewer Notes: They want the cross-server delivery problem stated clearly and a pub/sub backplane as the answer. A correct take on when sticky sessions matter (fallback transports) separates experienced candidates from those repeating folklore.
❌ Common Mistakes: "Use sticky sessions" as the whole answer. Storing connection state only in memory with no way to route messages across servers. Using round-robin balancing and ending up with uneven connection counts after deploys.
Q28: What is backpressure in a real-time stream, and how do you handle slow consumers?
Technical Answer: Backpressure is what happens when a producer is faster than a consumer. In a fan-out server, one client on a bad mobile connection cannot read as fast as you publish. If you keep queueing messages for it, its buffer grows without limit until the server runs out of memory and every client suffers.
Bound every per-connection buffer and choose a policy for when it fills:
| Policy | When to use |
|---|---|
| Drop oldest / keep latest (conflate) | State snapshots: prices, dashboards, positions — only the latest value matters |
| Drop newest | Best-effort notifications where older items are more important |
| Disconnect the slow client | Ordered streams where gaps are not allowed; client reconnects and resyncs from a cursor |
| Block the producer | Only for 1:1 pipelines; never let one consumer block a shared fan-out |
import asyncio
class ClientQueue:
"""Bounded per-connection queue. Drops the oldest item when full and counts drops."""
def __init__(self, maxsize: int = 100):
self.q: asyncio.Queue[str] = asyncio.Queue(maxsize=maxsize)
self.dropped = 0
def offer(self, msg: str) -> None:
if self.q.full():
self.q.get_nowait() # discard the oldest so the newest gets through
self.dropped += 1
self.q.put_nowait(msg)
async def demo() -> None:
cq = ClientQueue(maxsize=3)
for i in range(10): # producer is much faster than the consumer
cq.offer(f"tick {i}")
items = [cq.q.get_nowait() for _ in range(cq.q.qsize())]
print(items, "dropped:", cq.dropped) # ['tick 7', 'tick 8', 'tick 9'] dropped: 7
if __name__ == "__main__":
asyncio.run(demo())
A separate writer task per connection drains its queue with await ws.send(...); the fan-out loop only calls offer(), which never blocks.
🔍 Interviewer Notes: The key insight is isolating slow consumers so they cannot hurt fast ones, and choosing the drop policy based on message semantics. Senior candidates expose a metric for dropped messages and disconnections.
❌ Common Mistakes: Unbounded queues. Awaiting send() for each client inside the fan-out loop, so one slow client delays everyone. Dropping messages in a stream where the client assumes completeness, with no gap detection.
Q29: What delivery and ordering guarantees can a real-time system realistically provide?
Technical Answer:
- At-most-once: send and forget; messages can be lost (plain WebSocket push or Redis Pub/Sub with no persistence).
- At-least-once: store, deliver, retry until acknowledged; duplicates are possible.
- Exactly-once delivery over an unreliable network is not achievable end to end. What systems provide is effectively-once processing: at-least-once delivery plus idempotent handling (deduplicate by message ID).
Ordering is normally only guaranteed per key (per conversation, per document, per user), using a per-key sequence number assigned by one writer. Global ordering requires a single sequencer and does not scale.
The client uses the sequence number to deduplicate, detect gaps and request a resync:
class SequenceTracker:
"""Per-stream client-side tracker: drop duplicates, detect gaps."""
def __init__(self) -> None:
self.last_seq: dict[str, int] = {}
def accept(self, stream: str, seq: int) -> str:
last = self.last_seq.get(stream, 0)
if seq <= last:
return "duplicate" # already processed: ignore
if seq > last + 1:
return f"gap {last + 1}..{seq - 1}" # fetch missing range before applying
self.last_seq[stream] = seq
return "apply"
if __name__ == "__main__":
t = SequenceTracker()
for s in [1, 2, 2, 4]:
print(s, t.accept("conv-9", s))
# 1 apply / 2 apply / 2 duplicate / 4 gap 3..3
🔍 Interviewer Notes: They want the candidate to reject "exactly-once delivery" and explain at-least-once plus idempotency, and to scope ordering to a key. Sequence-number gap detection on the client is a strong practical answer.
❌ Common Mistakes: Claiming WebSocket or TCP gives end-to-end delivery guarantees (they cover one connection, not reconnects, crashes or restarts). Promising global ordering. Using timestamps from many servers to order messages.
Q30: How would you build a presence system ("who is online")?
Technical Answer: Presence is soft state: it must be fast, it can be slightly stale, and it must clean itself up when clients vanish without saying goodbye.
Design:
- Each connected client sends a heartbeat (every ~30 s). The server records
user → last_seenin a shared store, such as a Redis sorted set scored by timestamp. - "Online" means
last_seenwithin a window (for example 2 × heartbeat interval + slack). Expiry is implicit, so crashed servers and dropped phones go offline automatically. - A user with several devices is online if any device is online, so track per connection or per device.
- Publish presence changes (not every heartbeat) to interested parties, and debounce offline transitions so brief network blips do not cause online/offline flapping.
- For large audiences, avoid broadcasting presence to everyone; send it only to people who are looking (open conversation, visible contact list).
import asyncio
import time
import redis.asyncio as redis
r = redis.Redis()
KEY = "presence:last_seen"
ONLINE_WINDOW = 75 # seconds: 2 x 30 s heartbeat + slack
async def heartbeat(user_id: str, device_id: str) -> None:
await r.zadd(KEY, {f"{user_id}:{device_id}": time.time()})
async def online_users(user_ids: list[str]) -> set[str]:
cutoff = time.time() - ONLINE_WINDOW
members = await r.zrangebyscore(KEY, cutoff, "+inf")
online = {m.decode().split(":", 1)[0] for m in members}
return online & set(user_ids)
async def reap_stale() -> int:
"""Run periodically to keep the set small."""
return await r.zremrangebyscore(KEY, "-inf", time.time() - ONLINE_WINDOW)
async def main() -> None:
await heartbeat("alice", "phone")
await heartbeat("bob", "laptop")
print(await online_users(["alice", "bob", "carol"])) # {'alice', 'bob'}
if __name__ == "__main__":
asyncio.run(main())
🔍 Interviewer Notes: Expect heartbeats with TTL-based expiry (not relying on disconnect events), multi-device handling and debouncing. At scale, they want you to limit presence fan-out to interested viewers.
❌ Common Mistakes: Marking users offline only on socket close (misses crashes and network loss). Broadcasting every heartbeat. Treating presence as strongly consistent data.
Webhooks, WebTransport & Protocol Selection
Q31: How do you sign and verify webhooks securely?
Technical Answer: A webhook endpoint is a public URL, so the receiver must prove that a request really came from the sender and was not modified or replayed.
Standard scheme (used in similar forms by Stripe, GitHub, Slack and the Standard Webhooks spec):
- Sender and receiver share a per-endpoint secret.
- The sender computes
HMAC-SHA256(secret, timestamp + "." + raw_body)and sends the timestamp and signature in headers. - The receiver recomputes the HMAC over the raw bytes of the body (before any JSON parsing), compares in constant time, and rejects timestamps outside a tolerance window (for example 5 minutes) to block replays.
- Support multiple active secrets so secrets can be rotated without downtime.
import hashlib
import hmac
import time
TOLERANCE_SECONDS = 300
def sign(secret: bytes, body: bytes, ts: int | None = None) -> dict[str, str]:
ts = ts or int(time.time())
sig = hmac.new(secret, f"{ts}.".encode() + body, hashlib.sha256).hexdigest()
return {"Webhook-Timestamp": str(ts), "Webhook-Signature": f"v1={sig}"}
def verify(secrets: list[bytes], body: bytes, headers: dict[str, str]) -> None:
ts = int(headers["Webhook-Timestamp"])
if abs(time.time() - ts) > TOLERANCE_SECONDS:
raise PermissionError("timestamp outside tolerance (possible replay)")
received = headers["Webhook-Signature"].removeprefix("v1=")
for secret in secrets: # current + previous during rotation
expected = hmac.new(secret, f"{ts}.".encode() + body, hashlib.sha256).hexdigest()
if hmac.compare_digest(expected, received):
return
raise PermissionError("signature mismatch")
if __name__ == "__main__":
body = b'{"id":"evt_1","type":"invoice.paid"}'
hdrs = sign(b"whsec_new", body)
verify([b"whsec_new", b"whsec_old"], body, hdrs)
print("verified")
try:
verify([b"whsec_new"], body.replace(b"paid", b"void"), hdrs)
except PermissionError as e:
print("tampered:", e)
🔍 Interviewer Notes: The must-haves are HMAC over the raw body, constant-time comparison, and a timestamp to stop replays. Mentioning secret rotation and that IP allow-lists are a supplement, not a replacement, shows maturity.
❌ Common Mistakes: Verifying a re-serialised JSON body (key order and whitespace change the signature). Comparing with == (timing side channel). No replay protection.
Q32: How do you design reliable webhook delivery on the sending side, and idempotent handling on the receiving side?
Technical Answer: Sender:
- Write the event to a durable outbox in the same transaction as the business change, then deliver asynchronously from a queue.
- Retry non-2xx responses and timeouts with exponential backoff and jitter over hours or days; give up into a dead-letter state the customer can see and replay.
- Use short timeouts (a few seconds); do not let slow receivers tie up workers.
- Isolate endpoints so one failing customer does not delay others; disable endpoints that fail for a long time and notify the owner.
- Include a unique event ID and the event type; do not promise ordering (or include a sequence/version so receivers can handle out-of-order events).
Receiver:
- Verify the signature, store the event, return 2xx quickly, and process asynchronously.
- Deduplicate by event ID (delivery is at-least-once).
- Prefer fetching current state from the sender's API when order matters, instead of trusting the event payload order.
import sqlite3
db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE processed_events (event_id TEXT PRIMARY KEY, received_at TEXT DEFAULT CURRENT_TIMESTAMP)")
db.execute("CREATE TABLE ledger (invoice_id TEXT, amount_cents INTEGER)")
def handle_webhook(event: dict) -> int:
"""Returns the HTTP status to send back. Duplicate deliveries are acknowledged but not reprocessed."""
try:
with db: # one transaction: dedup record + side effect commit together
db.execute("INSERT INTO processed_events (event_id) VALUES (?)", (event["id"],))
if event["type"] == "invoice.paid":
db.execute("INSERT INTO ledger VALUES (?, ?)",
(event["data"]["invoice_id"], event["data"]["amount_cents"]))
except sqlite3.IntegrityError:
return 200 # already processed: still 2xx so the sender stops retrying
return 200
if __name__ == "__main__":
evt = {"id": "evt_123", "type": "invoice.paid", "data": {"invoice_id": "inv_9", "amount_cents": 5000}}
print(handle_webhook(evt), handle_webhook(evt)) # 200 200
print(db.execute("SELECT COUNT(*) FROM ledger").fetchone()[0]) # 1
🔍 Interviewer Notes: They want the outbox pattern on the sender, at-least-once delivery acknowledged openly, and a dedup record written in the same transaction as the side effect on the receiver.
❌ Common Mistakes: Doing slow work before returning 2xx (sender times out and retries, causing duplicates). Returning an error for duplicates (sender keeps retrying). Dedup check and side effect in separate transactions.
Q33: What is WebTransport, and when would you consider it over WebSockets?
Technical Answer: WebTransport is a browser API and protocol for client-server communication over HTTP/3 (QUIC). Compared with a WebSocket, which is one ordered byte stream over TCP, a WebTransport session offers:
- Multiple independent streams (bidirectional and unidirectional) inside one session, without head-of-line blocking between them.
- Unreliable datagrams: messages that may be dropped rather than retransmitted, useful when stale data is worthless.
- QUIC benefits: fast connection setup and connection migration when the client's network changes.
Good fits: multiplayer games, live media and low-latency telemetry where some loss is acceptable and independent streams matter.
Reasons to stay with WebSockets today: universal browser and library support, mature server ecosystem, and network paths that block or throttle UDP (some corporate networks), where WebTransport cannot work and you need a fallback anyway.
WebSocket (TCP) WebTransport (QUIC / HTTP/3)
one ordered stream session
[m1][m2 lost][m3 waits][m4 waits] stream A: [a1][a2] independent
stream B: [b1][b2 lost] only B waits
datagrams: [d1] [d2 x] [d3] never retransmitted
🔍 Interviewer Notes: They want the two capabilities WebSocket lacks: multiple independent streams and unreliable datagrams. Honest discussion of support and UDP-blocking networks shows judgement.
❌ Common Mistakes: Calling WebTransport "WebSockets over HTTP/3." Choosing it for an ordinary chat or notification feature where WebSocket or SSE already works everywhere.
Q34: How do you choose a protocol for chat, notifications, live dashboards and collaborative editing?
Technical Answer: Ask four questions: Which direction does data flow? How fresh must it be? Can updates be lost or coalesced? What clients and networks must be supported?
| Feature | Direction | Good default | Why |
|---|---|---|---|
| In-app notifications | Server → client | SSE (or long polling fallback) | One-way, low rate, plain HTTP, auto-reconnect with Last-Event-ID |
| Live dashboard / prices | Server → client, high rate | SSE or WebSocket with conflation | Only the latest value matters; drop intermediate updates for slow clients |
| Chat | Both ways, low latency | WebSocket | Typing indicators, read receipts, sends and receives on one connection |
| Collaborative editing | Both ways, ordered ops | WebSocket + CRDT/OT engine | Needs bidirectional, ordered operations and presence |
| Mobile background alerts | Server → device | Platform push (APNs/FCM) | App is not running; no socket is open |
| Service-to-service streams | Both ways | gRPC streaming | Typed contracts, HTTP/2 flow control |
| Rare status checks | Client → server | Polling | Simplest; nothing to keep open |
Does the client need to send frequent messages on the same channel?
├─ no ─> Is the app in the foreground?
│ ├─ no ──> platform push notification
│ └─ yes ─> updates more often than about once a minute?
│ ├─ no ──> polling
│ └─ yes ─> SSE
└─ yes ─> browser client? ── yes ─> WebSocket
└─ no ──> gRPC bidirectional streaming
🔍 Interviewer Notes: Interviewers reward reasoning from requirements over naming a favourite technology. Mentioning platform push for mobile and conflation for dashboards shows real product experience.
❌ Common Mistakes: "WebSockets for everything." Forgetting mobile apps cannot keep sockets open in the background. Ignoring how corporate proxies or CDNs handle long-lived connections.
Q35: How do HTTP/2 and HTTP/3 change things for SSE and WebSockets?
Technical Answer:
- SSE over HTTP/2 or HTTP/3: each
EventSourceis just one stream on a shared connection, so the HTTP/1.1 per-origin connection limit (which several SSE tabs could exhaust) no longer applies. Nothing changes in the SSE format or the server code. - WebSockets over HTTP/2: the classic handshake uses the HTTP/1.1
Upgrademechanism, which does not exist in HTTP/2. RFC 8441 defines an "extended CONNECT" method to run a WebSocket as one HTTP/2 stream. Support varies across browsers, proxies and servers, so many deployments still negotiate HTTP/1.1 for WebSocket connections even when the rest of the site uses HTTP/2. - Practical consequence: a CDN or load balancer may terminate HTTP/2 or HTTP/3 from the browser and speak HTTP/1.1 to your origin. Check what each hop supports and configure idle timeouts on every hop for long-lived streams.
# NGINX (inside the http { } block): serve HTTP/2 to browsers;
# WebSocket upgrades still proxy over HTTP/1.1 to the app
upstream app_upstream {
server 127.0.0.1:8000;
keepalive 32;
}
server {
listen 443 ssl;
http2 on;
server_name app.example.com;
location /events { # SSE
proxy_pass http://app_upstream;
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_buffering off; # deliver each event immediately
proxy_read_timeout 1h;
}
location /ws { # WebSocket
proxy_pass http://app_upstream;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 1h;
}
}
🔍 Interviewer Notes: The two key facts: HTTP/2 removes the SSE connection-limit problem, and WebSockets over HTTP/2 need a separate mechanism that is not universally deployed. Asking "what does each hop speak?" is a senior habit.
❌ Common Mistakes: Assuming WebSockets automatically run over HTTP/2 once it is enabled. Leaving proxy buffering on for SSE. Forgetting proxy_read_timeout so the proxy closes quiet streams after its default 60 seconds.
Senior/Principal Real-Time Scenarios
Q36: WebSocket connections drop every 60 seconds in production but not locally. What is wrong?
Level: Senior
Scenario: A support-chat feature works perfectly on developer machines. In production, behind NGINX and a cloud load balancer, idle conversations disconnect after exactly 60 seconds and the client reconnects, sometimes losing messages sent in between.
Impact: Agents miss customer messages; reconnect churn doubles server load during quiet hours.
Root Cause: Idle timeouts on intermediate hops. NGINX's proxy_read_timeout defaults to 60 s, and the cloud load balancer's idle timeout is also 60 s. When nobody types, no bytes flow, so each hop treats the connection as idle and closes it. Locally there is no proxy, so the problem never appears. Messages are lost because the client does not resume from a cursor after reconnecting.
Fix:
- Send WebSocket pings from the server more often than the smallest idle timeout on the path (for example every 25 s).
- Raise proxy timeouts for WebSocket routes and forward the upgrade headers correctly.
- Make reconnects lossless: clients resume with their last sequence number and the server replays from history.
# Inside the http { } block
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
upstream chat_backend {
server 127.0.0.1:8765;
}
server {
listen 80;
server_name chat.example.com;
location /ws {
proxy_pass http://chat_backend;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_set_header Host $host;
proxy_read_timeout 3600s; # app pings every 25 s keep the connection active
proxy_send_timeout 3600s;
}
}
import asyncio
import websockets
async def handler(ws) -> None:
async for message in ws:
await ws.send(message)
async def main() -> None:
# Ping every 25 s: below both NGINX's and the load balancer's 60 s idle timeouts
async with websockets.serve(handler, "127.0.0.1", 8765, ping_interval=25, ping_timeout=20):
await asyncio.Future()
if __name__ == "__main__":
asyncio.run(main())
Workflow Diagram:
browser <--> cloud LB (idle 60s) <--> NGINX (proxy_read_timeout 60s) <--> app
Broken: no traffic for 60 s -> LB/NGINX close -> client reconnects -> messages in gap lost
Fixed: app ping every 25 s -> every hop sees traffic -> connection stays up
client reconnect (deploys, network loss) -> resume ?after=<last_seq> -> no gap
Interviewer Notes: "Exactly 60 seconds" should trigger "idle timeout somewhere." They want heartbeats tuned below every hop's timeout and lossless resume, not just bigger timeouts.
Red Flags: Setting every timeout to infinity without heartbeats (dead connections are never detected). Blaming the client library. Ignoring message loss across reconnects.
Q37: Every deploy causes a reconnect storm that knocks over the real-time cluster. How do you fix it?
Level: Senior
Scenario: A notifications service holds 400,000 WebSocket connections across 20 pods. During a rolling deploy, each terminated pod drops ~20,000 clients at once. They all reconnect within a second, overload the authentication service and the remaining pods, which fail health checks and restart, dropping even more clients.
Impact: 10-20 minute partial outages on every deploy; deploys are now scheduled at night.
Root Cause: Synchronised reconnects (thundering herd) caused by (1) clients reconnecting immediately with no jitter, (2) pods closing every connection at the same instant, and (3) no server-side admission control, so a reconnect spike turns into cascading failure.
Fix:
- Client: exponential backoff with full jitter; the first reconnect also waits a random delay.
- Server drain: on SIGTERM, stop accepting new connections and close existing ones gradually over the termination grace period, using close code 1012 (Service Restart) so clients know it is planned.
- Admission control: limit new connections per second per pod; reject the excess quickly (HTTP 503 with
Retry-After) instead of accepting more than the pod can authenticate. - Cheaper reconnects: cache auth results or use short-lived resumable session tokens so reconnecting does not hit the auth service every time.
- Deploy tuning: fewer pods replaced at once (
maxUnavailable), and aterminationGracePeriodSecondslong enough to drain.
import asyncio
import random
import signal
import websockets
CONNECTIONS: set = set()
DRAIN_SECONDS = 60
async def handler(ws) -> None:
CONNECTIONS.add(ws)
try:
async for msg in ws:
await ws.send(msg)
finally:
CONNECTIONS.discard(ws)
async def drain() -> None:
"""Close connections spread over DRAIN_SECONDS instead of all at once."""
conns = list(CONNECTIONS)
random.shuffle(conns)
if not conns:
return
gap = DRAIN_SECONDS / len(conns)
for i, ws in enumerate(conns):
if i:
await asyncio.sleep(gap)
await ws.close(code=1012, reason="service restart") # 1012 = Service Restart
async def main() -> None:
stop = asyncio.get_running_loop().create_future()
asyncio.get_running_loop().add_signal_handler(signal.SIGTERM, stop.set_result, None)
server = await websockets.serve(handler, "0.0.0.0", 8765)
await stop
server.close(close_connections=False) # stop accepting; keep existing sockets for now
await drain()
await server.wait_closed()
if __name__ == "__main__":
asyncio.run(main())
# Kubernetes: give the pod time to drain and limit how many pods restart at once
spec:
strategy:
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
template:
spec:
terminationGracePeriodSeconds: 90 # > DRAIN_SECONDS
Workflow Diagram:
Before: pod killed -> 20k sockets closed at t=0 -> 20k reconnects at t=0.1s
-> auth service + other pods overloaded -> health checks fail -> more pods restart
After: SIGTERM -> stop accepting -> close 20k sockets over 60 s (code 1012)
-> clients wait random(0, backoff) -> reconnects spread over minutes
-> admission control rejects excess with 503 + Retry-After
Interviewer Notes: They want fixes on both sides (client jitter and server-side drain plus admission control) and recognition of the cascading-failure loop. Principal-level candidates also reduce the per-reconnect cost (auth caching, resumable sessions).
Red Flags: "Add more pods." Relying only on client changes when old app versions in the field will never update. Deploying only at night as the long-term answer.
Q38: A live dashboard server's memory climbs until it is OOM-killed during market spikes. Why?
Level: Senior
Scenario: A trading dashboard pushes price updates to 30,000 browser clients over WebSockets. During volatile periods (10× normal update rate), server memory grows by gigabytes and the process is OOM-killed, disconnecting everyone.
Impact: Dashboards go dark exactly when users need them most.
Root Cause: No backpressure. The fan-out loop queues every update for every client. Clients on slow networks or busy laptops cannot keep up; their per-connection send buffers grow without limit. Most of the queued messages are obsolete anyway — a price from 3 seconds ago is useless once a newer one exists.
Fix: Conflation — keep only the latest value per symbol per client, and have a writer task that sends whatever is current when the socket is ready. Memory per client is now bounded by the number of symbols it subscribes to, not by the update rate. Add a hard limit: disconnect clients whose socket has been unwritable for too long.
import asyncio
import json
class ConflatingSender:
"""Latest-value-wins buffer per client: memory is O(subscribed symbols), not O(update rate)."""
def __init__(self, send, flush_interval: float = 0.1):
self._send = send # coroutine function, e.g. ws.send
self._pending: dict[str, float] = {} # symbol -> latest price
self._wake = asyncio.Event()
self._interval = flush_interval
def update(self, symbol: str, price: float) -> None:
self._pending[symbol] = price # overwrite: older value is obsolete
self._wake.set()
async def run(self) -> None:
while True:
await self._wake.wait()
self._wake.clear()
batch, self._pending = self._pending, {}
await asyncio.wait_for(self._send(json.dumps(batch)), timeout=5) # slow client -> error -> disconnect
await asyncio.sleep(self._interval) # cap at ~10 frames per second per client
async def demo() -> None:
sent: list[str] = []
async def fake_send(msg: str) -> None:
sent.append(msg)
sender = ConflatingSender(fake_send)
task = asyncio.create_task(sender.run())
for i in range(1000): # 1000 updates arrive in a burst
sender.update("AAPL", 190 + i / 100)
sender.update("MSFT", 410 + i / 100)
await asyncio.sleep(0.05)
task.cancel()
print(len(sent), sent[-1]) # 1 frame with only the latest prices
if __name__ == "__main__":
asyncio.run(demo())
Workflow Diagram:
market feed (10k updates/s)
|
fan-out loop --update()--> per-client latest map {AAPL: 191.2, MSFT: 412.0}
| (overwrites, never grows)
writer task (<= 10 frames/s, 5 s send timeout)
|
slow client? -> timeout -> disconnect -> client resyncs snapshot
Interviewer Notes: Spotting that stale prices are worthless, and therefore conflation is safe, is the key insight. They also want a bounded, per-client design and a policy for clients that never catch up.
Red Flags: Adding RAM or raising the memory limit. Global locks around sends. Throttling the market feed for everyone because of a few slow clients.
Q39: After mobile users reconnect, chat messages appear twice or out of order. How do you fix it?
Level: Senior
Scenario: In a chat app, users on flaky mobile networks report duplicated messages and messages appearing in the wrong order after the app reconnects. Investigation shows clients resend messages that did not get an acknowledgement before the disconnect, and the server orders messages by the arrival time at whichever server handled them.
Impact: Confusing conversations, duplicate payment requests sent through chat, loss of user trust.
Root Cause:
- Retries without idempotency: a message reached the server, but the acknowledgement was lost in the disconnect. The client resent it and the server stored it twice.
- No authoritative per-conversation order: different servers stamped messages with their own clocks, so ordering depended on clock skew and network timing.
Fix:
- The client generates a message ID (UUID) once and reuses it on every retry; the server deduplicates on
(conversation_id, client_msg_id). - The server assigns a per-conversation sequence number atomically when storing the message; clients render by sequence, not by local or server timestamps.
- On reconnect, clients request everything after their last seen sequence number.
import sqlite3
import uuid
db = sqlite3.connect(":memory:")
db.executescript("""
CREATE TABLE messages (
conversation_id TEXT NOT NULL,
seq INTEGER NOT NULL,
client_msg_id TEXT NOT NULL,
body TEXT NOT NULL,
PRIMARY KEY (conversation_id, seq),
UNIQUE (conversation_id, client_msg_id)
);
""")
def store_message(conversation_id: str, client_msg_id: str, body: str) -> int:
"""Idempotent append. Returns the message's sequence number (same value on retries)."""
with db:
row = db.execute(
"SELECT seq FROM messages WHERE conversation_id = ? AND client_msg_id = ?",
(conversation_id, client_msg_id),
).fetchone()
if row:
return row[0] # retry of a message we already stored
(next_seq,) = db.execute(
"SELECT COALESCE(MAX(seq), 0) + 1 FROM messages WHERE conversation_id = ?",
(conversation_id,),
).fetchone()
db.execute("INSERT INTO messages VALUES (?, ?, ?, ?)",
(conversation_id, next_seq, client_msg_id, body))
return next_seq
def messages_after(conversation_id: str, after_seq: int) -> list[tuple[int, str]]:
return db.execute(
"SELECT seq, body FROM messages WHERE conversation_id = ? AND seq > ? ORDER BY seq",
(conversation_id, after_seq),
).fetchall()
if __name__ == "__main__":
mid = str(uuid.uuid4())
print(store_message("c1", mid, "hi"), store_message("c1", mid, "hi")) # 1 1 (retry deduplicated)
store_message("c1", str(uuid.uuid4()), "how are you?")
print(messages_after("c1", 0)) # [(1, 'hi'), (2, 'how are you?')]
In a multi-writer production database, the sequence must be assigned atomically: a per-conversation counter row updated in the same transaction, or a single partition owner per conversation (for example, all writes for one conversation go through one Kafka partition). The PRIMARY KEY (conversation_id, seq) constraint makes a race fail loudly instead of creating two messages with the same number.
Workflow Diagram:
client: msg id=abc, send ----> server stores seq=17, ACK ----X (lost in disconnect)
client: reconnect, resend id=abc ----> server finds (c1, abc) -> returns seq=17 (no duplicate)
client: GET ?after=16 ----> [17 abc "hi"] [18 ...] -> render in seq order
Interviewer Notes: They want both halves: client-generated idempotency keys for duplicates and a server-assigned per-conversation sequence for order. Senior answers explain how the sequence stays correct with concurrent writers.
Red Flags: Ordering by client timestamps. Deduplicating by message body. Claiming "WebSockets guarantee ordering" across reconnects.
Q40: Polling for notifications is costing too much. How do you migrate a million clients to push?
Level: Principal
Scenario: A web and mobile app polls GET /notifications every 5 seconds. With 1 million concurrently active clients that is about 200,000 requests per second, over 99% returning "nothing new." The API fleet, database read replicas and load balancer costs are dominated by this endpoint.
Impact: Large infrastructure cost, database load that hurts other features, and still up to 5 seconds of notification delay.
Root Cause: Pull-based design where request rate scales with clients × poll frequency, not with the actual rate of notifications. Each poll also hits the database.
Fix (phased):
- Immediate relief: make polls cheap. Keep a per-user "latest notification version" in a cache; return 304 / empty from the cache without touching the database. Add jitter and server-controlled poll intervals.
- Push channel: SSE for web clients (one-way, HTTP-friendly, built-in
Last-Event-IDresume); platform push (APNs/FCM) for mobile apps in the background; WebSocket only if the app already has one. - Fan-out architecture: notification service publishes to a pub/sub backplane partitioned by user; edge connection servers hold SSE connections and subscribe for their connected users.
- Rollout: feature flag per percentage of users, keep polling as fallback (slow interval, for example 60 s) for clients where streaming fails, measure connection counts per server and memory per connection.
import asyncio
import json
from fastapi import FastAPI, Header, Request
from fastapi.responses import StreamingResponse
app = FastAPI()
subscribers: dict[str, set[asyncio.Queue]] = {} # user_id -> queues on THIS server
async def deliver(user_id: str, event_id: int, payload: dict) -> None:
"""Called by the pub/sub consumer when a notification for user_id arrives."""
for q in subscribers.get(user_id, ()):
if q.full():
q.get_nowait() # bounded: drop oldest for slow clients
q.put_nowait((event_id, payload))
@app.get("/notifications/stream")
async def stream(request: Request, user_id: str, last_event_id: int = Header(default=0, alias="Last-Event-ID")):
q: asyncio.Queue = asyncio.Queue(maxsize=100)
subscribers.setdefault(user_id, set()).add(q)
async def gen():
try:
# A real service would first replay events after last_event_id from storage.
yield "retry: 5000\n\n"
while not await request.is_disconnected():
try:
event_id, payload = await asyncio.wait_for(q.get(), timeout=15)
yield f"id: {event_id}\nevent: notification\ndata: {json.dumps(payload)}\n\n"
except TimeoutError:
yield ": ping\n\n" # keep intermediaries from closing the stream
finally:
subscribers[user_id].discard(q)
return StreamingResponse(gen(), media_type="text/event-stream",
headers={"Cache-Control": "no-cache", "X-Accel-Buffering": "no"})
Workflow Diagram:
Before: 1M clients x (1 req / 5 s) = 200k req/s -> API -> DB (>99% empty)
After:
notification service --publish(user)--> pub/sub (partitioned by user)
|
SSE edge servers (subscribed for their connected users)
|
web clients: 1 long-lived stream each, events only when something happens
mobile background: APNs / FCM
fallback: cached poll every 60 s
Interviewer Notes: Principal-level answers quantify the before/after, phase the migration (quick win first), pick different channels for web and mobile background, and plan for fallback and observability. Calculating connection capacity per server is a plus.
Red Flags: "Switch to WebSockets" with no fan-out design or fallback. Ignoring mobile background constraints. Big-bang migration with no flag or rollback.
Q41: Users' presence indicators flicker between online and offline on mobile. How do you fix it?
Level: Senior
Scenario: A messaging app shows a green "online" dot. On mobile networks, users' status flips online/offline several times a minute. Each flip triggers a presence broadcast to all their contacts, and the presence service is now the largest source of WebSocket traffic.
Impact: Misleading UI, battery drain on recipients' phones, and presence traffic that grows with contacts × flaps.
Root Cause: Presence is tied directly to socket open/close events. Mobile connections drop and reconnect constantly (network handovers, app backgrounding), so every short blip becomes an "offline" then "online" event, and each event is broadcast immediately.
Fix:
- Drive presence from heartbeats with a TTL, not from socket events.
- Debounce offline: when the last connection closes, wait a grace period (for example 30-60 s) before declaring the user offline; cancel if they reconnect.
- Publish only state changes after debouncing, and only to contacts who are currently viewing.
- Aggregate across devices: online if any device is online.
import asyncio
class PresenceDebouncer:
"""Emits 'offline' only if a user stays disconnected for the whole grace period."""
def __init__(self, publish, grace_seconds: float = 45.0):
self._publish = publish # async callable(user_id, status)
self._grace = grace_seconds
self._connections: dict[str, int] = {} # user_id -> open connections (all devices)
self._pending_offline: dict[str, asyncio.Task] = {}
self._online: set[str] = set()
async def connected(self, user_id: str) -> None:
self._connections[user_id] = self._connections.get(user_id, 0) + 1
if task := self._pending_offline.pop(user_id, None):
task.cancel() # came back within grace: no flap
if user_id not in self._online:
self._online.add(user_id)
await self._publish(user_id, "online")
def disconnected(self, user_id: str) -> None:
self._connections[user_id] -= 1
if self._connections[user_id] == 0 and user_id not in self._pending_offline:
self._pending_offline[user_id] = asyncio.create_task(self._go_offline_later(user_id))
async def _go_offline_later(self, user_id: str) -> None:
await asyncio.sleep(self._grace)
self._pending_offline.pop(user_id, None)
if self._connections.get(user_id, 0) == 0:
self._online.discard(user_id)
await self._publish(user_id, "offline")
async def demo() -> None:
events = []
async def publish(user: str, status: str) -> None:
events.append((user, status))
p = PresenceDebouncer(publish, grace_seconds=0.2)
await p.connected("alice")
p.disconnected("alice") # network blip
await asyncio.sleep(0.05)
await p.connected("alice") # back within grace
p.disconnected("alice") # really leaves
await asyncio.sleep(0.3)
print(events) # [('alice', 'online'), ('alice', 'offline')]
if __name__ == "__main__":
asyncio.run(demo())
In a multi-server deployment, the connection count and pending timers live in a shared store (for example a Redis hash per user with per-connection keys that expire), but the debouncing logic is the same.
Workflow Diagram:
socket close --> connections == 0? --no--> nothing
| yes
start 45 s grace timer
|
reconnect before timer? --yes--> cancel timer (no events published)
| no
publish "offline" once --> only to contacts currently viewing
Interviewer Notes: They want "presence is soft state; debounce it" and broadcast reduction. Measuring flap rate before and after, and accepting slightly stale presence as a product trade-off, are senior signals.
Red Flags: Making heartbeats more frequent (more battery and traffic, same flapping). Broadcasting every connection event to every contact.
Q42: One gRPC server pod is at 90% CPU while the others are idle. Why, and how do you fix it?
Level: Senior
Scenario: A Python gRPC client service calls a recommendations service with 6 pods behind a Kubernetes ClusterIP Service. After a scale-up from 2 to 6 pods, the new pods receive almost no traffic; the original two run hot and their latency climbs.
Impact: p99 latency up 4×; autoscaling does not help because new pods stay idle.
Root Cause: gRPC uses long-lived HTTP/2 connections and multiplexes all calls over them. kube-proxy balances at L4: it picks a backend pod once per TCP connection. Each client opened its connections when only two pods existed, and those connections never close, so every subsequent call goes to the same two pods. New pods only get traffic from brand-new connections.
Fix (choose by environment):
- Client-side load balancing: resolve all pod IPs via a headless Service (
clusterIP: None) and use gRPC'sround_robinpolicy so the client opens a connection to each pod and spreads calls. - L7 proxy / service mesh (Envoy, Linkerd, Istio) that balances individual HTTP/2 requests.
- Server-side
max_connection_age: servers periodically ask clients to reconnect (GOAWAY), so connections redistribute after scaling. This is a useful complement to option 1 or 2.
apiVersion: v1
kind: Service
metadata:
name: recommendations-headless
spec:
clusterIP: None # headless: DNS returns every ready pod IP
selector:
app: recommendations
ports:
- name: grpc
port: 50051
import grpc
# Client: resolve all pod IPs and round-robin calls across them
channel = grpc.insecure_channel(
"dns:///recommendations-headless.default.svc.cluster.local:50051",
options=[
("grpc.lb_policy_name", "round_robin"),
("grpc.dns_min_time_between_resolutions_ms", 10_000),
],
)
# Server: force periodic reconnects so traffic rebalances after scale-up
from concurrent import futures
server = grpc.server(
futures.ThreadPoolExecutor(max_workers=32),
options=[
("grpc.max_connection_age_ms", 5 * 60 * 1000), # send GOAWAY after ~5 minutes
("grpc.max_connection_age_grace_ms", 30 * 1000), # let in-flight calls finish
],
)
Workflow Diagram:
Before (L4, one long-lived connection per client):
client-1 ==HTTP/2==> pod-a (all calls)
client-2 ==HTTP/2==> pod-b (all calls)
pod-c, pod-d, pod-e, pod-f: idle
After (client-side round_robin over headless DNS):
client-1 ==> pod-a, pod-b, pod-c, pod-d, pod-e, pod-f (calls spread per request)
+ max_connection_age: connections recycle so new pods join within minutes
Interviewer Notes: The core insight is "L4 balances connections; gRPC multiplexes requests over few connections." Strong candidates compare client-side LB, mesh and connection-age approaches with their trade-offs (client complexity versus extra hop).
Red Flags: "Add more replicas" (they stay idle). Disabling HTTP/2 connection reuse to force a new connection per call. Not knowing what a headless Service is.
Q43: WebSocket clients miss messages whenever a server restarts its Redis subscription. Why?
Level: Senior
Scenario: A collaboration app fans out document updates through Redis Pub/Sub to WebSocket servers. After Redis failovers, network blips or WebSocket server restarts, some users' documents silently diverge: they never received certain updates.
Impact: Users overwrite each other's work; support tickets about "lost edits."
Root Cause: Redis Pub/Sub is fire-and-forget (at-most-once). Messages are delivered only to subscribers connected at that instant; nothing is stored. Any gap in a server's subscription — reconnecting to Redis, failover, a restart — loses every message published during the gap, and neither the server nor the client knows.
Fix:
- Use a durable, replayable log: Redis Streams (
XADD/XREADfrom a last-seen ID), Kafka, or NATS JetStream. - Each WebSocket server tracks the last stream ID it processed and resumes from it after reconnecting.
- Clients also track a per-document version and request missing updates on reconnect (end-to-end gap detection, independent of the backplane).
- Trim the stream by length or age (
MAXLEN ~) to bound memory.
import asyncio
import redis.asyncio as redis
r = redis.Redis()
async def publish_update(doc_id: str, payload: str) -> bytes:
# Durable append; approximate trimming keeps roughly the last 100k entries
return await r.xadd(f"doc:{doc_id}", {"payload": payload}, maxlen=100_000, approximate=True)
async def follow(doc_id: str, deliver, last_id: str = "$") -> None:
"""Read new entries; after a reconnect, resume from last_id so nothing is skipped."""
stream = f"doc:{doc_id}"
while True:
try:
results = await r.xread({stream: last_id}, block=5000, count=100)
for _stream, entries in results:
for entry_id, fields in entries:
await deliver(entry_id, fields[b"payload"])
last_id = entry_id # advance only after delivering
except (redis.ConnectionError, redis.TimeoutError):
await asyncio.sleep(1) # reconnect and continue from last_id
Workflow Diagram:
Pub/Sub: publisher --> Redis --> [server subscribed?] yes -> delivered
no -> LOST (no record)
Streams: publisher --XADD--> stream doc:42 [1-0][2-0][3-0][4-0] (stored)
server reconnects --XREAD from 2-0--> receives 3-0, 4-0 -> no gap
client reconnects with version=2 --> server replays 3, 4
Interviewer Notes: They want the candidate to know Redis Pub/Sub's delivery semantics and to replace it with a log when completeness matters, plus end-to-end gap detection on the client. Calling out that "it worked in testing" because tests never hit a disconnect is a good insight.
Red Flags: Adding retries on the publisher (does not help absent subscribers). Assuming Redis Pub/Sub persists messages. No client-side versioning.
Q44: A public GraphQL API is being taken down by a handful of expensive queries. How do you protect it?
Level: Principal
Scenario: A public GraphQL API lets partners query organization → teams → members → projects → tasks. One partner's dashboard sends a deeply nested query that fans out to millions of rows; a few concurrent copies saturate the database and the API times out for everyone. Request-per-minute rate limiting did not trigger because the request count was low.
Impact: Platform-wide outage caused by a few requests that each looked legitimate.
Root Cause: GraphQL lets the client choose the shape and size of the work. Without query cost controls, one request can cost millions of times more than another, so request-count rate limits are meaningless. Missing pagination limits on list fields and no per-request timeout made it worse.
Fix (defence in depth):
- Depth limit and cost analysis before execution: estimate cost from list sizes (
first: N) and reject queries above a budget. - Mandatory pagination with a maximum page size on every list field.
- Rate limit by cost, not by request count (each client has a cost budget per minute).
- Persisted / allow-listed queries for known first-party clients; arbitrary queries only for authenticated partners with budgets.
- Execution timeouts and DataLoader batching to bound database load.
from graphql import (
FieldNode,
FragmentDefinitionNode,
FragmentSpreadNode,
InlineFragmentNode,
OperationDefinitionNode,
SelectionSetNode,
parse,
)
MAX_DEPTH = 5
def analyse(query: str) -> tuple[int, int]:
"""Return (max_depth, estimated_cost) for a query document."""
doc = parse(query)
fragments = {d.name.value: d for d in doc.definitions if isinstance(d, FragmentDefinitionNode)}
def walk(sel: SelectionSetNode | None, depth: int, multiplier: int, seen: frozenset) -> tuple[int, int]:
if sel is None:
return depth, 0
max_d, cost = depth, 0
for node in sel.selections:
if isinstance(node, FieldNode):
size = next((int(a.value.value) for a in (node.arguments or ())
if a.name.value in ("first", "last") and hasattr(a.value, "value")), None)
child_mult = multiplier * (size if size is not None else 1)
d, c = walk(node.selection_set, depth + 1, child_mult, seen)
max_d, cost = max(max_d, d), cost + multiplier + c
elif isinstance(node, InlineFragmentNode):
d, c = walk(node.selection_set, depth, multiplier, seen)
max_d, cost = max(max_d, d), cost + c
elif isinstance(node, FragmentSpreadNode) and node.name.value not in seen:
frag = fragments[node.name.value]
d, c = walk(frag.selection_set, depth, multiplier, seen | {node.name.value})
max_d, cost = max(max_d, d), cost + c
return max_d, cost
results = [walk(d.selection_set, 0, 1, frozenset())
for d in doc.definitions if isinstance(d, OperationDefinitionNode)]
return max(r[0] for r in results), sum(r[1] for r in results)
def enforce(query: str, budget: int = 10_000) -> None:
depth, cost = analyse(query)
if depth > MAX_DEPTH:
raise ValueError(f"query depth {depth} exceeds {MAX_DEPTH}")
if cost > budget:
raise ValueError(f"query cost {cost} exceeds budget {budget}")
if __name__ == "__main__":
q = """
query {
organization(id: "1") {
teams(first: 50) { members(first: 100) { projects(first: 100) { tasks(first: 100) { id title } } } }
}
}"""
print(analyse(q)) # (6, 100505052): depth 6, cost about 100 million
try:
enforce(q)
except ValueError as e:
print("rejected:", e)
This sketch only counts first/last arguments. A production implementation reads list types and per-field cost annotations from the schema, and rejects (or assigns a default size to) list fields queried without a page size.
Workflow Diagram:
request --> parse --> validate --> depth/cost analysis --> cost > budget? --yes--> reject (400 + cost info)
| no
v
per-client cost bucket has room? --no--> 429 with retry hint
| yes
v
execute with timeout + DataLoader batching + page-size caps
Interviewer Notes: Principal-level answers recognise that request-count rate limiting is the wrong unit for GraphQL, apply several layers (static analysis, pagination caps, cost budgets, persisted queries, timeouts) and give partners visibility into their query costs.
Red Flags: Disabling introspection as the main defence. Only adding database indexes. Per-request rate limiting with no cost model.
Q45: Webhook deliveries to one large customer are backing up and delaying everyone else's. What do you do?
Level: Principal
Scenario: A SaaS platform sends webhooks to thousands of customer endpoints from a shared worker pool consuming one queue. One large customer's endpoint starts responding in 25 seconds (just under the 30 s timeout) and receives 40% of all events. Within an hour, all workers are busy waiting on that endpoint, and webhooks for every other customer are delayed by over an hour.
Impact: Platform-wide webhook latency SLO breached; customers' integrations (order sync, payment confirmation) stall.
Root Cause: No isolation between tenants. A single shared queue and worker pool means one slow endpoint consumes the shared capacity (head-of-line blocking at the queue level). A long timeout made each slow delivery hold a worker for 25 seconds. Retries for the slow endpoint re-entered the same queue, adding more load.
Fix:
- Per-endpoint concurrency limits: each endpoint may use at most N workers at a time; excess events wait in that endpoint's own queue, not the shared one.
- Shorter timeouts (for example 5-10 s) and a published contract: receivers must acknowledge quickly and process asynchronously.
- Circuit breaker per endpoint: after repeated failures or timeouts, pause deliveries to that endpoint, back off, and notify the customer.
- Separate retry scheduling from first-attempt delivery so retries never block fresh events for healthy endpoints.
- Fair scheduling across tenants (for example, round robin across endpoint queues).
import asyncio
from collections import defaultdict
class PerEndpointLimiter:
"""Caps concurrent deliveries per endpoint so one slow receiver can't consume all workers."""
def __init__(self, per_endpoint: int = 4):
self._sems: dict[str, asyncio.Semaphore] = defaultdict(lambda: asyncio.Semaphore(per_endpoint))
def slot(self, endpoint: str) -> asyncio.Semaphore:
return self._sems[endpoint]
async def deliver(limiter: PerEndpointLimiter, endpoint: str, event_id: int, latency: float, log: list) -> None:
async with limiter.slot(endpoint):
await asyncio.sleep(latency) # stands in for the HTTP POST with a short timeout
log.append((endpoint, event_id))
async def demo() -> None:
limiter = PerEndpointLimiter(per_endpoint=2)
log: list = []
slow = [deliver(limiter, "big-customer", i, 0.5, log) for i in range(20)]
fast = [deliver(limiter, "small-customer", i, 0.01, log) for i in range(5)]
await asyncio.gather(*slow, *fast)
first_ten = [e for e, _ in log[:10]]
print(first_ten.count("small-customer"), "of the first 10 deliveries went to the healthy endpoint")
if __name__ == "__main__":
asyncio.run(demo())
Workflow Diagram:
Before: events --> [one shared queue] --> 50 workers (all stuck on big-customer, 25 s each)
After: events --> router --> queue per endpoint
big-customer [###########] max 4 in flight, breaker open on failures
small-customer [#] max 4 in flight
...
fair scheduler (round robin) --> worker pool (5 s timeout)
retries --> delayed retry queue (never blocks fresh events)
Interviewer Notes: The core issue is noisy-neighbour isolation. They want per-tenant concurrency limits, short timeouts, circuit breakers and fair scheduling, with customer notification as part of the operational answer.
Red Flags: Scaling the worker pool (the slow endpoint absorbs it). Dropping the slow customer's events silently. Raising the timeout.