Using Retry Hints for Load Shedding

Part of Event ID & Retry Mechanism Design, under SSE Protocol Fundamentals & Architecture.

When an SSE service is overloaded — after an outage, during a deploy, when a popular event starts — the instinct from request/response systems is to return 503 Service Unavailable and let clients back off. With EventSource that instinct backfires: a 503 does not make the browser retry later, it makes it stop forever. The retry: field is the tool the protocol provides for pacing reconnects, and used deliberately it turns a reconnect storm into a smooth ramp. This guide covers the browser’s actual behaviour and a shedding strategy built on it.

Symptom & Developer Intent

  • After an outage, the service comes back and immediately falls over again under the reconnect wave.
  • The team added 503 responses under load, and users now have to reload the page to get live updates back.
  • Every client reconnects at the same moment because they all received the same retry: value.
  • A deploy causes a load spike on the remaining nodes as every drained client arrives at once.

The intent is to reduce connection pressure when overloaded while guaranteeing that every client eventually reconnects on its own, with arrivals spread over time.

Root Cause Analysis

EventSource reconnects in exactly two situations: a network error, and a successful (200 with text/event-stream) response that ends. Any other outcome — a 4xx or 5xx status, a wrong content type — “fails the connection”: readyState becomes CLOSED, an error event fires, and the browser never tries again unless application code creates a new EventSource.

What EventSource does with each server response Decision tree mapping server responses to EventSource behaviour: network errors and ended 200 streams reconnect after the retry interval, 204 stops quietly, and other statuses fail permanently. What EventSource does with each server response Network error, or 200 stream ended? Reconnect after retry ms yes no HTTP 204 No Content? Stop, no reconnect yes no 4xx / 5xx or wrong type? Fail permanently yes no Fail permanently
Only the first branch reconnects on its own. Shedding load with any other status silently turns off live updates for that user.

The reconnect delay is the last retry: value the stream sent, or a browser default of a few seconds if none was sent. Browsers may add their own backoff, but you cannot rely on it. If every client holds the same value and loses its connection at the same moment, they return in the same instant.

Step-by-Step Resolution

Step 1 — Send a jittered retry value on every stream 

function retryMs(base = 3000, spread = 7000) {
  return base + Math.floor(Math.random() * spread);    // 3–10 s, different per client
}
res.write(`retry: ${retryMs()}\n\n`);

Randomising per connection spreads any mass reconnect — an outage, a network blip at a large office — over the whole interval instead of a single instant.

Step 2 — Shed new connections softly: 200, a long retry, then close

When a node is over its connection budget, accept the request just enough to tell the client when to come back:

app.get('/api/stream', (req, res) => {
  if (openStreams.size >= MAX_STREAMS || overloaded()) {
    res.writeHead(200, { 'Content-Type': 'text/event-stream', 'Cache-Control': 'no-cache' });
    res.end(`retry: ${retryMs(20_000, 40_000)}\n: shedding load\n\n`);   // come back in 20–60 s
    metrics.shed.inc();
    return;
  }
  // …normal stream…
});

This is a successful, ended stream, so the browser schedules a reconnect after the new retry value. The client stays live-capable, and arrivals are spread over forty seconds.

Soft rejection keeps the client coming back Sequence diagram of an overloaded server answering a connection with 200, a long retry value and an immediate end, the browser waiting, and a later reconnect being accepted. Soft rejection keeps the client coming back Browser Overloaded node Recovered node GET /api/stream 200 retry: 41000, end EventSource waits 41 s, as instructed GET Last-Event-ID: 8812 replay, then live
The same user, with a 503, would have been left with a dead stream until the next page load.

Step 3 — Handle genuine errors in application code

For cases where a non-200 is correct — 401 when a session expired, 403 when access was revoked — handle the permanent failure explicitly:

es.onerror = () => {
  if (es.readyState !== EventSource.CLOSED) return;          // browser is retrying by itself
  // Permanent failure: find out why, then decide whether to reopen.
  fetch('/api/stream/status', { credentials: 'include' }).then((r) => {
    if (r.status === 401) return refreshSessionThen(openStream);
    setTimeout(openStream, backoff.next());                   // e.g. a proxy returned 502
  });
};

This also protects against 502s and 504s from proxies, which your server did not send but which fail the connection all the same. The full client pattern is in distinguishing fatal from transient SSE errors.

Step 4 — Budget reconnects per node

A node recovering from a restart receives a burst of reconnects, each of which may trigger an expensive replay. Rate-limit accepted connections per second and shed the excess softly:

const bucket = tokenBucket({ ratePerSec: 500, burst: 1000 });
if (!bucket.take()) return softReject(res, retryMs(5_000, 25_000));

With 20,000 clients and a budget of 500 per second, the node fills over about 40 seconds instead of receiving 20,000 connections in two.

Pair the admission budget with cheap connection setup. The first moments of a stream are usually its most expensive — authentication, a replay query, a snapshot — so a reconnect wave concentrates exactly that work. Cache snapshots for a second or two so simultaneous reconnects share one computation, bound replay size per connection, and make authentication a local signature check rather than a database lookup where possible. Each of these lowers the cost per admitted connection and lets the budget be set higher.

Step 5 — Raise retry values during incidents

A central setting can raise the base retry value for all streams during an incident, sent in a comment-plus-retry frame to existing connections:

function broadcastRetry(ms) {
  for (const res of openStreams) res.write(`retry: ${ms + Math.floor(Math.random() * ms)}\n\n`);
}

Existing streams are unaffected until they drop; if they do, they come back slowly.

Connection arrivals after a 60-second outage, 30,000 clients Line chart of reconnect arrivals per second after an outage ends, comparing a fixed three-second retry with jittered retry plus a per-node admission budget. Connection arrivals after a 60-second outage, 30,000 clients fixed retry 3 s jitter + budget 0 7.5 15 22.5 30 0 12 24 36 48 60 seconds after the service recovers arrivals per second (thousands)
A fixed retry concentrates every client into one spike. Jitter plus an admission budget turns it into a ramp the service can absorb.

Validation & Monitoring 

# Soft rejection returns 200 with a long retry and ends immediately.
curl -si http://localhost:3000/api/stream | head -8

# The browser must reconnect later on its own: watch the Network panel for a second request
# after roughly the retry interval, with no page reload.

Track accepted and shed connections per second, the retry values being sent, and the time for connection counts to recover after an incident. An incident review should be able to show that no client needed a reload to get live updates back.

Production Checklist

Frequently Asked Questions

Does EventSource retry after a 503?

No. Any non-200 status fails the connection permanently. The browser only reconnects after network errors or after a successful stream ends.

What is the default retry interval?

It is browser-defined, typically a few seconds. Always send your own value so behaviour is predictable across browsers.

Can the server tell a client to stop reconnecting?

Yes: respond with 204 No Content, which the specification defines as a signal to stop. Use it for finished streams, not for load shedding.

Should the retry value change over the life of a stream?

It can. A new retry field takes effect for the next reconnect, which makes it a lightweight control channel for raising delays during incidents.