Understanding the Event Stream Format #Permalink to this section

Part of SSE Protocol Fundamentals & Architecture.

The text/event-stream format is a deliberately minimal line-delimited protocol. Its simplicity is both its strength and its primary source of production bugs: a single missing newline, an unexpected BOM, or a proxy that buffers the response is enough to silently break your stream. This guide dissects every field, framing rule, and parser edge case so you can write correct server code, audit existing streams with curl, and build parsers that survive real network conditions.

How text/event-stream Works #Permalink to this section

Wire Format Grammar #Permalink to this section

The WHATWG HTML specification (“Server-sent events”, §9.2) defines the format precisely. Every event is a sequence of UTF-8 text lines ending with LF (\n). A CRLF (\r\n) line ending is also legal per spec, but most production tooling uses LF exclusively. An event is dispatched when the parser encounters an empty line — a line containing only \n.

The grammar in plain terms:

stream        = *event
event         = *field empty-line
field         = field-name ":" SP? field-value LF
             / ":" comment-text LF          ; comment, ignored
empty-line    = LF
field-name    = "data" / "event" / "id" / "retry" / token

A space after the colon is optional but conventional; parsers MUST strip exactly one leading space from the field value if present. data: foo and data:foo are equivalent; data: foo (two spaces) retains one trailing space in the value.

Fields Reference #Permalink to this section

data is the only field that accumulates. If an event block contains:

data: line one
data: line two

The browser dispatches a single MessageEvent whose .data property is "line one\nline two". This concatenation is what powers multi-line data field formatting.

id with an empty value (id: or id: ) resets the last-event-ID to the empty string — it does not clear the field. A NULL byte in the id value causes the field to be ignored entirely (spec §9.2.6).

retry takes effect immediately upon parsing; it does not require a blank line to activate. If the value is not a valid decimal integer the field is ignored.

Required HTTP Headers #Permalink to this section

The server response MUST include:

Content-Type: text/event-stream
Cache-Control: no-cache

Connection: keep-alive is implicit in HTTP/1.1 but worth setting explicitly for clarity. Transfer-Encoding: chunked is set by the runtime when Content-Length is absent and the response streams. Do not set Content-Length — doing so causes the browser to close the connection after receiving that many bytes, breaking the stream. For MIME type validation workflows see how to parse the text/event-stream MIME type correctly.

Server-Side Implementation #Permalink to this section

Node.js (built-in http) #Permalink to this section

import http from "node:http";

const HEARTBEAT_MS = 20_000; // 20 s — under typical LB idle timeout

http
  .createServer((req, res) => {
    if (req.url !== "/events") {
      res.writeHead(404).end();
      return;
    }

    // Required headers — flush immediately
    res.writeHead(200, {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      Connection: "keep-alive",
      // CORS if the client is on a different origin
      "Access-Control-Allow-Origin": "*",
    });

    // Announce reconnect interval (3 s) once at stream open
    res.write("retry: 3000\n\n");

    let counter = 0;

    // Heartbeat keeps the connection alive through idle load-balancer timeouts
    const heartbeat = setInterval(() => {
      res.write(": heartbeat\n\n");
    }, HEARTBEAT_MS);

    // Example: push a named event every second
    const ticker = setInterval(() => {
      counter++;
      const payload = JSON.stringify({ seq: counter, ts: Date.now() });
      // id field: monotonic counter sent as padded decimal
      res.write(`id: ${String(counter).padStart(10, "0")}\n`);
      res.write(`event: tick\n`);
      res.write(`data: ${payload}\n\n`); // blank line = dispatch
    }, 1000);

    // Clean up when the client disconnects (TCP FIN or RST)
    req.on("close", () => {
      clearInterval(heartbeat);
      clearInterval(ticker);
    });
  })
  .listen(3000);

Key points:

• res.write() in Node.js HTTP/1.1 flushes each chunk automatically when Transfer-Encoding: chunked is active.
• The req.on("close") handler is critical; without it, closed connections accumulate timers and leak memory.
• Never res.end() mid-stream unless you intend to close the connection and let the client reconnect.

Python (FastAPI / Starlette) #Permalink to this section

import asyncio
import time
from fastapi import FastAPI
from fastapi.responses import StreamingResponse

app = FastAPI()

async def event_generator(last_id: int = 0):
    """Yields SSE-formatted bytes; resumes from last_id on reconnect."""
    counter = last_id
    # Send retry interval once
    yield "retry: 3000\n\n"
    while True:
        await asyncio.sleep(1)
        counter += 1
        payload = f'{{"seq":{counter},"ts":{int(time.time())}}}'
        # Each field on its own line; blank line terminates the event
        yield f"id: {counter}\nevent: tick\ndata: {payload}\n\n"

@app.get("/events")
async def stream(last_event_id: str | None = None):
    last_id = int(last_event_id) if last_event_id else 0
    return StreamingResponse(
        event_generator(last_id),
        media_type="text/event-stream",
        headers={
            "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no",   # disables Nginx proxy buffering
        },
    )

X-Accel-Buffering: no is the single most-missed header in Python deployments behind Nginx. Without it, Nginx accumulates chunks until its buffer fills, causing visible stalls. See the full guide at Python FastAPI SSE Implementation Guide.

Go (net/http + http.Flusher) #Permalink to this section

package main

import (
    "fmt"
    "net/http"
    "time"
)

func eventsHandler(w http.ResponseWriter, r *http.Request) {
    flusher, ok := w.(http.Flusher)
    if !ok {
        http.Error(w, "streaming unsupported", http.StatusInternalServerError)
        return
    }

    w.Header().Set("Content-Type", "text/event-stream")
    w.Header().Set("Cache-Control", "no-cache")
    w.Header().Set("Connection", "keep-alive")
    w.Header().Set("X-Accel-Buffering", "no")

    // Announce retry interval
    fmt.Fprint(w, "retry: 3000\n\n")
    flusher.Flush()

    ticker := time.NewTicker(time.Second)
    defer ticker.Stop()

    var seq int64
    for {
        select {
        case <-r.Context().Done(): // client disconnect
            return
        case t := <-ticker.C:
            seq++
            fmt.Fprintf(w, "id: %d\nevent: tick\ndata: {\"seq\":%d,\"ts\":%d}\n\n",
                seq, seq, t.UnixMilli())
            flusher.Flush() // must flush after every write or bytes stay in buffer
        }
    }
}

func main() {
    http.HandleFunc("/events", eventsHandler)
    http.ListenAndServe(":3000", nil)
}

http.Flusher is the Go-specific obligation. Without the Flush() call after each write, the bufio.Writer wrapping http.ResponseWriter will hold bytes until the buffer fills (4 KB by default) before transmitting — a subtle source of artificial latency in low-throughput streams.

Client-Side Consumption #Permalink to this section

Native EventSource #Permalink to this section

// Browser / service worker
const es = new EventSource("/events", { withCredentials: false });

// Default handler — fires for events with no "event:" field, or event: message
es.onmessage = (e) => {
  console.log("seq:", JSON.parse(e.data).seq);
  console.log("lastEventId:", e.lastEventId); // mirrors the id: field
};

// Named-event handler — fires only for event: tick
es.addEventListener("tick", (e) => {
  const { seq, ts } = JSON.parse(e.data);
  console.log(`tick #${seq} at ${new Date(ts).toISOString()}`);
});

es.onerror = (e) => {
  // readyState: 0 = CONNECTING, 1 = OPEN, 2 = CLOSED
  if (es.readyState === EventSource.CLOSED) {
    console.error("Connection closed permanently — check server logs");
  }
  // readyState === CONNECTING: browser is auto-reconnecting using retry interval
};

// Clean up on page unload to avoid ghost connections
window.addEventListener("beforeunload", () => es.close());

When the browser reconnects it automatically includes Last-Event-ID: <last received id> in the request headers. The server must read this header and resume the stream from the next event after that ID — not replay the entire backlog.

fetch + ReadableStream (non-browser runtimes, POST bodies) #Permalink to this section

EventSource does not support POST requests or custom headers (beyond withCredentials). For those cases, parse the stream manually:

async function consumeSSE(url, signal) {
  const res = await fetch(url, {
    headers: { Accept: "text/event-stream", Authorization: "Bearer <token>" },
    signal,
  });

  if (!res.ok) throw new Error(`HTTP ${res.status}`);
  if (!res.body) throw new Error("No body");

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buf = "";
  let lastEventId = "";

  // Accumulate chunks into lines; parse events on blank line
  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buf += decoder.decode(value, { stream: true });

    const lines = buf.split("\n");
    buf = lines.pop() ?? ""; // last item may be incomplete

    let currentEvent = { type: "message", data: [], id: lastEventId };

    for (const line of lines) {
      if (line === "" || line === "\r") {
        // Blank line — dispatch event if data is non-empty
        if (currentEvent.data.length > 0) {
          dispatchEvent(currentEvent.type, currentEvent.data.join("\n"), currentEvent.id);
        }
        currentEvent = { type: "message", data: [], id: lastEventId };
      } else if (line.startsWith(":")) {
        // Comment — ignore (heartbeat)
      } else {
        const colonIdx = line.indexOf(":");
        const field = colonIdx === -1 ? line : line.slice(0, colonIdx);
        const value = colonIdx === -1 ? "" : line.slice(colonIdx + 2); // strip one space

        switch (field) {
          case "data":   currentEvent.data.push(value); break;
          case "event":  currentEvent.type = value; break;
          case "id":     if (!value.includes("\0")) lastEventId = value; break;
          case "retry":  /* update reconnect timer */ break;
        }
      }
    }
  }
}

This pattern is also necessary in Node.js server-side code that consumes upstream SSE feeds, and in test harnesses where you need to assert on individual events.

Edge Cases and Network Interference #Permalink to this section

Proxy and CDN Buffering #Permalink to this section

The single most common production failure: a reverse proxy or CDN sits between the server and client and buffers the entire response body before forwarding. The client sees nothing until the proxy’s buffer fills (often 4 KB or 64 KB) then gets a large burst.

BOM and Encoding Issues #Permalink to this section

A UTF-8 BOM (\xEF\xBB\xBF) at the very start of the stream violates the spec and corrupts the first field. The WHATWG parser strips it if it appears at byte offset 0, but some server frameworks prepend BOMs when writing text files. Disable BOM output explicitly in your framework’s text encoder.

CRLF vs LF #Permalink to this section

Both \n and \r\n are valid line terminators per spec (§9.2.6 step 6). However:

• Some proxy implementations split lines on \n only, turning \r\n into a trailing \r in the field value.
• Emit \n exclusively on the server side.

Empty id Field #Permalink to this section

id: (empty value) sets the last-event-ID to "" (empty string), which is not the same as never having sent an id. On reconnect, the browser sends Last-Event-ID: with an empty value — servers that parse this as “no ID” will replay from the beginning, producing duplicate events. Handle the empty-string case explicitly.

Premature Stream Termination #Permalink to this section

If the server sends \n\n without any data: field, the browser fires a zero-data message event (e.data === ""). This is sometimes used as a keep-alive ping but surprises handlers that unconditionally call JSON.parse(e.data). Guard with:

es.onmessage = (e) => {
  if (!e.data) return; // skip heartbeat pings
  const payload = JSON.parse(e.data);
  // ...
};

Connection Count and Payload Size #Permalink to this section

Maximum payload size limits for SSE streams covers the thresholds in detail, but operationally: each data: value lives in the browser’s event loop until your handler returns. Payloads over ~1 MB can cause frame drops in the main thread. Chunk large objects across multiple events or use a cursor to let the client fetch the full payload via a separate request.

Performance and Scale Considerations #Permalink to this section

Connection Cost Model #Permalink to this section

SSE uses one persistent HTTP connection per client. In HTTP/1.1, browsers cap connections per origin at 6 (shared across tabs). In HTTP/2, this cap is lifted — one multiplexed connection per origin, unlimited streams. For large fan-out deployments (> 10 000 concurrent clients) you will reach OS file-descriptor limits before you reach CPU limits; see configuring connection pools for high-concurrency SSE.

Memory Per Connection #Permalink to this section

A typical SSE connection in Node.js consumes ~50–150 KB of resident memory (socket buffers, parser state, write-stream internals). At 10 000 connections that is 500 MB–1.5 GB — plan accordingly. Avoid holding large per-connection in-memory buffers; instead keep a reference to the response object plus the last-event-ID and pull data from a shared cache (e.g., Redis) on demand.

For fan-out across multiple server instances, a Redis Pub/Sub fan-out pattern decouples event production from connection management: each server node subscribes to a channel and writes arriving messages to its local open connections.

Backpressure #Permalink to this section

EventSource has no backpressure mechanism. If the server writes faster than the client reads, the kernel TCP send buffer fills, then the writer blocks (in Node.js, res.write() returns false). Handle this by checking the return value and pausing production:

const ok = res.write(chunk);
if (!ok) {
  // pause upstream source until drain
  await once(res, "drain");
}

For rate limiting strategies see rate limiting and backpressure handling.

CPU: JSON Serialisation at Scale #Permalink to this section

At 10 000 connections receiving the same event, serialising the same JSON object 10 000 times wastes CPU. Serialise once outside the connection loop:

const serialised = `data: ${JSON.stringify(payload)}\n\n`;
for (const res of connections) {
  res.write(serialised);
}

Validation and Debugging #Permalink to this section

curl Stream Inspection #Permalink to this section

# -N disables output buffering; -v shows response headers
curl -v -N \
  -H "Accept: text/event-stream" \
  -H "Last-Event-ID: 0" \
  http://localhost:3000/events

# Verify chunked encoding is active (look for Transfer-Encoding: chunked)
# and Content-Length is absent (would break streaming)
curl -sI http://localhost:3000/events | grep -iE "content-type|transfer-encoding|content-length|cache-control"

Expected output:

Content-Type: text/event-stream
Cache-Control: no-cache
Transfer-Encoding: chunked

Content-Length must be absent. Any Content-Encoding: gzip also breaks SSE — gzip is a block codec and cannot flush partial output.

Browser DevTools #Permalink to this section

1. Open Network tab → filter by EventStream (Chrome) or XHR (Firefox).
2. Click the /events request row → select the EventStream sub-tab.
3. Each dispatched event appears with its type, data, lastEventId, and timestamp.
4. Response tab shows raw bytes including field names — useful for catching BOM issues.

Structured Logging #Permalink to this section

Emit a log entry for each event dispatched, keyed by connection ID, event type, and id field value. This lets you reconstruct the event sequence for any client and detect gaps:

// Wrap res.write to log each SSE write
function writeEvent(res, connId, { id, type = "message", data }) {
  const frame = `id: ${id}\nevent: ${type}\ndata: ${data}\n\n`;
  const ok = res.write(frame);
  logger.info({ connId, eventId: id, eventType: type, bytes: frame.length });
  return ok;
}

CI Validation Script #Permalink to this section

#!/usr/bin/env bash
# Receive 5 events then validate format
output=$(curl -sN --max-time 10 http://localhost:3000/events | head -n 30)

# Must contain at least one data: line
echo "$output" | grep -q "^data:" || { echo "FAIL: no data field"; exit 1; }

# Must not contain Content-Length (checked separately via headers)
headers=$(curl -sI http://localhost:3000/events)
echo "$headers" | grep -qi "content-length" && { echo "FAIL: Content-Length present"; exit 1; }
echo "$headers" | grep -qi "text/event-stream" || { echo "FAIL: wrong Content-Type"; exit 1; }

echo "PASS"

Production Checklist #Permalink to this section

• Response header Content-Type: text/event-stream confirmed in staging with curl -sI
• Cache-Control: no-cache present; no Expires or ETag headers
• Content-Length absent from response headers
• Transfer-Encoding: chunked confirmed active
• X-Accel-Buffering: no set (or proxy_buffering off in Nginx config)
• Heartbeat comment emitted every ≤ 20 s (tested by idling for 30 s with curl -v -N)
• Server reads Last-Event-ID request header and resumes stream correctly
• Empty Last-Event-ID value handled distinctly from absent header
• id: field included on every event (not only on “important” events)
• retry: field sent at stream open to override browser default
• Server cleans up connection resources on req.close / context cancellation
• File-descriptor limits raised (ulimit -n) to accommodate peak concurrent connections
• No Content-Encoding: gzip on SSE routes (check Nginx gzip_types directive)
• BOM disabled in all server-side text encoding paths

Frequently Asked Questions #Permalink to this section

Related #Permalink to this section

• SSE Protocol Fundamentals & Architecture
• How to Parse the text/event-stream MIME Type Correctly
• Maximum Payload Size Limits for SSE Streams
• Formatting Multi-Line data Fields in SSE
• Event ID & Retry Mechanism Design
• SSE vs WebSockets vs HTTP Polling