Streaming an AI API response over server-sent events (SSE) means delivering the model’s output to the caller incrementally instead of waiting for the complete completion. In practice you enable it with a request flag — the OpenAI Responses API uses stream: true — and then consume a sequence of events rather than a single JSON body. The difficult part is rarely the flag. It is buffering, event parsing, and deciding what the application should do when a stream stops halfway through.
This guide is written for developers and technically curious practitioners who already have a working non-streaming API call and want to convert it. It covers the wire format, minimal implementations in Python, JavaScript and Go, a raw curl check you can run before touching application code, and the failure modes that most often make a stream arrive all at once at the very end.
What SSE actually sends on the wire
Server-sent events are a text protocol carried over a long-lived HTTP response. The response uses the content type text/event-stream and stays open while the server writes messages into it. Each message is a small block of text whose payload lines begin with data:.
The Chrome for Developers guide on how LLMs stream responses shows this concretely. A curl call to the Gemini API’s streamGenerateContent endpoint with alt=sse and --no-buffer logs a sequence of data: lines. An early line carries a short fragment such as “A T-Rex”; the next carries a longer continuation. Each payload is a JSON object containing a candidates array, and later messages carry finishReason and usageMetadata with token counts. The exact JSON differs between providers, but the shape does not: many small payloads, each prefixed by data:, delivered as the model generates them.
Two details matter when you write a parser. First, the data: prefix is framing, not payload — strip it, along with the single following space, before parsing JSON. Second, an event’s payload can be spread across multiple data: lines, so a robust parser accumulates lines until a blank line terminates the event. Some providers also send a terminal sentinel such as [DONE] instead of a JSON object. Treat a payload that is not valid JSON as a possible end marker rather than letting it crash the reader.
Chunks are not tokens
A frequent confusion is treating a “chunk” as one token or one word. A chunk is a transport-level delivery unit chosen by the provider. The Gemini example above shows a first chunk of two words and a second containing a full clause. Never build logic that assumes chunk boundaries align with sentences, words, or token counts. If you need exact token accounting, read the usage metadata the provider sends at the end of the stream, as the Chrome example shows in its usageMetadata field.
Prerequisites
- A working non-streaming request to your provider, with the API key stored in an environment variable rather than in source code.
- An HTTP client that can read a response body incrementally: an official SDK, or
curl --no-bufferfor manual checks. - If a reverse proxy, CDN, or web server sits between your client and the provider, the ability to change its buffering configuration.
Enabling streaming with an SDK
The OpenAI streaming guide shows the same pattern across languages: set the stream flag, then iterate over events as they arrive.
Python (Responses API)
from openai import OpenAI
client = OpenAI()
stream = client.responses.create(
model="your-model",
input=[
{"role": "user", "content": "Say 'double bubble bath' ten times fast."},
],
stream=True,
)
for event in stream:
print(event)
The expected result is that events print with visible gaps between them, not in a single burst after a pause.
JavaScript
import { OpenAI } from "openai";
const client = new OpenAI();
const stream = await client.responses.create({
model: "your-model",
input: [
{ role: "user", content: "Say 'double bubble bath' ten times fast." },
],
stream: true,
});
for await (const event of stream) {
console.log(event);
}
Go
The Go SDK exposes explicit iteration rather than a callback. You call client.Responses.NewStreaming(...), loop with for stream.Next(), read the current event through stream.Current().Type, and finish by checking stream.Err(). That final error check is the Go equivalent of an exception handler: a stream that ends because of a network fault and a stream that ends because the model finished can otherwise look identical.
Verify the transport before writing application code
Before wiring SSE into an application, confirm that the endpoint genuinely streams and that nothing between you and the provider is buffering. A raw request is the fastest way to find out. The Chrome for Developers guide uses this form against the Gemini API:
curl "https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:streamGenerateContent?alt=sse&key=$GOOGLE_API_KEY"
-H 'Content-Type: application/json'
--no-buffer
-d '{"contents":[{"parts":[{"text": "Tell me a long T-rex joke, please."}]}]}'
Keep the key out of shell history and logs, and mask it in anything you paste into a forum. The observable check is timing: text should appear progressively. If everything appears at once after a pause, buffering is happening somewhere — most often --no-buffer was omitted, or an intermediary is holding the response.
Forwarding a stream to a browser
If your frontend talks directly to the provider, remember that the browser’s EventSource object cannot set custom request headers, so it cannot carry an Authorization header. Two patterns work in practice.
- Server proxy. Your backend holds the API key, opens the provider stream, and re-emits
data:lines to the browser. This also gives you a place to log, meter, and redact. - fetch plus a readable stream. The browser sends a normal POST with headers and reads the response body incrementally, parsing SSE frames in your own code.
EventSource brings automatic reconnection, which is a benefit for notification feeds and a liability for generation. A reconnect mid-generation typically restarts the model call and can duplicate tokens already displayed. For chat-style output, prefer fetch-based streaming and treat reconnection as an explicit application decision.
Streaming through PHP or WordPress
WordPress’s REST API and most PHP setups buffer output by default, because the HTTP API is designed around complete responses rather than incremental ones. An illustrative design — a pattern to evaluate, not a configuration tested for this article — is a dedicated endpoint that disables output buffering, sets Content-Type: text/event-stream, writes each data: line, and calls flush() after every write, with the web server configured not to buffer that specific route. Use the same curl check against your own endpoint: if the browser shows nothing until the end, buffering is still active somewhere in the chain.
Failure modes and how to diagnose them
| Symptom | Likely cause | Diagnostic | Fix |
|---|---|---|---|
| Everything arrives at once at the end | Output buffering in the proxy, framework, or client | Run curl --no-buffer directly against the provider | Disable buffering at the layer that is holding the response |
| Stream stops after roughly a minute | Idle read timeout on a proxy or load balancer | Compare timings against the gateway’s timeout settings | Raise the read timeout and emit periodic keepalive comment lines |
| JSON parse errors partway through | Parsing the framing, or hitting a non-JSON sentinel | Log the raw line before parsing | Strip the data: prefix and handle [DONE] explicitly |
| Text appears twice after a network blip | Automatic reconnection replaying the request | Inspect your client library’s retry behavior | Disable automatic retries for generation streams |
| Token usage is unclear | Usage metadata only arrives at the end of the stream | Inspect the final event’s usage fields | Persist usage from the terminal event; check your provider’s policy for interrupted generations |
A verification checklist
- Raw
curlagainst the provider shows progressive output. - The same
curlagainst your own endpoint also shows progressive output. - Your parser strips the
data:prefix and tolerates multi-line payloads and a non-JSON sentinel. - Errors raised mid-stream surface to the user instead of silently truncating the answer.
- Usage or completion metadata from the final event is captured for accounting.
- Reconnection behavior is deliberate, not inherited from a library default.
What to do next
Start with the raw curl check, because it separates “the provider is not streaming” from “my stack is buffering” in a single step. Once that passes, move the same check one layer inward — your proxy, then your endpoint, then the browser — until the buffering layer reveals itself. If you are running local models, the same framing applies to any OpenAI-compatible server that supports stream: true, and the same buffering questions apply to whatever sits in front of it.
If you hit a case this guide does not cover — an unusual proxy, a framework that buffers in a surprising place, or a provider whose event format differs — share your environment, the raw output you see, and the error text in the community so others can reproduce it.
FAQ
Do I need EventSource to consume an AI stream in the browser?
No. EventSource cannot set custom headers such as Authorization, so it is a poor fit for direct provider calls. A fetch request with a readable response body works and lets you parse SSE frames yourself. EventSource's automatic reconnection is also undesirable for generation streams, because a reconnect typically restarts the model call.
Why does my streamed response arrive all at once?
Almost always because something is buffering the response rather than because streaming is unsupported. Test with a raw curl request using --no-buffer directly against the provider. If that streams correctly, the buffering is in your proxy, framework, or application layer.
Is each streamed chunk one token?
No. A chunk is a delivery unit chosen by the provider and can contain a few words or a longer clause. For accurate token counts, read the usage metadata that providers send in the final event of the stream.
How should I handle a stream that fails midway?
Detect the failure through your client's error path rather than treating a short response as a complete one, surface the truncation to the user, and disable automatic retries that would replay the generation. Whether an interrupted generation is still billed depends on the provider, so check its policy rather than assuming.