How to Stream AI API Responses Without Blocking Your UI

Setting stream: true on an AI API request is only half of what makes an interface feel fast. The model’s answer arrives as a sequence of server-sent events (SSE), and the page only benefits if those events are read incrementally, batched into state updates, and painted without blocking the main thread. This guide covers the request side, the consumption side, and the infrastructure in between — including the WordPress and local-server cases where streaming most often breaks silently.

What streaming actually changes

By default, a request to the OpenAI API generates the model’s entire output before it is returned in a single HTTP response. Streaming responses let the client start printing or processing the beginning of the model’s output while the rest is still being generated [1].

The transport for HTTP streaming is server-sent events, enabled by setting stream=true on the Responses endpoint [1]. The server holds the connection open and writes a series of small events; the client reads them as they arrive rather than waiting for a complete body.

Two consequences follow from that. First, the first useful text can appear in a fraction of the time it would take to generate the whole answer. Second, the client is now responsible for reassembling a stream that can be split at any byte boundary, and for handling a connection that can fail midway.

Blocking versus streaming interfaces

The user-experience argument is well documented. A blocking interface waits until the full response is available before displaying anything; for long generations, users “might easily find themselves staring at loading spinners for 5, 10, even up to 40s” [2]. A streaming interface transmits parts of the response as they become available, so the reader can begin consuming the answer immediately [2].

The same source is explicit that streaming is not automatically the right choice: if the desired functionality can be achieved with a smaller, faster model, skipping streaming “can often lead to simpler and more manageable development processes” [2]. Streaming adds state, cancellation, and error-recovery complexity. It earns that complexity when outputs are long, when perceived latency matters, or when a human is reading along.

Enabling streaming on the request

With the official SDKs, enabling streaming is a single flag. The JavaScript client returns an async iterable of events:

import { OpenAI } from "openai";

const client = new OpenAI();

const stream = await client.responses.create({
  model: "gpt-6-astra",
  input: [
    { role: "user", content: "Say 'double bubble bath' ten times fast." },
  ],
  stream: true,
});

for await (const event of stream) {
  console.log(event);
}

The Python client uses the same stream=True parameter and yields events from a plain for loop. The Go client exposes a NewStreaming call with Next() and Current() methods, and the Java client follows the same shape [1]. The model identifier in the documentation example is illustrative; substitute whatever model your account is provisioned for.

Whichever SDK you use, the important detail is that the loop body runs once per event, not once per completed answer. Everything you do inside that loop — parsing, state updates, DOM writes — happens many times per response.

Reading the stream without freezing the page

Parse SSE correctly, including split chunks

If you are not using an SDK, you are reading a text/event-stream body yourself. The classic bug is assuming each network chunk contains whole events. It does not. Keep a text buffer, split on newlines, and retain the trailing partial line:

const controller = new AbortController();

const res = await fetch("/api/chat", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ prompt: userInput }),
  signal: controller.signal,
});

const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

while (true) {
  const { value, done } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split("n");
  buffer = lines.pop();

  for (const line of lines) {
    if (!line.startsWith("data:")) continue;
    const payload = line.slice(5).trim();
    if (payload === "[DONE]") continue;
    handleEvent(JSON.parse(payload));
  }
}

Pass { stream: true } to TextDecoder.decode so that multi-byte characters split across chunk boundaries are not corrupted into replacement characters.

Batch updates instead of repainting per token

Events arrive far faster than a display refreshes. Calling a full component re-render for every event wastes work and can make the interface feel worse than a blocking one, especially if each update re-parses Markdown or re-measures layout.

A reliable pattern is to accumulate incoming text into a buffer and flush it on a timer or on requestAnimationFrame — typically every 30–60 ms. The visual result is close to indistinguishable from per-token rendering, but the number of layout and paint passes drops by an order of magnitude. If the answer contains Markdown or syntax-highlighted code, render the partial text as plain text and switch to full formatting only when the stream completes.

Keep a cancel path

Streaming makes long generations tolerable, which means users will start them more often and abandon them more often. Wire an AbortController to the request and abort on navigation, on a new submission, or on an explicit stop button. Without it, an abandoned stream keeps consuming tokens and server time.

Partial output, errors, and retries

Streaming changes what “failure” looks like. A connection can drop after most of an answer has already been rendered. At that point you already have partial text on screen, and the clean options are limited: mark the answer as incomplete, or restart the generation and accept a visible discontinuity. Decide this policy before shipping rather than after the first incident.

Structured output is a special case. A JSON object streamed incrementally is not valid JSON until the final event arrives, so parsing must be deferred or done with an incremental parser. Safety and moderation checks that operate on the complete response also cannot gate the first tokens — if that matters for your application, streaming is the wrong transport for that particular request.

Infrastructure that silently breaks streaming

Most “streaming works locally but not in production” reports trace back to something between the model and the browser holding bytes until it has enough of them. Common culprits:

• Reverse-proxy buffering. Nginx buffers proxied responses by default. Disabling proxy_buffering for the endpoint, or sending X-Accel-Buffering: no, restores pass-through behaviour.
• Compression middleware. Gzip and Brotli buffers can defeat incremental delivery; exclude the SSE route.
• Serverless timeouts. A function with a short execution limit will terminate a long stream mid-answer.
• Idle timeouts on load balancers. A model that pauses before the first token can trip a short idle timeout even though the connection is healthy.

A quick diagnostic is to request the endpoint with curl -N from the server itself and again through the public URL. If the local call prints progressively and the public one arrives in a single burst, the buffering is in your infrastructure, not your code.

Streaming from WordPress and PHP

WordPress can serve an SSE endpoint, but PHP’s defaults work against you. A minimal REST route needs to send Content-Type: text/event-stream and Cache-Control: no-cache, then write each event and call flush() (after ob_flush() if an output buffer is active). Output buffering must be disabled for the route, and the execution time limit raised for long generations.

On the server side, PHP-FPM’s fastcgi_buffering and Nginx’s proxy_buffering both need to be off for the endpoint. Many managed WordPress hosts buffer at the edge in ways you cannot configure; test before building a feature on top of it. If the host will not pass through incremental output, a small separate service for the streaming route is usually more reliable than fighting the stack.

Local model servers

Local inference servers that expose an OpenAI-compatible API generally follow the same SSE pattern, and several stream by default. The event shapes are not guaranteed to be identical to a hosted provider’s, so verify against the specific server’s documentation and confirm with curl -N that tokens appear incrementally. The client-side advice in this article — buffered parsing, batched rendering, cancellation — applies unchanged.

A verification checklist

1. Confirm the request sets stream: true and that the client library returns an iterable rather than a single object [1].
2. Log the arrival timestamp of the first and last event. A large gap means buffering upstream.
3. Test with curl -N both locally and through the public URL.
4. Instrument the render path: count state updates per response. If it equals the event count, add batching.
5. Trigger a cancel mid-stream and confirm the request actually aborts.
6. Simulate a dropped connection and confirm the UI marks the answer as incomplete.

Streaming is a transport decision with a UI consequence. The API change is one flag; the interface change is a small amount of buffering, batching, and cancellation logic. Get those three right and the perceived speed of a long generation improves dramatically. Get them wrong and you have added complexity without the benefit. If you hit a case where events arrive in one burst, share your stack — host, proxy, and runtime — in the community so others can compare notes.