Streaming is a product decision, not a performance one
Streaming doesn't make the model faster. It changes what the user is doing while they wait — and that's the whole point.
A streamed response and a buffered one finish at the same time. The tokens arrive at the same rate either way. What changes is that with streaming the user starts reading at token one instead of token nine hundred.
That distinction matters more than it sounds, because it decides where you spend engineering effort.
What streaming actually buys you
The user can bail early. Half the value of a streamed answer is that three sentences in, someone realizes it's going the wrong direction and stops it — which they cannot do if the first thing they see is the finished output.
It also converts an unexplained wait into visible progress. A spinner for twelve seconds and a response that types itself over twelve seconds feel like different products, and only one of them gets accused of being broken.
What it costs
Streaming makes error handling harder in a way most implementations get wrong. Once you have flushed bytes to the client, you cannot take them back — so a failure halfway through is not a failed request you can retry, it is a half-written answer the user is already reading.
You need to decide what that looks like before you ship. Appending a plain sentence to the stream is usually better than silence, and always better than a thrown error the client renders as a blank box.
When not to stream
If the output is structured and the UI can't render it partially, streaming buys nothing and costs complexity. A JSON payload that only becomes meaningful once complete should be a normal request — show a real loading state and skip the machinery.
The question isn't "is this slow". It's "can the user do something useful with a partial answer".