Streaming

This page explains how to stream responses from the Engy API, what the stream contains, and how errors and billing work for streamed requests. It is intended for developers who display output as the model generates it.

Enable streamingCopy Anchor Link

Set stream to true in the request body. Engy returns the response as server-sent events (text/event-stream).

curl https://api.engy.ai/v1/chat/completions \
  -H "Authorization: Bearer $ENGY_API_KEY" \
  -H "Content-Type: application/json" \
  -N \
  -d '{
    "model": "glm-5.2",
    "stream": true,
    "messages": [{"role": "user", "content": "Write a haiku about the sea."}]
  }'

Streaming is available on /v1/chat/completions, /v1/messages, and /v1/responses. The /v1/completions endpoint does not stream. A request to it with stream set to true fails with HTTP status 400.

Chat completions stream formatCopy Anchor Link

Each event is a data: line that contains a JSON chunk. The stream proceeds as follows:

1. The first chunk contains {"role": "assistant"} in delta.
2. Content chunks contain the generated text in delta.content. For reasoning models, they also contain delta.reasoning_content. For details, see Reasoning.
3. The final chunk contains finish_reason, the usage object, and the x_engy object.
4. The stream ends with data: [DONE].

Engy can merge several tokens into one chunk. Do not assume that each chunk contains exactly one token.

Usage in the streamCopy Anchor Link

The final chunk always contains usage information, even if you do not request it:

{
  "choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
  "usage": {
    "prompt_tokens": 14,
    "completion_tokens": 18,
    "total_tokens": 32,
    "prompt_tokens_details": {"cached_tokens": 0},
    "completion_tokens_details": {"reasoning_tokens": 0},
    "cost": 0.000037
  },
  "x_engy": {"request_id": "...", "charged_micro": 37}
}

If you set stream_options.include_usage to true, Engy also sends the standard OpenAI usage chunk, with an empty choices array, before [DONE].

Keepalive commentsCopy Anchor Link

If the model produces no output for 15 seconds, for example while it processes a long prompt, Engy sends an SSE comment line (: keepalive). Comment lines keep the connection open. Standard SSE clients ignore them.

Other stream formatsCopy Anchor Link

On /v1/messages, reasoning is streamed as thinking content blocks unless thinking is disabled. On /v1/responses, reasoning is not returned.

Errors during a streamCopy Anchor Link

If an error occurs after Engy has started the stream, the HTTP status is already 200. Engy reports the error inside the stream instead:

Errors that occur before generation starts, such as an insufficient balance or no available capacity, can also arrive this way when stream is true.

To detect a failed stream on /v1/chat/completions, check each chunk for an error field, and treat a stream that ends without [DONE] as incomplete. The error object has the same format as other errors. For details, see Errors and limits.

import json
import os

import httpx

with httpx.stream(
    "POST",
    "https://api.engy.ai/v1/chat/completions",
    headers={"Authorization": f"Bearer {os.environ['ENGY_API_KEY']}"},
    json={
        "model": "glm-5.2",
        "stream": True,
        "messages": [{"role": "user", "content": "Hello"}],
    },
    timeout=600,
) as response:
    completed = False
    for line in response.iter_lines():
        if not line.startswith("data: "):
            continue
        data = line.removeprefix("data: ")
        if data == "[DONE]":
            completed = True
            break
        chunk = json.loads(data)
        if "error" in chunk:
            raise RuntimeError(chunk["error"]["message"])
        for choice in chunk.get("choices", []):
            print(choice["delta"].get("content") or "", end="", flush=True)

if not completed:
    raise RuntimeError("The stream ended before it completed.")

Billing for streamed requestsCopy Anchor Link

• When the stream completes, Engy charges for the tokens that the model generated.
• If the request fails, Engy does not charge for it.
• If you close the connection before the stream completes, Engy stops generation and does not charge for the request.

LimitationsCopy Anchor Link

• Streamed chunks always use choice index 0. Do not use n greater than 1 with streaming.