Raw prompts and log probabilities

This page explains how to send a raw prompt without a chat template, and how to retrieve log probabilities for tokens. It is intended for developers who need exact control over the model input, such as for evaluation or scoring.

Raw promptsCopy Anchor Link

The /v1/completions endpoint takes a prompt instead of a list of messages. Engy does not apply a chat template, add a system prompt, or parse reasoning output. The model receives exactly the text or tokens that you send.

The prompt can be a string, a list of token IDs, or a list of either for a batch.

Continue a promptCopy Anchor Link

The model continues from the end of your prompt. Build the prompt with the model's own chat template, for example with tokenizer.apply_chat_template(messages, add_generation_prompt=True) from the Hugging Face Transformers library. The following prompt is the output of the glm-5.2 template for one user message:

curl https://api.engy.ai/v1/completions \
  -H "Authorization: Bearer $ENGY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2",
    "prompt": "[gMASK]<sop><|system|>Reasoning Effort: Max<|user|>Prove that 2 is irrational.<|assistant|><think>",
    "max_tokens": 512,
    "temperature": 0
  }'

The /v1/completions endpoint does not support streaming. A request with stream set to true fails with HTTP status 400.

Log probabilities of generated tokensCopy Anchor Link

On /v1/chat/completions, use the standard logprobs and top_logprobs parameters. The value of top_logprobs must be an integer from 0 to 30. If you set top_logprobs above 0, Engy turns on logprobs automatically. Log probabilities are also available in streaming responses.

On /v1/completions, logprobs is an integer from 1 to 30.

Score supplied textCopy Anchor Link

To retrieve the log probabilities of text that you supply, use /v1/completions with one of the following methods:

Score a spanCopy Anchor Link

Set logprob_start_len to the token offset where the span begins. The offset counts from the start of the prompt. To score the last n tokens of a prompt, set it to the prompt length minus n.

Send the prompt as token IDs, so that the offset matches your own tokenization, and set max_tokens to 1:

import os

from openai import OpenAI
from transformers import AutoTokenizer

client = OpenAI(
    base_url="https://api.engy.ai/v1",
    api_key=os.environ["ENGY_API_KEY"],
)
tokenizer = AutoTokenizer.from_pretrained("zai-org/GLM-5.2")

prefix = "The capital of France is"
span = " Paris."

ids = tokenizer.encode(prefix + span)
span_len = len(tokenizer.encode(span, add_special_tokens=False))

response = client.completions.create(
    model="glm-5.2",
    prompt=ids,
    max_tokens=1,
    temperature=0,
    extra_body={"logprob_start_len": len(ids) - span_len},
)

logprobs = response.choices[0].logprobs
print(sum(x for x in logprobs.token_logprobs if x is not None))

The response contains token_logprobs and token_ids, with one entry for each scored token. It does not contain text_offset, so align the results by token_ids.

The limit of 1,024 tokens applies to the scored span, not to the prefix. The prefix can be as long as the context window allows. A span of more than 1,024 tokens fails with HTTP status 400. Split longer spans across several requests.

If you send the prompt as a string, Engy estimates its length instead of counting tokens. A string prompt whose estimated length exceeds the limit fails, even if the span is short.

Score the whole promptCopy Anchor Link

Set echo to true and logprobs to at least 1:

curl https://api.engy.ai/v1/completions \
  -H "Authorization: Bearer $ENGY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "glm-5.2",
    "prompt": "The capital of France is",
    "max_tokens": 1,
    "temperature": 0,
    "echo": true,
    "logprobs": 1
  }'

The limit of 1,024 tokens applies to the whole prompt. For a longer prompt, use logprob_start_len instead. On this path, text_offset is -1 for every token.

Best practicesCopy Anchor Link

• Always set max_tokens to 1 for scoring requests. If you omit it, the model generates up to its maximum output length, and you are charged for that output.
• Do not set max_tokens or logprobs to 0. Both values fail with HTTP status 400.
• Include logprobs when you use echo. Without it, the request succeeds but returns no log probabilities.
• Check that the number of returned log probabilities matches the number of tokens that you intended to score. An incorrect logprob_start_len scores a different span without an error.
• Log probabilities can differ slightly between servers that serve the same model. If you plan to compare log probabilities across many requests, contact support first.