Serverless chat completions chunk object

Represents a streamed chunk of a chat completions response returned by model, based on the provided input.

Response

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "meta-llama-3.1-8b-instruct",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": { "role": "assistant", "content": "This" },
      "finish_reason": null,
      "logprobs": null
    }
  ],
  "usage": null,
  "created": 1726294381
}

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "meta-llama-3.1-8b-instruct",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": { "content": " is" },
      "finish_reason": null,
      "logprobs": null
    }
  ],
  "usage": null,
  "created": 1726294381
}

...

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "meta-llama-3.1-8b-instruct",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": {},
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "usage": null,
  "created": 1726294383
}

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "meta-llama-3.1-8b-instruct",
  "object": "chat.completion.chunk",
  "choices": [],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 4,
    "total_tokens": 12
  },
  "created": 1726294402
}

data: [DONE]

id · string · required

A unique ID of the chat completion.

object · string · required

The object type, which is always set to chat.completion.chunk.

model · string · required

The model to generate the completion.

choices · object[] · required

Show child attributes

index · integer · required

The index of the choice in the list of generated choices.

delta · object · required

Show child attributes

role · string | null

Role of the generated message author, in this case assistant.

content · string | null

The contents of the assistant message.

tool_calls · object[] | null

Show child attributes

index · integer · required

The index of tool call being generated.

id · string | null

The ID of the tool call.

type · string · required

The type of the tool, which is always set to function.

function · object · required

Show child attributes

name · string | null

The name of the function to call.

arguments · string · required

The arguments for calling the function, generated by the model in JSON format. Ensure to validate these arguments in your code before invoking the function since the model may not always produce valid JSON.

finish_reason · enum<string> | null

Termination condition of the generation. stop means the API returned the full chat completions generated by the model without running into any limits. length means the generation exceeded max_tokens or the conversation exceeded the max context length. tool_calls means the API has generated tool calls.

Available options: stop, length, tool_calls

logprobs · object | null

Log probability information for the choice.

Show child attributes

content · object[] | null

A list of message content tokens with log probability information.

Show child attributes

token · string · required

The token.

logprob · number · required

The log probability of this token.

bytes · integer[] | null

A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token.

top_logprobs · object[] · required

List of the most likely tokens and their log probability, at this token position.

Show child attributes

token · string · required

The token.

logprob · number · required

The log probability of this token.

bytes · integer[] | null

A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token.

usage · object | null

Show child attributes

prompt_tokens · integer · required

Number of tokens in the prompt.

completion_tokens · integer · required

Number of tokens in the generated chat completions.

total_tokens · integer · required

Total number of tokens used in the request (prompt_tokens + completion_tokens).

created · integer · required

The Unix timestamp (in seconds) for when the token sampled.