Serverless tool assisted chat completions chunk object

Represents a streamed chunk of a tool assisted chat completions response returned by model, based on the provided input.

This API is currently in Beta. While we strive to provide a stable and reliable experience, this feature is still under active development. As a result, you may encounter unexpected behavior or limitations. We encourage you to provide feedback to help us improve the feature before its official release.

• Feature request & feedback
• Contact support

Response

event: tool_status
data: {
  "tool_call_id": "call_3QrfStXSU6fGdOGPcETocIAq",
  "name": "math:calculator",
  "status": "STARTED",
  "parameters": [{ "name": "expression", "value": "150 * 1.60934" }],
  "result": null,
  "files": null,
  "message": null,
  "error": null,
  "usage": null,
  "timestamp": 1726277121
}

event: tool_status
data: {
  "tool_call_id": "call_3QrfStXSU6fGdOGPcETocIAq",
  "name": "math:calculator",
  "status": "ENDED",
  "parameters": [{ "name": "expression", "value": "150 * 1.60934" }],
  "result": "\"{\\\"result\\\": \\\"150 * 1.60934=241.401000000000\\\"}\"",
  "files": null,
  "message": null,
  "error": null,
  "usage": null,
  "timestamp": 1726277121
}

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "meta-llama-3.1-8b-instruct",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": { "role": "assistant", "content": "To" },
      "finish_reason": null,
      "logprobs": null
    }
  ],
  "created": 1726277121
}

...

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "meta-llama-3.1-8b-instruct",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": { "role": "assistant", "content": "." },
      "finish_reason": null,
      "logprobs": null
    }
  ],
  "created": 1726277121
}

data: {
  "id": "chatcmpl-4b71d12c86d94e719c7e3984a7bb7941",
  "model": "meta-llama-3.1-8b-instruct",
  "object": "chat.completion.chunk",
  "choices": [
    {
      "index": 0,
      "delta": {},
      "finish_reason": "stop",
      "logprobs": null
    }
  ],
  "created": 1726277121
}

data: [DONE]

id · string · required

A unique ID of the chat completion.

object · string · required

The object type, which is always set to chat.completion.chunk.

model · string · required

The model to generate the completion.

choices · object[] · required

Show child attributes

index · integer · required

The index of the choice in the list of generated choices.

delta · object · required

Show child attributes

role · string | null

Role of the generated message author, in this case assistant.

content · string | null

The contents of the assistant message.

tool_calls · object[] | null

Show child attributes

index · integer · required

The index of tool call being generated.

id · string | null

The ID of the tool call.

type · string · required

The type of the tool, which is always set to function.

function · object · required

Show child attributes

name · string | null

The name of the function to call.

arguments · string · required

The arguments for calling the function, generated by the model in JSON format. Ensure to validate these arguments in your code before invoking the function since the model may not always produce valid JSON.

finish_reason · enum<string> | null

Termination condition of the generation. stop means the API returned the full chat completions generated by the model without running into any limits. length means the generation exceeded max_tokens or the conversation exceeded the max context length. tool_calls means the API has generated tool calls.

Available options: stop, length, tool_calls

logprobs · object | null

Log probability information for the choice.

Show child attributes

content · object[] | null

A list of message content tokens with log probability information.

Show child attributes

token · string · required

The token.

logprob · number · required

The log probability of this token.

bytes · integer[] | null

A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token.

top_logprobs · object[] · required

List of the most likely tokens and their log probability, at this token position.

Show child attributes

token · string · required

The token.

logprob · number · required

The log probability of this token.

bytes · integer[] | null

A list of integers representing the UTF-8 bytes representation of the token. Useful in instances where characters are represented by multiple tokens and their byte representations must be combined to generate the correct text representation. Can be null if there is no bytes representation for the token.

usage · object | null

Show child attributes

prompt_tokens · integer · required

Number of tokens in the prompt.

completion_tokens · integer · required

Number of tokens in the generated chat completions.

total_tokens · integer · required

Total number of tokens used in the request (prompt_tokens + completion_tokens).

created · integer · required

The Unix timestamp (in seconds) for when the token sampled.

event: tool_status chunk object

event: tool_status tracks the execution progress of built-in tools, such as calculator or web search functions. It provides real-time updates on their status and results.

tool_call_id · string · required

The ID of the tool call.

name · enum<string> · required

The name of the built-in tool.

Available options: math:calculator, math:statistics, math:calendar, web:search, web:url, code:python-interpreter, file:text

status · enum<string> · required

Indicates the current execution status of the tool.

Available options: STARTED, UPDATING, ENDED, ERRORED

parameters · object[] · required

Show child attributes

name · string · required

The name of the tool’s function parameter.

value · string · required

The value of the tool’s function parameter.

result · string | null

The output from the tool’s execution.

files · object[] | null

Show child attributes

name · string · required

The name of the file generated by the tool’s execution.

url · string · required

URL of the file generated by the tool’s execution.

message · string | null

Message generated by the tool’s execution.

error · object | null

Show child attributes

type · string · required

The type of error encountered during the tool’s execution.

msg · string · required

The message of error.

timestamp · number · required

The Unix timestamp (in seconds) for when the event occurred.