The Vision feature is available when the model supports vision capabilities.
Friendli is equipped with a new Vision feature that can understand and analyze images, opening up exciting possibilities for multi-modal interactions. This guide explains how to work with images in Friendli, including best practices and code examples.

​
How to Use Vision

Utilize Friendli’s Vision features through the following:
  • Select and test a vision model at friendli.ai/playground.
  • Use the API to process images and receive the model’s responses, referring to the methods described in this document.

​
Supported Image Formats

Supports formats supported by the PIL library, including jpg, png and avif.
  • JPEG (.jpeg and .jpg)
  • PNG (.png)
  • AVIF (.avif)

​
Using the API

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.friendli.ai/dedicated/v1",
    api_key=os.environ.get("FRIENDLI_TOKEN"),
)

image_url = "https://upload.wikimedia.org/wikipedia/commons/9/9e/Ours_brun_parcanimalierpyrenees_1.jpg"

completion = client.chat.completions.create(
    # Replace YOUR_ENDPOINT_ID with the ID of your endpoint, e.g. "zbimjgovmlcb"
    model="YOUR_ENDPOINT_ID",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "What kind of animal is shown in the image?",
                },
                {"type": "image_url", "image_url": {"url": image_url}},
            ],
        },
    ],
    stream=False
)

print(completion.choices[0].message.content)