Deploy from W&B Registry with Webhook

Hands-on tutorial for launching and deploying LLMs using Friendli Dedicated Endpoints with Weights & Biases artifacts through webhook automation.

Introduction

This tutorial is designed to guide you through the process of easily deploying your models from the W&B Registry to Friendli Dedicated Endpoints in the W&B UI. Through a series of step-by-step instructions and hands-on examples, you’ll learn how to:

• Configure a webhook in W&B to trigger deployments to Friendli Dedicated Endpoints.
• Create a webhook automation to automatically deploy model artifacts when adding new versions.
• Deploy a model artifact to Friendli Dedicated Endpoints by adding an alias in the W&B Registry.
• Understand how adding and removing aliases affects deployments on Friendli Dedicated Endpoints.

Why use W&B webhook automation with Friendli Dedicated Endpoints?

W&B users often rely on W&B Registry to manage the lifecycle of models – from tracking experiment artifacts to promoting the best-performing models for production use. As a W&B user, integrating Friendli Dedicated Endpoints directly into this workflow allows you to:

• Streamline deployment: Transition your models from experimentation to production with minimal effort. By leveraging W&B’s aliasing system and FriendliAI’s automated infrastructure, you eliminate the need for custom scripts or manual configurations.
• Ensure deployment consistency: Friendli Dedicated Endpoints include support for idempotencyKey to ensure the reliability of automated workflows. Each deployment trigger via webhook automation is tracked with a unique idempotencyKey, ensuring that operations like endpoint creation or updates are processed exactly once. It prevents duplicate or conflicting operations, giving you confidence in the consistency of your deployment.

By the end of this tutorial, you’ll be equipped with the knowledge and skills necessary to seamlessly transfer your models from W&B Registry to Friendli Dedicated Endpoints for efficient deployment. So, let’s get started and explore the possibilities of Friendli Dedicated Endpoints!

Prerequisites

• A Friendli Suite account with access to Friendli Dedicated Endpoints.
• A personal access token generated through Friendli Suite.

Step 1: Create a secret

1. Navigate to the team’s page on W&B and click on Team settings.
2. Scroll down to the Team secrets section and click New secret.
3. Go to Friendli Suite and navigate to Personal settings > Tokens and click Create new token.
4. Copy your personal access token.
5. Return to W&B and fill in the Secret with the personal access token generated through Friendli Suite.

Step 2: Configure a webhook

1. From the same W&B team settings page, click on New webhook in the Webhooks section.
2. Fill in the URL field with Friendli Suite Rest API URL (see more details here) and Access token field with the secret already created through Friendli Suite.

Step 3: Create a webhook automation

1. Go to your W&B Registry Model and click on View details of the model you want to deploy.
2. Click on Create automation in the Automations section.

3. Select An artifact alias is added for the Event.
4. Enter an alias you want to use to trigger the deployment for the Alias regex.

5. Select the Webhooks for Action type.
6. Select the webhook configured with Friendli Dedicated Endpoints for Webhook.
7. Fill out the box by referring to the following example for Payload.

Example: Configuration for payload

Required fields

{
  "wandbArtifactVersionName": "${artifact_version_string}"
}

Field	Description
wandbArtifactVersionName	Specific model artifact version from W&B.

View more details about each field here.

Step 4: Deploy a model artifact

Deploy your model artifact to Friendli Dedicated Endpoints by simply adding the alias set in Step 3 to a model artifact version!

After adding the alias, you can see the endpoint created in Friendli Dedicated Endpoints.

Step 5: Roll out a model artifact (Advanced)

To roll out an endpoint to a new model artifact version, simply add the same alias to the new version you want to deploy. This updates the endpoint to use the new model artifact version. After assigning the alias, the endpoint will update to reflect the new version in Friendli Dedicated Endpoints.

Change model artifact from Version 0 to Version 1

An idempotencyKey is required to roll out an endpoint between different model artifact versions.

{
  "wandbArtifactVersionName": "${artifact_version_string}",
  "name": "Generated from WandB ${project_name}/${artifact_collection_name}",
  "accelerator": {
    "type": "NVIDIA H100",
    "count": 1
  },
  "projectId": "project-id",
  "idempotencyKey": "${alias}",
  "autoscalingPolicy": {
    "minReplica": 0,
    "maxReplica": 2,
    "cooldownPeriod": 300
  }
}

Step 6: Track the history of deployment versions

Use the Friendli Dedicated Endpoints versioning feature to track the history of your model deployments and maintain a clear record of every update. By adding an alias to a model artifact version, you can deploy models and roll out updates across versions efficiently, without needing to create a new endpoint from scratch.

• When an alias is reassigned to a different version, the existing endpoint will automatically roll out to the new version.

Versions

In the diagram,

• v0 represents the first deployed version of the model when the endpoint was created.
• v1 is a newer model artifact version that the alias was reassigned to, triggering a rollout to update the endpoint accordingly.

View more details about the versioning feature here.

Frequently Asked Questions

What happens if multiple aliases are added to a single model artifact version?

The model artifact version will be deployed as the number of aliases added. Within a model collection, only one artifact version can have a given alias at any time. Therefore, adding an alias to a new artifact version will automatically remove it from the previously aliased version with the same alias. One webhook automation is assigned to one Friendli Dedicated Endpoint.

Will removing an alias added to a model artifact version affect the endpoints deployed with that version?

Nothing happens to the endpoint. Removing an alias will not delete the endpoint. However, if you add the removed alias to a new model artifact version, the deployed endpoint will roll out to that version.

What happens if I move an alias to another model version?

If an idempotencyKey is included in the payload, moving an alias to a different model artifact version will reassign the created endpoint to the new version within the same project.

What happens when I add an alias to a model artifact for the first time?

When adding an alias to a model artifact version for the first time, an endpoint will be created in either an existing or a new project within your default team of Friendli Suite. If projectId is specified, the endpoint will be made in an existing project. Otherwise, a new project will be created.

Feedback or issue

If you have any feedback or issues about the integration with Friendli Dedicated Endpoints, please ask for support by sending an email to Support.