Friendli Dedicated Endpoints
Frequently Asked Questions and Troubleshooting
While following through our tutorials, you might have had questions regarding the details of the requirements and specifications. We have listed out the frequently asked questions and as a separate document.
While following through our tutorials, you might have had questions regarding the details of the requirements and specifications.
We have listed out the frequently asked questions and as a separate document. Please refer to the relevant information below:
Format Requirements
General requirements for a model
- A model should be in safetensors format.
- The model should NOT be nested inside another directory.
- Including other arbitrary files (that are not in the list) is totally fine. However, those files will not be downloaded nor used.
| Required | Filename | Description |
|---|---|---|
| Yes | safetensors | Model weight, e.g. model.safetensors. Use model.safetensors.index.json for split safetensors files |
| Yes | config.json | Model config that includes the architecture. (Supported Models on Friendli) |
| No | tokenizer.json | Tokenizer for the model |
| No | tokenizer_config.json | Tokenizer config. This should be present & have a chat_template field for the Friendli Engine to provide chat APIs |
| No | special_tokens_map.json |
General requirements for a dataset
- Read our documentation on the fine-tuning dataset format for information on the dataset requirements.
3rd-party account integration
How to integrate a Hugging Face account
- Log in to Hugging Face, then navigate to user settings → access tokens → User Access Tokens. Acquire a token.
- You may use a fine-grained token. In this case, please make sure the token has view permission for the repository you’d like to use.
- Integrate the key in Friendli Suite → Personal settings → Account → Integrations
If you revoke / invalidate the key, you will have to update the key in order to not disrupt ongoing deployments, or to launch a new inference deployment / fine-tuning job.
How to integrate a W&B account
- Log in to W&B, then navigate to user settings → danger zone → API keys. Acquire a token.
- Integrate the key in Friendli Suite → Personal settings → Account → Integrations
If you revoke / invalidate the key, you will have to update the key in order to not disrupt ongoing deployments, or to launch a new inference deployment / fine-tuning job.
Extra: How to upload a safetensors format model to W&B using W&B CLI
- Install the CLI and log in using the API key → Command Line Interface | Weights & Biases Documentation
- Upload the model as an W&B artifact using the command below:
- With all this, the W&B artifact will look like this:
Using 3rd-party model
How to use a W&B artifact as a model
- Use the full name of the artifact
- The artifact name must be in the format of:
org/project/artifact_id:version
How to use a Hugging Face repository as a model
- Use the repository id of the model. You may select the entry from the list of autocompleted model repositories.
- You may choose specific branch, or manually enter a commit hash.
Using W&B with Dedicated Fine-tuning
- When launching a fine-tuning job, you can designate a W&B project that the metrics will be exported to. If you provide a W&B project name that already exists, your job will be added to that project. Otherwise, a new W&B project will be automatically created in your integrated W&B account. If the project name is not provided, it defaults to ‘friendliai’.
- As the training starts, you will be able to see a new ‘Run’ in the project you chose.
- By clicking the project, you can easily track & monitor the status of the training job.
If new runs are not displayed in your project, please check that the default team is set correctly on W&B user settings.
Troubleshooting
Can’t access the artifact
- The artifact might be nonexistent, or hidden so that you cannot access it.
You don’t have access to this gated model
- The repository is gated. Please follow the steps and gain approval from the owner using Hugging Face Hub.
The repository / artifact is invalid
- The model does not meet the requirements. Please check if the model follows a correct safetensors format.
The architecture is not supported
- The model architecture is not supported. Please refer to Supported Models on Friendli.