Hugging Face's logo

Spaces:
as-cle-bert
/
Llama-3.1-405B-FP8
Running

App Files Community
1

why

#1
by YaserDS-777 - opened
Discussion
YaserDS-777

client = OpenAI(
base_url="/static-proxy?url=https%3A%2F%2Fapi-inference.huggingface.co%2Fv1%2F%3C%2Fa%3E",
api_key=TOKEN,
)
i cant understand what is use of above object? is it necessary add it

as-cle-bert
Owner

HI! You can check out reference code here: https://github.com/huggingface/huggingface-llama-recipes/blob/main/inference-api.ipynb
My guess is that this piece of code is there because OpenAI client is more easy to use than the HF Client itself for Llama 3.1 :)

Sign up or log in to comment