text
🚀 It's now easier than ever to switch from OpenAI to open LLMs 
🚀 It's now easier than ever to switch from OpenAI to open LLMs 
new_line
new_line
text
Hugging Face's TGI now supports an OpenAI compatible Chat Completion API
Hugging Face's TGI now supports an OpenAI compatible Chat Completion API
new_line
new_line
text
This means you can transition code that uses OpenAI client libraries (or frameworks like LangChain 🦜  and LlamaIndex 🦙) to run open models by changing just two lines of code 🤗
This means you can transition code that uses OpenAI client libraries (or frameworks like LangChain 🦜  and LlamaIndex 🦙) to run open models by changing just two lines of code 🤗
new_line
new_line
text
⭐ Here's how:
⭐ Here's how:
new_line
code_fence
```
from openai import OpenAI

# initialize the client but point it to TGI
client = OpenAI(
    base_url="<ENDPOINT_URL>" + "/v1/",  # replace with your endpoint url
    api_key="<HF_API_TOKEN>",  # replace with your token
)
chat_completion = client.chat.completions.create(
    model="tgi",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Why is open-source software important?"},
    ],
    stream=True,
    max_tokens=500
)

# iterate and print stream
for message in chat_completion:
    print(message.choices[0].delta.content, end="")
```
from openai import OpenAI

# initialize the client but point it to TGI
client = OpenAI(
    base_url="<ENDPOINT_URL>" + "/v1/",  # replace with your endpoint url
    api_key="<HF_API_TOKEN>",  # replace with your token
)
chat_completion = client.chat.completions.create(
    model="tgi",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Why is open-source software important?"},
    ],
    stream=True,
    max_tokens=500
)

# iterate and print stream
for message in chat_completion:
    print(message.choices[0].delta.content, end="")
new_line
new_line
text
🔗 Blog post ➡ 
🔗 Blog post ➡ 
link
https://huggingface.co/blog/tgi-messages-api
https://huggingface.co/blog/tgi-messages-api
new_line
text
🔗 TGI docs ➡ 
🔗 TGI docs ➡ 
link
https://huggingface.co/docs/text-generation-inference/en/messages_api
https://huggingface.co/docs/text-generation-inference/en/messages_api
🚀 It's now easier than ever to switch from OpenAI to open LLMs 

Hugging Face's TGI now supports an OpenAI compatible Chat Completion API

This means you can transition code that uses OpenAI client libraries (or frameworks like LangChain 🦜  and LlamaIndex 🦙) to run open models by changing just two lines of code 🤗

⭐ Here's how:
```
from openai import OpenAI

# initialize the client but point it to TGI
client = OpenAI(
    base_url="<ENDPOINT_URL>" + "/v1/",  # replace with your endpoint url
    api_key="<HF_API_TOKEN>",  # replace with your token
)
chat_completion = client.chat.completions.create(
    model="tgi",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Why is open-source software important?"},
    ],
    stream=True,
    max_tokens=500
)

# iterate and print stream
for message in chat_completion:
    print(message.choices[0].delta.content, end="")
```

🔗 Blog post ➡ https://huggingface.co/blog/tgi-messages-api
🔗 TGI docs ➡ https://huggingface.co/docs/text-generation-inference/en/messages_api
❤️
osanseviero clem samusenps ashesashes johko Tonic julien-c victor Chunte johnnybio Alisherfullhd Yogeek Glavin001 nbroad Nymbo
🤗
samusenps hunkim kramp Tonic julien-c Chunte jeffboudier natolambert Felladrin Nymbo
👍
mvaloatto hunkim Tonic julien-c Glavin001 Nymbo
🤯
clem hunkim osanseviero Tonic julien-c
https://cdn-avatars.huggingface.co/v1/production/uploads/5dd96eb166059660ed1ee413/NQtzmrDdbG0H8qkZvRyGk.jpeg
Julien Chaumond
julien-c
user
https://cdn-avatars.huggingface.co/v1/production/uploads/61d375fd733d3a83ecd1bba9/oIXwvvs1-HaCnJXMCZgkc.jpeg
Andrew Reed
andrewrreed
user
/avatars/44a78ae88e7113ef1919de851aefddd4.svg
Odin Nøsen
myonlyeye
user