text
The new NIM Serverless API by HF and Nvidia is a great option if you want a reliable API for open-weight LLMs like Llama-3.1-405B that are too expensive to run on your own hardware.
The new NIM Serverless API by HF and Nvidia is a great option if you want a reliable API for open-weight LLMs like Llama-3.1-405B that are too expensive to run on your own hardware.
new_line
new_line
text
- It's pay-as-you-go, so it doesn't have rate limits like the standard HF Serverless API and you don't need to commit to hardware like for a dedicated endpoint.
- It's pay-as-you-go, so it doesn't have rate limits like the standard HF Serverless API and you don't need to commit to hardware like for a dedicated endpoint.
new_line
text
- It works out-of-the box with the new v0.25 release of our huggingface_hub.InferenceClient
- It works out-of-the box with the new v0.25 release of our huggingface_hub.InferenceClient
new_line
text
- It's specifically tailored to a small collection of popular open-weight models. For a broader selection of open models, we recommend using the standard HF Serverless API.
- It's specifically tailored to a small collection of popular open-weight models. For a broader selection of open models, we recommend using the standard HF Serverless API.
new_line
text
- Note that you need a token from an Enterprise Hub organization to use it.
- Note that you need a token from an Enterprise Hub organization to use it.
new_line
new_line
text
Details in this blog post: 
Details in this blog post: 
link
https://huggingface.co/blog/inference-dgx-cloud
https://huggingface.co/blog/inference-dgx-cloud
new_line
text
Compatible models in this HF collection: 
Compatible models in this HF collection: 
resource
https://huggingface.co/collections/nvidia/nim-serverless-inference-api-66a3c6fcdcb5bbc6e975b508
collection
nvidia/nim-serverless-inference-api-66a3c6fcdcb5bbc6e975b508
https://huggingface.co/collections/nvidia/nim-serverless-inference-api-66a3c6fcdcb5bbc6e975b508
new_line
text
Release notes with many more features of huggingface_hub==0.25.0: 
Release notes with many more features of huggingface_hub==0.25.0: 
link
https://github.com/huggingface/huggingface_hub/releases/tag/v0.25.0
https://github.com/huggingface/huggingface_hub/releases/tag/v0.25.0
new_line
new_line
text
Copy-pasteable code in the first comment: 
Copy-pasteable code in the first comment: 
The new NIM Serverless API by HF and Nvidia is a great option if you want a reliable API for open-weight LLMs like Llama-3.1-405B that are too expensive to run on your own hardware.

- It's pay-as-you-go, so it doesn't have rate limits like the standard HF Serverless API and you don't need to commit to hardware like for a dedicated endpoint.
- It works out-of-the box with the new v0.25 release of our huggingface_hub.InferenceClient
- It's specifically tailored to a small collection of popular open-weight models. For a broader selection of open models, we recommend using the standard HF Serverless API.
- Note that you need a token from an Enterprise Hub organization to use it.

Details in this blog post: https://huggingface.co/blog/inference-dgx-cloud
Compatible models in this HF collection: https://huggingface.co/collections/nvidia/nim-serverless-inference-api-66a3c6fcdcb5bbc6e975b508
Release notes with many more features of huggingface_hub==0.25.0: https://github.com/huggingface/huggingface_hub/releases/tag/v0.25.0

Copy-pasteable code in the first comment: 
👀
John6666 davanstrien nbroad
https://cdn-avatars.huggingface.co/v1/production/uploads/1613511937628-5fb15d1e84389b139cf3b508.jpeg
Moritz Laurer
MoritzLaurer
user
https://cdn-avatars.huggingface.co/v1/production/uploads/1627505688463-60107b385ac3e86b3ea4fc34.jpeg
Daniel van Strien
davanstrien
user