Spaces:
Running
Running
Can you use the HuggingFace serverless Inference API in a chat frontend such as librechat without setting anything else up?
#437
by
SquigglyFruit
- opened
To set this up you need a chat completions address.
Reading: https://huggingface.co/docs/api-inference/quicktour
eg. /static-proxy?url=https%3A%2F%2Fapi-inference.huggingface.co%2Fmodels%2Fmeta-llama%2FMeta-Llama-3-8B%3C%2Fa%3E%3C%2Fp%3E
(or is it /static-proxy?url=https%3A%2F%2Fapi-inference.huggingface.co%2Fmodels%2FMeta-Llama-3-8B%3C%2Fa%3E )? and use a header of Authorization : Bearer {api key} eg using model: "meta-llama/Meta-Llama-3-8B" I've tried this in TypingMind with errors. Is the API is OpenAI compatible?
Any help would be much appreciated.