How do I deploy my trained LORA adapter to a ChatUI in huggingface?
Here’s my trained LORA adapter on Mistral8x7b:dominic5/Project5_V3_Mistral8x7b_V2.2.5 · Hugging Face. How do I deploy it to a huggingchatUI interface such that it looks like this?Zephyr Gemma Chat - a Hugging Face Space by HuggingFaceH4.I’m asking since the tutorials and guides I’ve seen so far are only for models that aren’t adapters.Thank you.
2024-03-02T18:40:22Z
Hi,ChatUI works with a TGI backend, hence you can follow this threadcurious about the plans for supporting PEFT and LoRa. · Issue #482 · huggingface/text-generation-inference · GitHub.See alsoGitHub - huggingface/chat-ui: Open source codebase powering the HuggingChat app
2024-05-08T19:14:12Z
Update, vLLM now supports LoRa serving:Using LoRA adapters — vLLM. vLLM is a framework similar to TGI.
2024-06-28T10:46:00Z
Update, TGI now supports multi-LoRa inference:Enable multiple LoRa adapters by drbh · Pull Request #2010 · huggingface/text-generation-inference · GitHub