HF hosted interface API for a finetuned model with LoRA
Hi All,I need a help. When I fine-tuned a flan-t5 model with lora, I got adapter_config.json and adaptor_model.bin instead of config.json and pytorch_model.bin. I used trainer pushing to hub. And if I am understanding correctly, it seems that the hf interface API doesn’t real the adapter_config because it’s asking for config.json.How can I make this work? The model works fine generating outputs in notebook. Any idea to fix it?
2023-05-02T19:29:45Z
Got an answer. According to HF, “Inference API does not yet support adapter-transformers models for this pipeline type.”
2024-01-15T22:45:31Z
Looking at this for example, surely you can host a finetuned model using the Inference API?huggingface.cojuliensimon/llama2-7b-qlora-openassistant-guanaco · Hugging FaceWe’re on a journey to advance and democratize artificial intelligence through open source and open science.