PEFT + Inference
Is there a way to get PEFT to work with inference endpoints?Ideally, we should be able to support multiple PEFT models with a common inference endpoint for the base model.
2023-08-01T11:22:55Z
any updates here?
2023-08-15T09:44:18Z
You could configure a custom handler that allows you to specify code to load the model and its adaptersCreate custom Inference Handler
2024-01-15T09:19:22Z
How can we use it text-generation-inference? I assume that just loading it through a custom handler will serve it via transformers, no?