Request: Add vLLM Support for This Model

#12
by kira - opened

Hi team!

Thank you for creating and sharing this amazing model with the community.

I'm interested in using this model with vLLM for improved inference performance and efficiency. Would it be possible to add official vLLM support for this model? This would help users leverage vLLM's continuous batching and other optimizations.

If you're open to this, I'd be happy to help test or provide any additional information needed. Looking forward to your thoughts on this integration.

thanks

Sign up or log in to comment