How to use llava with huggingface
I want to use a 7b llava model with huggingface but I can’t really find any docs to use it? Any help would be great
2023-09-27T20:49:00Z
I deployed a model to SageMaker with the SageMaker deployment card HF provides. Currently this model: hxxps://huggingface.co/liuhaotian/llava-llama-2-13b-chat-lightning-preview/discussions/3However one of my concerns is that the card states'HF_TASK': 'text-generation'whereas Llava Llama is rather a text to image / image ‘question-answering’ type of model.This topic states transformers need tinkering: hxxps://discuss.huggingface.co/t/can-text-to-image-models-be-deployed-to-a-sagemaker-endpoint/20120So I still haven’t got it working. Plus I didn’t have enough quota on AWS to deploy it in a half decent box with GPU so it’ll be another question if the box can carry its weight at all. I’m surprised noone helped so far to me neither in HF model discussions, GitHub discussions (hxxps://github.com/haotian-liu/LLaVA/discussions/454) or other forums.
2023-09-28T07:49:03Z
This HuggingFace discussion says hxxps://discuss.huggingface.co/t/can-text-to-image-models-be-deployed-to-a-sagemaker-endpoint/20120 that an inference.py need to be created. I don’t know what the Llava Llama has though. I tried to look at the files of the model, but I don’t see relevant meta data about this.This StackOverflow entry hxxps://stackoverflow.com/questions/76197446/how-to-do-model-inference-on-a-multimodal-model-from-hugginface-using-sagemaker is about a serverless deployment case, but it uses a custom TextImageSerializer serializer. Shoudl I try to use something like that?My Stackoverflow entry: hxxps://stackoverflow.com/questions/77193088/how-to-perform-an-inference-on-a-llava-llama-model-deployed-to-sagemake-from-hugReddit: hxxps://www.reddit.com/r/LocalLLaMA/comments/16pzn88/how_to_parametrize_a_llava_llama_model/
2024-01-02T22:16:41Z
Check out the following blog post:RunhouseCreate a Visual Chatbot on AWS EC2 with LLaVA-1.5 and RunhouseGet started with multimodal conversational models using the open-source LLaVA-1.5 model, Hugging Face Transformers, and Runhouse.It uses HuggingFace Transformers Llava and RunhouseFor AWS SageMaker, you can check out this one: hxxps://www.run.house/blog/quickest-aws-sagemaker-deployment
2024-01-03T09:49:01Z
Hi,LLaVa and BakLLaVa are now supported natively in the Transformers libraryDocs:LLaVaCheckpoints are on the hub:llava-hf (Llava Hugging Face).
2024-01-19T17:16:43Z
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.