upload model weight

README.md

@@ -1,3 +1,51 @@
 ---
 license: bsd-3-clause
 ---
+inference: false
+language:
+- en
+pipeline_tag: visual-question-answering
+library_name: transformers
+---
+
+<br>
+<br>
+
+# LoViM Model Card
+
+## Model details
+
+**Model type:**
+LoViM is an open-source Vision-Languagde model trained by initializing from InstructBLIP and alignment with Vicuna on multimodal instruction-finetuning data.
+It composes of an EVA-CLIP vision encoder, a Q-Former, a projection layer and an auto-regressive language model, based on the decoder only transformer architecture.
+
+**Model date:**
+LoViM was trained in July 2023.
+
+**Paper or resources for more information:**
+https://project page
+
+**License:**
+BSD 3-Clause License
+
+**Where to send questions or comments about the model:**
+https://github.com/
+
+## Intended use
+**Primary intended uses:**
+The primary use of LoViM is research on large multimodal models. 
+
+**Primary intended users:**
+The primary intended users of the model are researchers and hobbyists in computer vision, natural language processing, machine learning, and artificial intelligence.
+
+## Training dataset
+Pre-train data: 558K filtered image-text pairs from LAION,CC-3M, and SBU. Selected by LLaVA.
+
+Instruction-finetuning data: COCO-Caption, TextCaps, VQAv2, OKVQA, A-OKVQA, LLaVA-150K, OCR-VQA. 
+
+## Evaluation dataset
+For zero-shot evaluation on general image task, we selected Nocaps, Flickr30K, VizWiz, Visual Spaial Reasoning (VSR), IconQA, Visual Dialog, ScienceQA, MSRVTT QA, TextVQA and Hateful Memes. 
+
+For zero-shot evaluation on text-rich image OCR task, we selected ST-VQA, OCR-VQA, Text-VQA, and Doc-VQA.
+
+More detials are in our github, https://github.com/

lovim_vicuna7b.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:01915359bc5e2ae516104fba908c1c9e25bc9c7f63bda195fc8b37849d6f7622
+size 17786154252